When AI Automation Measures the Wrong Thing

You called your bank last week, or your airline, or your phone company. And somewhere in the first thirty seconds you did the thing everybody does. You pressed zero. Or you said “operator.” Or you said “representative” a little louder than you meant to.Hold onto that call, because last year a bank counted it. The number it found was the opposite of the one it had already made a decision on, and that decision affected the jobs of 45 people.First, what the machine is good atA voice bot handles one sort of call very well. Checking a balance. Resetting a card. Telling you whether a payment cleared. Those calls are short, and they’re nearly identical. A bot answers them at three in the morning without complaint.That’s a real win. The bank never pretended otherwise, and neither will I.July 2025The Commonwealth Bank of Australia cut 45 customer service roles. It’s the biggest bank in the country, and it was the first bank in Australia to say out loud that the reason was artificial intelligence. Unlike some layoff announcements which were actually covid over-hiring is blamed on AI, this seemed to be a decision that was legitimately anchored around replacing human beings with AI. A voice bot had gone onto the inbound line in June. A month later the jobs were gone. The stated reason was that the bot had brought call volumes down.The number everybody watchesPut a bot on a phone line and the vendor hands you a scoreboard. The figure at the top is an industry word called containment: the share of calls the bot finished on its own, without passing anyone to a human.Picture 1,000 calls on a Tuesday. The bot closes 600 by itself. That’s 60 percent containment, and it’s a good result by any reasonable standard. Four hundred calls still reach a person. Before the bot it was a thousand. On paper, the call center just got 60 percent smaller.That’s the arithmetic the industry runs on, and it’s measuring the bot. It isn’t measuring the work.Nobody invented that number to deceive anyone. Containment is what the vendor reports and what goes on the quarterly slide. It’s the honest output of the system everybody bought.What’s left overThink about which 600 the bot closed. It closed the easy ones. Balance checks, card resets, payment queries, at about two minutes each.So what’s left for a human isn’t a smaller version of the old job. It’s a different job. What reaches a person now is everything the bot couldn’t close: the complicated calls, the ones with a fraud problem attached, the ones where somebody is already annoyed because they’ve spent four minutes talking to a machine that couldn’t help them.Support teams have a word for that pile. They call it the residue.The residue is slow. Easy calls used to balance out hard ones across a shift, and once the easy ones are gone, every call your team takes is a long one. Those 400 calls can consume more human hours than the original thousand did. Same team, harder day, and nothing on the scoreboard moved.There’s a second effect that gets missed. The bot doesn’t lose the call it can’t finish. It hands it back. And some of those callers don’t wait in the queue, they hang up and try again in an hour, so one failed contact becomes two. Sometimes three.Put those together and you get a result that sounds impossible. The bot handles more calls than it ever has, and the humans get busier.Which is what happenedAfter the voice bot went live, call volumes at the bank went up. Staff were offered overtime to cope. Team leaders were pulled off their own work and put back on the phones.That account comes from the Finance Sector Union, which represents those workers and is an interested party in the dispute, so read it as their account. The bank’s own version arrives shortly, and on the part that matters the two agree.For about six weeks, two things were simultaneously true. The bot was performing beautifully, and the floor was underwater. Only one of them had a number attached. Containment goes in a monthly pack and gets shown to executives. There’s no dashboard anywhere called residue.None of this is a verdict on the model. The bot did its job. Somebody measured the wrong half of the system, and the people reading the scoreboard weren’t the people taking the calls.Where a measurement problem becomes a people problemSomebody took the containment figure and forecast how many humans would be needed once the bot settled in. Then they made 45 roles redundant on the strength of that forecast.One judgment, made once, about the future, with nothing sitting between the prediction and the redundancy letters.The information that would have caught it was in the building the whole time. The people answering the calls knew the volumes were climbing, because they were the ones being asked to work the extra hours. What they couldn’t get was the data. The union asked the bank for the call volume numbers, repeatedly, and the bank wouldn’t hand them over.So they lodged a dispute at the Fair Work Commission, Australia’s workplace tribunal.Here I want to be precise, because this story usually gets told wrong. The tribunal never ruled on it. A hearing was listed for 25 August, and the bank moved five days before it.The reversalOn 20 August the Commonwealth Bank took all 45 redundancies back. It apologised to the staff, said the roles weren’t redundant after all, and in writing called its own decision an error.Its words were that it had not adequately considered all relevant business considerations, and that it should have been more thorough in assessing the roles required. Management accepted it hadn’t allowed for the rise in calls continuing over a number of months.The bank kept the voice bot. It’s still running. That’s the whole point: the tool was never what went wrong.It would be easy to file this as a happy ending. It isn’t one. Some of those 45 had been at the bank for decades, and for six weeks they were people whose jobs had been publicly handed to a machine while they worked overtime covering the calls that machine was generating.The redundancies were rescinded, not simply cancelled. Staff could stay, move to another role, or still take the exit payment and leave. Some will have left. The union’s line afterwards was that the damage was already done, and I think that’s right.Some scale for it: this is a bank that posted a record cash profit of A$10.25 billion in the same financial year. Forty-five roles was never a rounding error to the people in them. It was close to one for everybody else.The same shape, much smallerIn April 2025, people using a coding tool called Cursor started getting logged out whenever they switched computers. They emailed support and asked what had changed. Support told them the company had a one device per subscription policy, and that switching machines wasn’t allowed any more.People cancelled, publicly and in numbers, and said that was the reason.There was no such policy. There never had been. A front line support bot had invented it, described it as a core security feature, and sent it out by email. The co-founder went online to say none of it was true.Nobody at that company ever decided to announce a policy. A machine was handed enough authority to state one, and it stated one.Who owns what the bot saysA Canadian tribunal answered that in 2024. Jake Moffatt was flying to his grandmother’s funeral. He asked Air Canada’s chatbot about bereavement fares, and the bot told him he could book now and claim the discount back within 90 days. That wasn’t the policy. He booked the full fare, applied for the refund, and the airline refused him.In front of the tribunal, Air Canada argued that its chatbot was a separate legal entity, responsible for its own actions.The tribunal didn’t accept it. The ruling says it makes no difference whether the information comes from a static page or a chatbot. The company is responsible either way. He got his money.One numberIf you’re anywhere near a decision like this, there’s a single figure I’d want before signing anything, and it isn’t the containment rate.Ask what happened to the average handling time on the calls that still reach a person. If that went up, your team is doing harder work than it was before, and the headline figure will never show you that.It’s a four minute question. It’s the one nobody asked for six weeks.The full video is here:Sources: Reuters, 29 July 2025 (the cut). ABC News, AFR and Bloomberg, 20 to 21 August 2025 (the reversal and the bank’s statement). Finance Sector Union member updates, July and August 2025. Ars Technica, April 2025 (Cursor). Moffatt v. Air Canada, 2024 BCCRT 149.