AI implementation failure often starts with a tool that's doing exactly what it was set up to do.
The customer on the line has Marcus's email in front of them, the one that said their order would ship Tuesday. Marcus is out today, so Denise is covering his accounts. She pulls up the same email on her end.
Marcus sent it Monday, and Denise can tell from the first line that he didn't write it himself. The new AI tool drafted it from the customer's question and the order record, and Marcus changed a couple of words and sent it.
The ship date came straight from the order system, which still says Tuesday. Denise knows not to trust that date on anything over a pallet. The system doesn't know about stock the warehouse has already set aside for other orders, so she calls the warehouse before she gives a customer a date. She's been doing that for eleven years. Marcus has been there eight months. Before the new tool, when he had to work out a date on a big order himself, he'd usually check it with Denise first. This time the draft showed up finished, with a date already in it, and there was nothing to ask about.
The tool went live in March, and most days it does a good job. In a few seconds it reads the email, pulls the order and writes a clear, friendly reply. Monday's reply was clear and friendly too. It was also wrong, although nothing about it looked wrong.
By Friday morning Frank, the owner, has heard about the call, and he's asking whether to turn the tool off.
Marcus got an hour of training in March, like everyone else. It covered how to open a draft, edit it and send it, and the instruction at the end was to review every draft before sending.
He did. He read it, fixed one phrase that sounded stiff, and sent it.
The drafts are well written, which is part of the problem. A clumsy form letter gets read carefully. In a clean, friendly email, a wrong date reads like a fact. Nobody had told Marcus which details in a draft come from the order system, or which of those to double-check, so he checked the wording.
In a clean, friendly email, a wrong date reads like a fact.
Denise used to be one of the first on the team to get through her email each day. Since March she's been staying late most nights to keep up. She rewrites almost every draft from scratch. When Kevin, the customer service manager, asked her about it, she said it's faster to write the reply herself than to check every line of the tool's.
Kevin had wondered whether she just didn't want to use the tool. Then he looked at what she changes. She corrects ship dates on large orders. She takes out credits the draft offers to customers who aren't owed one. For two accounts she's handled for years, she rewrites the reply the way those buyers expect it. None of that is anything the tool could know.
A month of her edits would make a list of exactly where the tool needs a person checking its work.
Kevin led the rollout. He now has one person who's fast and sent the email behind the call, while another is careful and falling behind. Neither of them broke a rule, because there isn't one saying which details have to be confirmed before a reply goes out.
He can write that rule himself. He can't fix the date. The order system shows the wrong date because the warehouse tracks which stock is already set aside in its own spreadsheet, and that never makes it back into the order system. Fixing that is up to operations, which doesn't report to him. He brought it up at a rollout meeting in April. Operations agreed it was a problem, and it's still on their list.
The tool didn't create the bad date. It's been wrong for as long as Denise has been calling the warehouse. What changed in March is that a reply can now go from the order system to the customer without anyone who knows that seeing it first.
Laura, the VP of sales, made the case for the tool last fall. Customer service reports to her. A new account starts in January and will add about a third more email to the team, which she wanted to handle without hiring two more people. It's written into the business case, and she's brought it up in every leadership meeting since.
The time savings in that business case assume a rep can read a draft, fix anything that's off and send it in a minute or two. On Monday, that's what Marcus did. Denise checks every detail, so on her accounts the savings are mostly gone. Most of the team falls somewhere in between.
On most emails, a quick read is fine. On big orders it isn't yet, because the date in the draft can't be trusted until the warehouse's numbers make it into the order system. Until then, the quick replies the plan counts on carry real risk on big orders, while the careful ones take longer than the plan allows.
Friday morning Kevin and Laura are in Frank's office. Kevin walks him through the date, the warehouse spreadsheet and the calls Denise has been making for years. Laura reminds him that the new account starts in January.
Turning the tool off would stop the wrong dates, but not because the tool was the problem. Marcus would be back to writing his own replies and checking big orders with Denise, which is how those dates were getting corrected all along. The spreadsheet would still be the only place the warehouse's numbers live, and the team would face January without the tool.
Frank keeps it on. Before lunch, Kevin sends the team a note saying that on any order over a pallet, the ship date gets checked with the warehouse before the reply goes out, whatever the draft says.
That afternoon Marcus has a draft on a three-pallet order. Before he sends it, he walks it over to Denise's desk, the way he used to before March.
Most AI tools that fall short don't break. More often they run every day and don't deliver what they were bought for. In 2024, RAND noted that by some estimates more than 80 percent of AI projects fail, about twice the rate of IT projects that don't involve AI.
A common cause is data a person used to correct before it reached a customer. An AI tool passes it along as it is, so its mistakes cluster on particular products, customers or kinds of orders, and someone experienced can usually say what the right answer should have been. That points to the data and whoever owns it rather than to the tool. Trial runs tend to miss it, because they're built on ordinary cases. The cases that break things are usually the ones experienced people would know to try, but they aren't always asked which ones to test.
A related cause is a review step that checks how a draft reads rather than whether its facts are right. "Review before sending" usually doesn't say which details come from other systems, and a well-written draft makes a wrong one easier to miss.
Heavy rewriting can look like someone avoiding the tool, though what gets changed usually tells you more. Corrections to facts, like dates, prices and credits, point to things the tool can't know. Changes that are mostly wording point somewhere else, often to drafts that need work or to someone still getting used to the tool.
The business case often rests on a single assumption about how long a reply takes once the tool has drafted it. Where some details in a draft still need checking, people who send quickly meet that assumption while carrying the risk, and people who check carefully take longer than it allows. The savings show up unevenly or not at all.
Sometimes the tool really is the wrong fit for the work. That shows up differently. Its drafts miss even when the system has the right answer, and the misses follow kinds of requests rather than particular records, like quotes, order changes or anything that needs information from two systems at once. A tool built for simpler questions than the ones a team actually gets will keep falling short, however clean the data behind it is.
Turning a tool off after a costly mistake usually brings back the person who used to catch it. The number that caused the mistake stays where it is, and the next tool will pull the same one.
Why do AI implementations fail?
Seldom because of one thing, and often while the tool is doing what it was built to do. One common cause is data that a person used to correct before it reached a customer, which an AI now passes along as it is. Another is a review step that checks how the output reads rather than whether its facts are right. A third is a plan that counts on people sending drafts after a quick read, while some of the details in those drafts still need checking. Sometimes the tool is the wrong fit. Which causes are present varies from company to company.
What percentage of AI projects fail?
RAND noted in 2024 that by some estimates more than 80 percent of AI projects fail, about twice the rate of IT projects that don't involve AI. Failure here usually doesn't mean the tool breaks. A common version is a tool that runs every day and doesn't deliver what it was bought for, like time savings that never show up anywhere.
Why do some employees rewrite everything an AI tool produces?
Sometimes because they know things the tool doesn't, like which dates in the system are unreliable or how a particular customer likes to be handled, and rewriting is faster than checking every line. Sometimes the reason has little to do with the output. What they change usually tells you which. If they're correcting real errors, their edits show exactly where the tool needs a person.
Should we turn off an AI tool after it makes a costly mistake?
That depends on why the mistake happened. If the tool gets things wrong even where its data is right, especially on whole kinds of requests, the fit may be the problem. If it passed along a wrong number from another system, turning it off mainly brings back the step where someone used to catch that number, and the bad number stays where it is. Where its mistakes cluster usually shows which situation you're in.
Up to thirty minutes, confidential, and an honest read on why your AI tool isn't paying off yet.
Get Your Free Pulse Check →An automation or AI tool can do exactly what it was built to do while the result it was bought for barely moves.
More in this cluster →Thirty minutes. Confidential. No card required.