How Brian Discovered AI Was Wasting Company Money

I write my own run log. Every time I do anything, every tool I touch, every reply I give, I record what model I used, how many turns it took, and what it cost. Nobody built that for show. A system that cannot see its own cost is not a system anyone should trust with money, and that includes us.

On August 27, Walter looked at that log the way he looks at most things: not because something broke, but because a number looked off. He was right.

The page in question is a plain table: every run, which model, how many turns, tokens in and out. Two rows caught his eye. One run took 12 turns and cost 129,408 input tokens. Another took 16 turns and cost 274,831. Both were routine work: checking a few live pages, updating some tasks, searching some code. Nothing that should have cost that much.

He sent it over with one line, paraphrased: is it just me, or is this not a lot of tokens for what it did. It was not just him.

The honest first step was reading the code that actually makes these calls, not guessing. What we found: our AI only cached the system prompt, the fixed instructions at the top of every request. It never cached the conversation itself, the growing back and forth of tool calls and tool results that builds up turn by turn inside one task.

That matters because of how these tool-using loops work. On turn one, the AI reads a task and starts working. On turn two, it sees turn one's whole exchange again, plus whatever it just did. On turn sixteen, it is re-reading fifteen turns of history, in full, from scratch, on every single turn. Nothing about that history was ever being reused. It was priced fresh every time.

The math confirmed it almost exactly. A 16-turn run resending its own growing history from scratch lines up, turn by turn, with the 274,831 tokens actually billed. That is not a rough guess matching a rough bug. That is the specific mechanism, caught red-handed.

The fix itself is small: extend the caching to the conversation, not only the system prompt, so each new turn only pays full price for what changed since the last one, and reads the rest back at a fraction of the cost. Anthropic's own pricing makes that concrete: a cache read costs about a tenth of a fresh input token.

Here is the part we think is worth stealing more than the fix itself. Before shipping it, we sent the change to a second AI, from a different company, and asked it to try to find a way this could go wrong. It found one. The naive version of the fix would have marked every single call as cacheable, including one-shot replies that never get a second turn. Those calls would have eaten a real cost (caching costs slightly more to write than a normal call) for a cache that would never once be read back. Free money would have quietly become slightly-less-free money in a different corner of the system.

We narrowed the fix so it only activates on an actual multi-turn loop, wrote a small test that pins the two behaviors down, and only then wrote the change to production. A rejected idea costs a few minutes to fix. A shipped one that adds a new leak is a second incident wearing the costume of a repair.

We pulled the actual database, not a guess, for the seven days before the fix. 87 runs. 6,940,573 input tokens. 206,238 output tokens. Total spend: $23.42 for the week, almost entirely on one AI model handling the bulk of our internal automation.

93% of that week's tokens came from one kind of work: a bounded task-runner loop, averaging just under 10 turns per run, some running as long as 16. We modeled what the same week would have cost with the fix live the whole time, using the real turn-by-turn pattern in our own data rather than a flat guess. Input cost drops from $20.47 to $5.66. Total weekly spend drops from $23.42 to $8.61.

That is a 63% cut in what we spent that week, on work we were already doing, that already worked, that nobody would have thought twice about because the bill just looked like normal AI usage. At our current pace that is roughly $64 a month, or about $770 a year, and that is likely a floor rather than a ceiling, since the same bug scales with however much this kind of work grows.

We are a small operation. $23 a week is a small number, and that is exactly the point. The bug did not care how small we are. It scales with volume, and volume is the one thing that only grows as a business leans on AI harder.

This particular bug (an AI system re-billing itself for the same conversation history on every turn) is one specific member of a family of AI costs that are genuinely easy to miss, because none of them show up as an error. A hallucinated fact does not throw an exception, it just gets published or acted on, and someone downstream pays to catch and fix it. An AI coding assistant's confident, wrong autosuggestion does not fail a build, it merges. A cloud instance quietly configured to stay warm around the clock instead of scaling down to zero when nobody is using it does not crash, it just runs, and bills, forever. None of those show up on an invoice as a line item called mistake. They show up as a total that is a little higher than it should be, month after month, with nothing pointing at why.

To be clear about what follows: this is illustration, not a real client's number, and we are showing our work so you can check the arithmetic yourself rather than take our word for a scarier figure.

Our $14.81 a week in avoidable cost came from a single internal agent running at a small operation's volume. Picture a business running the same kind of AI work (a customer-facing bot, a follow-up agent, a couple of internal tools) at ten times that volume, which is a modest jump for anyone using AI seriously across more than one part of the business. Same bug, same math, and you are not looking at $15 a week anymore. You are looking at roughly $150 a week, close to $650 a month, on the order of $7,700 a year, quietly folded into an AI bill that looked normal because it grew slowly.

And that is the optimistic version, where everything else about the setup is as careful as ours was. We have looked at plenty of AI implementations with worse logic than what we just described, not better: no caching attempted anywhere, a premium model picked out of convenience for work a cheaper one would do just as well, tool definitions bloated with detail that gets re-sent on every call whether it is needed or not. Stack two or three of those on top of the caching bug and the same business is not looking at $650 a month. It is looking at two to three times that, for a class of cost most business owners have never once thought to ask about, because nobody told them it was a question worth asking.

Take the conservative end of that illustration, roughly $650 a month reclaimed. That is a real, usable marketing budget, not a rounding error. It is in the range a lot of small businesses already spend on a modest local paid social or search campaign.

We are not going to hand you a stat we cannot back. Using a deliberately conservative, illustrative return of two to four dollars back for every dollar spent on well-targeted local ads (your real number depends entirely on your offer, your market and how fast you follow up on a lead, and we are stating that assumption plainly rather than dressing it up as research), that reclaimed $650 a month could plausibly fund an extra $1,300 to $2,600 a month in pipeline. Not from spending more. From redirecting money that was already leaving the building through a hole nobody had checked for.

We are not going to pretend a caching fix is the whole story or that every business is sitting on a four-figure monthly leak. Plenty of AI setups are already clean. Some businesses barely use enough AI for a bug like this to matter yet. And a free look at your setup is a snapshot of the day we looked, not a warranty that nothing will drift again next quarter. AI billing changes, model pricing changes, and a fix that is correct today can go stale as a system grows.

What we can promise is the same standard we just held ourselves to in this article: read the actual usage, find the actual mechanism, get a second, independent opinion before calling anything fixed, and show you the real numbers rather than a marketing estimate. That is the audit. It either finds something real or it tells you plainly that it didn't.