Nobody Can Forecast Their AI Bill

Token Prices Fell 53% and the Bill Got Harder to Predict

The LLM Token Expenditure Index, which CNBC reported on September 1, put the average price of a million tokens at $0.97. Its high was $2.07 on May 28. That is a 53% fall in three months. Two months later Apple cut some engineers' daily AI cap from $300 to $200 and said it plans to go as low as $100. Prices fell. Bills did not.

These facts do not conflict. They are one fact seen from two ends: the price list is not the bill. A company buys a finished, correct task. A vendor sells tokens. How many tokens a task takes, and how many tries until it is right, is not on the price list.

Jevons saw the same gap in coal in 1865. Cheaper steam engines did not cut coal use.

”

"It is wholly a confusion of ideas to suppose that the economical use of fuels is equivalent to a diminished consumption. The very contrary is the truth. As a rule, new modes of economy will lead to an increase of consumption according to a principle recognized in many parallel instances" — Stanley Jevons (1865), quoted in Vaclav Smil, Energy and Civilization (§ "Power densities of phytomass fuels")

Revolut shows it today. It moved coding to self-hosted GLM models at about $0.30 per million tokens, roughly a tenth of what commercial tools cost, and cut inference costs about 90% in under a year. Its AI infrastructure lead told The Register that engineers then used tokens more freely, mostly on debugging and tickets.

You Pay for Tokens and Buy Finished Work

The Stanford-led paper behind the Journal's 32% figure, with authors from Microsoft Research and Berkeley, measured what a rate card hides. Gemini 3 Flash listed 80% cheaper than GPT-5.4. Across the paper's tasks it cost 38% more. On one problem Gemini 3 Flash spent more than 60,000 thinking tokens where GPT-5.4 spent 25. On one agent task, a stronger model finished in 85 steps for about $1. The cheaper model took nearly 1,000 steps, ran up $14, and failed.

The same query on the same model varied in cost by up to 9.7 times across repeated runs. You cannot forecast that from a task count.

The paper measured cost, not correctness, so 32% understates the damage: a cheap run that fails costs its tokens, then a retry or a person.

Agents are where this matters most. SemiAnalysis estimates about 90% of Anthropic's business comes from agentic use (Financial Times, October 1), and agent runs are the workloads where tokens per task swing most. Apollo's Torsten Slok wrote on October 24 that outside tech AI has not paid off yet and that mainstream firms treat token spend as a cost that squeezes margins. Finance sees a cost center, a vendor that bills by the token and an agent whose use varies tenfold. It cannot forecast that.

One Agent, Four Days: Cost Per Correct Answer Moved 17x

MMNTM runs Clive, a Hermes agent that does a sales manager's admin work on a Medicare sales floor of about 20 reps. The HIPAA-ready build covers its design. Clive hands data questions to an analyst sub-agent, and we grade the analyst on a set of about 30 known-answer questions. We ran it four times in four days.

Costs are Hermes's estimates from its own price table, not reconciled with an invoice. Each row is one run of about 30 questions, so small gaps between rows are noise. The large ones are not.

Two things moved the number. Between November 8 and 10 the model and its price list did not change, and cost per correct answer doubled. A skill update changed what "the floor" means, and the analyst used about twice the input tokens an item: 66,500, against about 36,500 on the earlier run. A definition changed, and the bill followed.

Then we changed the model and gave the analyst its tools directly. Luna costs $0.0042 per correct answer and Sol $0.062, a 15-fold gap on the same questions. Across the four runs the range is 17 to 1. A forecast made on November 8 from the price list was wrong by November 10.

Cheaper Per Correct Answer Can Still Lose

Luna cost 15 times less per correct answer than Sol. That runs against the Stanford result, where the cheaper model cost more. It still lost on the measure that matters.

Luna got 22 of 29 right. Sol got 25. Luna's misses included summing every agent where the answer key wanted sales teams only, and a daily sales count of 588 where the board says 567. Those are wrong figures a manager might act on.

Clive's cost and value model scores each answer by how much of it a manager can use. A figure used as it came scores 1.0. A light edit scores 0.7, a rewrite 0.3, an ignored answer 0. A wrong figure that reaches a manager scores −5, because one bad number in a director's update costs more than a week of good posts earns.

Put a stand-in value of $0.50 on a correct answer, about 36 seconds of a manager's time at $50 an hour. Sol's extra $1.47 over the 29 questions buys three more correct answers, worth $1.50. Score each of the three avoided misses at −5 and it also avoids $7.50 of harm. A manager would catch some misses, so the harm is smaller, but the sign holds. Luna saves five cents a question. Its three extra misses cost far more.

”

"If the outcome of a decision in question is highly uncertain and has significant consequences, then measurements that reduce uncertainty about it have a high value." — Douglas W. Hubbard, How to Measure Anything (§ "The Proposal: It's about Decisions")

So neither the price list nor dollars per correct answer ranks the models. Value net of wrong answers does, and you get it only by deciding what a right answer is worth and what a wrong one costs. Most teams skip that step because it needs a number nobody wants to own. The −5 is a guess. Argue about it; an argued-over number beats a missing one.

The analyst runs on Sol for a second reason the table hides: on November 8, Flash stalled and fell back to another model on 17 of 29 eval items.

Agents Add a Tail, and the Tail Is Where the Surprise Lives

Averages hide the runs that break a budget. In the analyst's eval, mean input per item was 36,515 tokens and the 95th percentile was 113,466, three times higher. Averages Lie covers why agent cost has a fat tail. Clive showed it two ways.

The first live Slack request, a team's day with a chart, took 68 minutes and 23 model calls and cost $0.16. Twelve of the calls went to Gemini and carried 186,582 input tokens. A typical analyst answer costs about 3.7 cents, so the runaway cost four times the norm in dollars, and about 57 times the mean eval item in time. Two causes: a clarification step held the turn for up to an hour, and the instructions never defined "usual level," so the analyst stopped to ask. We fixed both in configuration. Neither fix has met a live request yet.

At Flash prices that tail is cents. Price the same 186,582 input tokens at the $1.50 per million that Google set for Gemini 4 Hydrogen on November 12, before caching and output, and the input alone costs $0.28. The tail's size in dollars depends on a price you do not control and that changed this week.

The second example is a blind spot. Vertex returned 429 capacity errors on 133 of 2,904 Flash calls over a week, 4.6%. When that happens Hermes falls back to the next model in its chain, which until this week was an OpenAI model. OpenAI bills outside Google's billing export and Google's budgets, so a capacity problem at one vendor shows up as spend at another, in a system your budget alerts do not watch. Our fix is blunt. Every five minutes the agent's heartbeat reports its OpenAI spend over the last 24 hours, and an alert fires above $5 a day.

The Google budget has its own flaw. It is $150 a month, set for one team's testing. Nine teams at the expected rate would pass it, so its alerts would fire with nothing wrong. Cache hit rate also moves the bill and is not yours to set: about half the analyst's input on Gemini came from cache, and about 86% of Luna's once we set a cache key.

Budget by Task, and Let the Ratio Say Where to Spend

The budget for Clive is a table, one row per task. Each row has the minutes of manager time the task replaces, the value of those minutes at quality 1.0, the model cost, and the ratio. Minutes come from the job description, not from what managers happen to do. The cost column rests on measured eval costs, with a likely range because the routines have not yet run with a model. Read it as a budget we set, not a result.

Three things follow.

Model cost is not the constraint. Quality is. At 100 to 1, doubling a task's spend pays if it lifts quality by one point in a hundred. The budget's job is to catch the runaway, not to save cents.

One wrong figure can erase a day. The morning post is worth $14 at quality 1.0. A wrong figure in it scores −5, which is −$70, about the value of everything Clive does for the team that day.

Questions on demand have the worst ratio. Routines have a fixed shape. A question is open-ended, so it carries the tail.

Each task also gets a ceiling per run. A ceiling stops the task and reports it. It is a line, not a target.

After two weeks of data, set each ceiling to three times the median cost of runs a manager used. Where quality is under 0.7, spend more before spending less. Stop any task whose ratio stays under 5 to 1 after tuning, because it is not worth a manager's attention.

The monthly figure a CFO asks for falls out of the rows: runs times expected cost is the forecast, and runs times ceiling is the worst case you will allow. Finance gets a range, and you can defend both ends.

Per-Person Caps Are the Blunt Version

Apple's cap is the same instinct with the wrong denominator. Bloomberg reported on November 4 that some engineers' daily limit went from $300 to $200, with $100 planned, and its reporter Mark Gurman added that other teams could see $25. A $300 day allows about $6,300 over 21 working days.

A cap per person bounds the damage and says nothing about where the money went. It cannot tell a $300 day that fixed an outage from one that ran a loop, and it makes the engineer ration instead of work. Revolut began with strict limits after some engineers spent $1,000 to $10,000 a month with little return, and lifted them once tokens were cheap.

A ceiling per task keeps the cap and fixes the denominator. It says what one unit of work may cost, so a person can do as many units as the work needs. A spend ceiling also belongs in the environment, not the prompt, for the reason Rogue Agents gives about every limit that matters.

Revolut also changed the forecasting problem. Its self-hosted GLM runs on 56 Nvidia B200 GPUs, so in effect the per-token bill became a capacity bill that does not move with tokens per task. That trades a variable cost for a fixed one, and it costs some capability: a router sends vision queries to Gemini or GPT Luna because GLM is text only. For steady, high-volume work that is a forecast you can sign.

Fixed-Answer Questions Do Not Need a Chat Model

Many agent calls are choices, not open questions: is this a report or an analyst question, does this message pass policy, which tool next. A chat model answers each by generating text, and generated text is the unpredictable part of the bill.

Both major vendors now sell the narrow tool. Google launched the Gemini Decisions API this week. It scores a fixed set of options and returns probabilities with a confidence from 0 to 1. A request can carry up to 1,000 questions over one shared input, billed once, at $0.10 per million input tokens and nothing for output, with end-to-end latency of 150 to 250 milliseconds. OpenAI made its Decisions API generally available on October 22 at the same $0.10 per million input tokens with no output charge, built on a reconfigured Luna model with latency of 100 to 180 milliseconds (TechCrunch).

For forecasting, what matters is the shape of the cost. With no generated output, a call cannot run away. Cost is a function of input size, which you control. The confidence score gives you a gate: act when it is high, escalate to the larger model when it is not.

Clive's routing step costs 0.3 to 3.5 cents a call today. A 2,000-token classification at $0.10 per million is $0.0002, 15 to 175 times less. That is list-price arithmetic, not a test. A decision endpoint will not do the analyst's work, only the sorting in front of it.

How to Build an AI Budget You Can Forecast

Hubbard's answer to "too expensive" fits step 1:

”

"If someone says a measurement would be too expensive, we have to ask, “Compared to what?” If a measurement that would just reduce uncertainty by half costs $50,000 but the EVPI is $5,000,000, then the measurement certainly is not “too expensive.” Indeed, it would be a bargain." — Douglas W. Hubbard, How to Measure Anything (§ "The Imperfect World: The Value of Partial Uncertainty Reduction")

Compared to a bill you cannot explain, an afternoon of tagging is cheap.

What to Do Monday

• List your three highest-volume agent tasks and tag every model call with one of them.
• Run 30 known-answer items per task and write down cost per correct answer.
• Put a number on a wrong answer that reaches a person. Use −5 if you have no better one.
• Set a ceiling on each task and on each request, and make it stop the run.
• Add a daily-spend alert at every vendor your cloud budget cannot see.
• Tell finance a range, with the ceiling as its top.

Limits of this evidence: each eval row is one run of about 30 questions, the costs are Hermes's estimates and not invoices, Clive's routines have not yet run with a model, and the $50 an hour and $0.50 an answer are stand-ins. The direction of the argument does not depend on them. The sizes do.