The real cost of AI: tokens, models and who pays the bill

Why AI bills keep rising as token prices fall, how caching and model choice move the cost curve, and when a smaller or local model is the better call.

Every few months a lab cuts its prices and someone in the room concludes that AI is getting cheaper. Then the quarterly bill arrives and it is bigger than the last one.

Both things are true. Token prices fall, and spend still rises, because cheaper work gets used for more things: agents that loop for longer, tasks nobody would have funded a year ago, and routing layers that move the cost somewhere it is harder to see. The cost of AI is set by engineering choices (caching, model choice, how much scaffolding you send with every call) far more than by the price list.

The posts below take the questions CTOs and finance teams end up asking: why the bill is rising, who should own it, where a smaller model is enough, and what running models locally really costs.

The posts, in reading order

  1. Cheaper tokens, bigger bill
    A team ran an AI cost-routing layer for four months, then shut it off. The savings had not vanished. They had moved to a budget nobody was measuring.
  2. Same model, ten times the bill
    A cached token costs a tenth of a fresh one, and coding agents hit cache most of the time. Your AI cost curve is an engineering choice, and most teams make it by accident.
  3. The engineer who picks the model never sees the bill
    Meta and Microsoft are reportedly cutting internal Claude use. The cut is a symptom: AI budgets run on usage counts until finance steps in, and a cap is a blunt tool.
  4. Three frontier labs shipped this week. Only one cut its price.
    Anthropic, OpenAI, and xAI all updated their flagship models within 48 hours. The benchmark charts are the least useful thing in any of the three announcements.
  5. Most teams still ration AI that is cheap enough to waste
    A developer says a near-free model changed what he bothers to attempt. For leaders, the useful question is which work never got funded, and who checks the results.
  6. Your best AI use case may not need a frontier model
    A ten-year-old neural network beat GPT-6 and Claude at spotting skin cancer. What the result says about where a general-purpose model belongs on your AI roadmap.
  7. 10 patterns where Jev, not your frontier model, should decide
    TypeSafe's Jev returns a typed agent decision in under 500 milliseconds for a fraction of a cent. Here are ten places in an agent stack where that trade actually wins.
  8. The real cost of running AI locally for coding
    Serious developers are dropping paid AI subscriptions for local models and reporting it actually works. The 'it's free' pitch is real - but it obscures a different set of costs.
  9. Local models stopped being toys
    Qwen 3.6 27B is running at 30 tokens per second on consumer hardware and doing real development work. That shifts the question from whether local AI is viable to when it is the right call.
  10. Someone is subsidizing your AI compute
    The AI infrastructure boom is partly lending to itself. That changes the cost curve you're planning around, and most companies building on it haven't noticed.
  11. The scariest cloud bill is the one that looks right
    AWS just showed customers trillion-dollar bills. The real exposure is the small metering error you can't see, on a consumption cost you can neither audit nor cap.

Common questions

Why is our LLM bill going up when token prices keep falling?

Because cheaper tokens get spent on more work. Agents run longer loops, teams try tasks they would never have funded, and routing layers move savings to budgets nobody measures. Price per token is the wrong number to watch; cost per finished task is the useful one.

How do you reduce AI inference costs without hurting quality?

Start with prompt caching, because a cached token costs about a tenth of a fresh one and coding agents hit cache most of the time. Then match the model to the task: many classification and routing decisions inside an agent do not need a frontier model. Finally, make the person who picks the model see the bill.

When should a company use a small or local model instead of a frontier model?

When the task is narrow, well defined and easy to check, such as a typed decision inside an agent or a specialised classifier. A local model can also make sense for coding, but the subscription you save is replaced by hardware, setup and maintenance costs that need counting too.

All guides · Book an AI advisory hour