Cost per token vs cost per task: why your AI bill grows while prices fall

On 30 July 2026, OpenAI cut the price of the GPT-5.6 Luna API by 80%, from one dollar to twenty cents per million input tokens. Terra dropped 20%. The headlines wrote themselves: AI is getting cheaper, the barrier to entry is falling.

Weeks later, several companies running those models in production saw their bill go up.

That is not a contradiction or a billing error. Price per token and the cost of your operation are two different things, and only one of them shows up in your budget.

The unit your provider bills is not the unit your business consumes

Your provider charges you per token. Your business does not consume tokens: it consumes resolved tasks. A support query handled. A document classified. An invoice extracted.

While one task equalled one model request, both units moved together and the distinction was academic. If the token dropped 30%, your bill dropped 30%.

That stopped being true when applications went from making a single request to orchestrating several. An agent that queries a tool, evaluates the result, decides the next step and calls the model again does not make one request per task: it makes eight, or fifteen. And it resends the entire context on every one of them.

An example with numbers

Take a support assistant that resolves customer queries. A 1,200-token system prompt, an 800-token user query, a 400-token answer.

Single-request version:

	Tokens	Cost
Input	2,000	$0.0040
Output	400	$0.0048
Total per task		$0.0088

The same task, resolved by an agent in eight steps. Each step resends the system prompt, the original query and everything accumulated so far. Assuming each step adds around 600 tokens of history:

Step	Input tokens
1	2,000
2	2,600
3	3,200
4	3,800
5	4,400
6	5,000
7	5,600
8	6,200
Total	32,800

With 250 output tokens per step, that is 2,000 output tokens. At Terra pricing ($2 input and $12 output per million), the cost per task rises to $0.0896.

Ten times more to resolve exactly the same thing.

Now apply the 80% cut. If your single-request task now costs a fifth of what it did, congratulations. But if in that same quarter you migrated to an agent architecture, you multiplied consumption per task by ten. The price fell 80% and your bill doubled.

The three multipliers that never appear on the pricing page

Resent context. This is the most expensive and the most invisible. In a twenty-turn conversation, the first message has been billed twenty times. Not because your application is badly written, but because that is how chat APIs work: they have no memory, and the history travels with every request.

Tool definitions. An agent with twelve available tools sends their full schemas on every request, whether it uses one or none. Those are billable input tokens on each of the eight steps.

Retries and dead ends. An agent that explores one path, realises it leads nowhere and tries another has paid for both. Your provider’s usage report does not distinguish them: it all shows up as consumption, with no label saying whether it accomplished anything.

How to measure cost per task

The bad news is that your provider’s dashboard will not give it to you. It gives you tokens by model and by day. Getting to cost per task requires something they do not have: knowing where a task starts and ends in your business.

What you can do without complex instrumentation:

Correlate against a business unit. Divide monthly model spend by the number of queries resolved, documents processed, or whatever your unit is. It is a crude average and it works anyway: if that number climbs month over month while the token price falls, you have your answer.

Segment by task type. Almost always a minority of task types accounts for the majority of the spend. Without that segmentation you will be optimising the wrong model.

Count requests per task. This is the most predictive indicator we know of. If it goes from three to nine, your cost is going up whether or not you change model or provider.

What to do about it

Before negotiating prices or switching providers, work out whether your problem is a rate problem or an architecture problem. It is usually the second, and it is attacked differently.

Prompt caching comes first, because it is the cheapest to apply: the system prompt you resend eight times can be billed at a substantial discount if your provider offers caching and your code takes advantage of it. In the example above, those repeated 1,200 tokens are nearly a third of all input.

Trim the history rather than dragging it along whole. Very few tasks need all twenty turns.

And route by complexity: reserve the expensive model for the tasks that genuinely need it. That is where the most saving is, and where the most care is needed, because it is also where it is easiest to degrade quality without noticing. Which is why quality should be measured before you touch anything, not after.

An honest caveat

The numbers in this article are a worked example built to illustrate the mechanism, not a measurement of your case. The real multiplier depends on how many requests your application makes per task, how much context it carries, and what share of tasks goes down the expensive path. It might be three times. It might be thirty.

And prices move fast: the ones here are from August 2026 and should be checked against each provider’s official page before you budget with them.

What does not change is the mechanism. As long as you measure in tokens, you will keep failing to understand why your bill does what it does.

The complete AI cost cluster

This guide is the centre; each piece develops one part of the problem.

Understanding the cost

Cutting it

Choosing the architecture

Keeping it under control

How to track AI spend: which metrics to follow

At e-ficient we measure cost per task for companies already running AI in production. The initial audit is free if you start a plan with a three-month commitment and returns a diagnosis within 72 hours.