Tokens are getting cheaper. That part is real. The price of comparable language-model capability has fallen sharply, and a quick estimate built from published token rates can make an AI feature look almost free.
That estimate usually describes one model call. A production feature is rarely so tidy. The prompt picks up system instructions, conversation history, retrieved documents, and tool definitions. An agent may call the model several times before it finishes. Failed calls run again. Some answers need another model to check them, and others still land in a person’s review queue. Meanwhile, the cloud credit that paid for the pilot eventually runs out.
The pricing page hasn’t misled anyone. It simply answers a narrower question than the business needs to ask.
Tokens work well as a billing unit. For product economics, the more useful unit is cost per successfully completed task. If the feature resolves support cases, measure the cost of a resolved case. If it reconciles invoices, measure accepted reconciliations, including the retries and review needed to get there.
This distinction also clears up much of the confusion around AI subsidies. The listed rate, today’s invoice, and the cost the product will carry under ordinary conditions may be three different numbers.
What an LLM invoice actually counts
A token is a piece of information processed by a model. It can come from the user’s request, a system instruction, a retrieved document, a tool definition, an image, the model’s reasoning, or the answer itself.
Providers meter these categories differently. OpenAI reports input, output, cached, and reasoning tokens. Google’s usage metadata includes input, output, thoughts, cached content, and tool use. Anthropic separates regular input from cache writes, cache reads, and output. So “10,000 tokens” is an incomplete cost description. You also need to know what kind of tokens they were and whether the task succeeded. OpenAI Google Anthropic
Consider an employee asking an internal support assistant whether a customer qualifies for a refund. The employee sees one question and one answer. Behind the interface, the application might classify the request, retrieve the current policy, fetch the account, ask for a recommendation, validate the structured response, and run a final check. If the account lookup times out, part of the sequence runs twice.
One task on screen can be a small workflow on the bill.
The anatomy of one AI task
The user sees one result. The application may pay for several model calls and a failed attempt along the way.
Examples, not benchmarks
Each example uses $1 per million input tokens and $5 per million output or reasoning tokens.
Single reply
One model call · 1,000 tokens in · 300 out · Succeeds 80% of the time
$0.0031per accepted task
$0.0025 each try
RAG answer
Answer and check · 8,000 tokens in · 600 out · Succeeds 85% of the time
$0.0129per accepted task
$0.0110 each try
Agent task
Six model calls · 30,000 tokens in · 3,000 out · Succeeds 70% of the time
$0.0643per accepted task
$0.0450 each try
Bar width shows expected model cost for an accepted task. Retrieved and tool context is material passed into the model, so database calls and external API charges are still outside this chart. Human review and infrastructure appear later in the article.
The chart puts architecture into the cost calculation. Agents and retrieval may earn their extra expense, but a one-call model price still can’t describe the completed workflow.
The cheaper model can produce the more expensive task
Imagine two models extracting and validating the same document. Model A has lower input and output rates, but it needs a longer prompt and produces a longer answer. It also fails often enough that one request in five has to run again. Model B charges more per token and gets a valid result more often with less context.
Model A wins on the pricing table. Model B may still cost less for each accepted document.
A 2026 study found this kind of price reversal in 21.8% of the model-pair comparisons it tested. In each reversal, the model with the lower listed rate produced the higher realized cost. The largest gap was 28 times. The Price Reversal Phenomenon
The reasons are fairly ordinary. Tokenizers can turn the same input into different token counts. Models also differ in how much reasoning and output they generate. When the first answer is weak, the system may retry it, fall back to another model, or send it to a person. A higher-priced model that works on the first attempt can be the cheaper choice.
This is why token spend alone is a poor optimization target. Suppose a smaller model cuts the API bill by forty percent while accepted task completion falls from ninety percent to sixty percent. The saving may disappear into repeat work, customer complaints, and review time.
Set the acceptance threshold first. Then compare which configuration reaches it at the lowest total cost.
Falling prices invite larger workloads
Model capability has become much cheaper. An analysis in the American Economic Association’s Journal of Economic Perspectives estimates that the price of intelligence has fallen by roughly a thousandfold. It also finds comparable open models selling for substantially less than closed ones. The exact curve depends on the benchmark and time period, but the drop is hard to miss. American Economic Association
Company AI bills don’t have to follow the same curve. Once a call gets cheaper, teams use the extra room. They include more context, add a verification pass, evaluate more outputs, or give an agent more steps. Often that is money well spent. Better evidence can improve an answer, and a verifier can catch a mistake that would cost far more than the tokens.
Cheap capacity also makes waste easier to overlook. A full conversation history gets sent with every request because it fits. Two agents read the same material. After a tool fails, the system replays an even larger context. Every output goes to a model grader when a simple rule or a sample would do.
In practice, lower unit prices often lead teams to consume more units.
A Stanford Digital Economy Lab study shows how wide the range can become in coding agents. Agentic tasks used up to one thousand times more tokens than code reasoning and code chat. Repeated runs of the same task varied by as much as thirty times, and the run that consumed more tokens wasn’t reliably more accurate. Performance often peaked somewhere in the middle. These figures describe the coding-agent setup in the study, so they shouldn’t be pasted onto every business workflow. Still, they expose a useful property of agents: when the model chooses how many calls to make, cost becomes an outcome of the run. Stanford Digital Economy Lab
An average hides that spread. A product can have an acceptable mean cost while a handful of long, failed runs eat a large share of the budget. Look at the median and the expensive tail separately. Then inspect the failures that spent the most.
What “subsidized” can mean
“Subsidized” gets used loosely in AI pricing discussions. It can refer to arrangements that look similar on a customer invoice but have very different economics.
Cloud credits are literal subsidies. Google, for example, offers substantial credits to qualifying AI-first startups. The provider pays for usage that the startup would otherwise have to fund. That is useful support for an experiment, but the credit says nothing about whether the product can afford its inference bill once the promotion ends. Google for Startups Cloud Program
Flat subscriptions are different. A monthly plan pools light users with people who use the service heavily. One person may consume more API-equivalent usage than their subscription fee, while another barely opens the product. But API-equivalent usage isn’t the provider’s compute bill. Public API prices include margin and pricing strategy. Without internal cost data, the gap tells us that the pricing models differ. It doesn’t tell us how much money the provider lost, if any.
Competitive pricing is harder to read from the outside. A provider may accept a thin margin to win distribution, or a cloud company may discount an AI service because it earns money elsewhere. The low price might also come from a genuine serving advantage, such as better hardware utilization or caching. Customers can’t separate those explanations from the pricing page.
Customers can’t settle that debate from the outside, and they don’t need to. Their practical risk is dependence on a price that may not last. If the product only works financially while a particular promotion or unusually aggressive rate remains available, that assumption belongs in the cost model.
One task, three prices
The same month produces three answers to the question, "What did this cost?"
Example: 10,000 requests, 8,000 accepted results, and a $500 cloud credit.
Metered usage
$500
5 cents for each request
$500 in model and tool charges
Before the cloud credit
Who pays?
The product company would normally pay it.
Cash leaving today
$4,400
$0 model bill after the credit
$4,000 human review
$400 monitoring and operations
Who pays?
The provider covers the model bill. The company still pays for people and operations.
After the credit
$4,900
61 cents for each accepted task
$500 usage + $4,000 review
+ $400 monitoring and operations
Who pays?
The company pays. The cost eventually reaches the customer or the margin.
Review takes 100 hours at $40 per hour. The example leaves out product development and general company overhead. Public API prices do not reveal the provider's compute cost.
Use credits and introductory rates, then run the same cost model with both removed.
A fixed subscription can hide variable AI costs
Once a credit expires, the product company has to absorb the usage that it was covering. This matters when customers pay a fixed monthly price while the AI work underneath it remains variable.
Two customers on the same plan can have very different costs. One sends short requests that finish on the first call. Another uploads long documents, triggers several agent steps, and sends more cases to review. The subscription revenue is the same, but the work required to serve them isn’t.
Customers don’t need to see any of this as token billing. A company can keep per-seat or per-workflow pricing. Internally, it still needs to know how much accepted work each plan buys at normal, unsubsidized rates. The fixed subscription leaves the variable cost with the vendor, where a portfolio average can easily hide it.
Calculating that exposure starts with the outcome the customer actually receives.
Measure the cost of accepted work
Count the work that met the product’s acceptance condition. A generated answer doesn’t qualify merely because it reached the screen.
For a support assistant, success might mean a resolved case that wasn’t reopened later. In document processing, it could mean a validated record accepted by the downstream system. For an agent that changes data, the right state must change and the action must stay within policy.
Start with a simple calculation:
Cost per successful task = total operating cost of the workflow / accepted completed tasks
The numerator includes model tokens, external tools, retrieval infrastructure, retries, validation, monitoring, and required human review. Fixed development work can sit in a separate investment calculation. The denominator excludes tasks that had to be redone or created repair work downstream.
Take an illustrative month with 10,000 requested tasks. Model calls, tools, and retries cost $600. Monitoring adds $400. Reviewers spend 120 hours at a loaded cost of $40 per hour. Eight thousand tasks meet the acceptance condition.
The API-only calculation is six cents per request. The operating calculation is $5,800 divided by 8,000 accepted tasks, or roughly seventy-three cents for each successful task. Both numbers are mathematically correct, but only the second can be compared with the value of the finished work.
Now each optimization can be tested against the same denominator. A smaller model helps when it lowers cost per accepted task without pushing more work into review. Caching helps when it removes repeated processing and leaves the outcome intact. A more expensive model can earn its rate if it prevents enough retries.
The entire fraction matters. A smaller API line item is useful only when accepted work also becomes cheaper.
Give the task a budget before it starts
Monitoring tells you what a run spent after the fact. A budget sets the limit before the run begins.
For one response, an output cap may be enough. A multi-step workflow also needs limits on model calls, tool attempts, retries, or total cost. The agent shouldn’t get to decide for itself that one more attempt is still worth the money.
What happens at the limit depends on the job. A research tool can return partial findings and say where it stopped. A customer workflow might pass the case to a person along with the evidence already collected. For a consequential action, stopping may be safer than forcing a result.
The budget belongs in product behavior because users experience what happens when the system reaches it.
Stress-test the feature with less comfortable assumptions. Remove the promotional credit and raise the model rate. Replace the average run with the observed 95th percentile. Lower the first-pass success rate, then add the review time that the pilot team has been supplying informally. If the feature still pays for itself, the economics are getting credible. If it collapses as soon as the credit disappears, the pilot has found a financing condition.
Follow one hundred tasks, not one million tokens
You don’t need an enterprise cost program to begin. Pick one production workflow and follow one hundred real tasks from the request to the accepted result.
For each task, record the model calls, token usage, tool charges, retries, review time, and final outcome. Look at the median run, then pull out the expensive tail and read those traces one by one. Calculate today’s cost and calculate it again without credits or temporary discounts.
The exercise usually finds mundane problems: the same context sent twice, a retry policy that is too generous, or an output that looks finished but creates repair work later.
Run it again whenever the model, prompt, tools, or acceptance criteria change. The repeat measurement will show whether cheaper tokens are reducing product cost or merely making room for more consumption.