Skip to content

LLM API pricing explained: input, output, cached and reasoning tokens

Published 10 min read

LLM APIs charge per token, quoted in dollars per million tokens, with separate rates for input (what you send) and output (what the model generates); output is typically several times more expensive. Repeated prompt prefixes can be billed at a cheaper cached-input rate, reasoning tokens are billed as output even though you don’t see them, and images, audio and embeddings use their own units. The same open-weight model can cost very different amounts at different providers.

A model’s price sheet looks simple: two numbers per model. In practice your bill depends on at least six variables, and the two headline numbers explain less than half of the variation between teams running similar workloads. This guide goes through each pricing dimension the way it actually shows up in a bill, then works through an example with round, hypothetical prices so the arithmetic is easy to follow.

What does "per million tokens" mean?

A token is the unit a model reads and writes: a chunk of text that is often a word or part of a word. In English, a rough rule is that 1,000 tokens is about 750 words, but it varies by language, by code vs prose, and by each model’s tokenizer. Tokens explained covers the conversion in detail.

Prices are quoted per million tokens (sometimes written MTok or 1M tokens) because individual requests are tiny in dollar terms. To get the cost of one request, divide the token count by one million and multiply by the rate:

text
cost = (input_tokens  / 1,000,000) * input_price
     + (cached_tokens / 1,000,000) * cached_input_price
     + (output_tokens / 1,000,000) * output_price   # includes reasoning tokens

Token counts come back in the usage object of every response, so you never have to estimate after the fact. Estimating before the fact (for capacity planning) is where the tokenizer details matter.

Why are output tokens more expensive than input tokens?

Input tokens are processed in parallel in a single forward pass, which uses GPUs efficiently. Output tokens are generated one at a time, each needing its own pass over the model, and each one holds memory for the growing context while it waits. Generating a token costs the provider far more compute time than reading one, and the price sheet reflects that: output rates are typically several times the input rate for the same model.

The practical consequence is that the shape of your workload matters as much as the model. A RAG system that reads 20,000 tokens and answers in 200 is input-dominated. A code generator that reads 1,000 tokens and writes 4,000 is output-dominated. Two teams on the same model can see completely different cost profiles, and the right optimization differs: trim context for the first, shorten or structure output for the second. How to reduce LLM API costs maps each lever to the workload shape.

What is cached input pricing?

Many providers offer prompt caching: when a request starts with the same long prefix as a recent request (same system prompt, same tool definitions, same document), the provider reuses its internal computation for that prefix and bills those tokens at a discounted cached-input rate. Some providers apply it automatically; others require you to mark cache breakpoints, and some charge a small premium to write the cache. Cache entries expire after a short idle time.

Cached input only applies to an identical prefix. Put stable content first and anything that varies per request last, or the cache misses on every call. This is different from response caching, where an identical request returns a stored answer and no model call is billed at all. LLM caching explained compares the two.

How are reasoning tokens billed?

Reasoning models generate internal reasoning before the visible answer. Those tokens are billed at the output rate, and they count toward the request’s output budget, even when the API returns only a summary of the reasoning or nothing at all. On a hard problem the reasoning can be many times longer than the answer, so a reasoning model’s effective price per useful answer can be far higher than its per-token rate suggests.

Two things follow. First, many reasoning models expose an effort or budget setting; lowering it on easy tasks is a direct cost control. Second, max_tokens is a ceiling, not a price: you pay for tokens actually generated. A ceiling set too low on a reasoning model returns finish_reason: "length" with a truncated or empty answer after the reasoning has already been paid for. On Smouter, that case adds an x-smouter-warning header naming the budget and how much reasoning consumed.

How are images, audio and embeddings priced?

Not everything is billed in text tokens. The common units:

Units differ by provider; always check the unit before comparing two prices.
KindCommon billing unitWhat drives the cost
Chat and text generationPer million input and output tokensPrompt length, answer length, reasoning, caching
EmbeddingsPer million input tokens (no output)Volume of text indexed and re-indexed
Image generationPer image, or per token on some modelsNumber of images, and on some providers size and quality
Image input (vision)Converted to input tokensImage resolution and count
Text-to-speechPer character of input textLength of text spoken
Speech-to-textPer second or minute of audioDuration of the uploaded audio

On Smouter, GET /v1/models returns a kind and a pricing_unit for every row: mtok for chat and embeddings, image for per-image models, char for text-to-speech. Images (generate and edit) bill per image, TTS per character, and STT per second of audio read from the uploaded file. If you build tooling on the catalog, branch on pricing_unit, not on kind; the docs explain why.

How do batch discounts work?

Batch APIs let you submit many requests at once and accept results asynchronously, usually within 24 hours. Because the provider can schedule them into idle capacity, batch requests are priced below the synchronous rate. The catch is latency and plumbing: you upload a file, poll for completion and reconcile results by a custom id. Batch suits evaluations, backfills and bulk enrichment, not anything a user is waiting on. The LLM batch API guide covers the workflow.

Smouter’s Batch API runs chat-completion lines with a 24-hour window and credits a discount on completed jobs. The rate shown in the dashboard is a ceiling, and a cancelled job earns no credit; see batch processing.

What are rate limits and usage tiers?

Rate limits are not a price, but they shape cost. Providers cap requests per minute and tokens per minute, often with tiers that rise as your account’s spend history grows. A new account may be throttled well below what a production workload needs, which pushes teams toward a more expensive model with looser limits, or toward a second provider. A limited request returns HTTP 429 with a retry-after header; naive retry loops that ignore it waste time and, if they retry after partial generation, money. LLM API fallbacks covers handling 429s properly.

On Smouter you can set your own tokens-per-minute cap per key and per organization; a rejection returns 429 tokens_per_minute_exceeded with retry-after. See rate limits and TPM caps.

Why does the same model cost different amounts at different providers?

Closed models are generally sold by their maker and a few licensed clouds. Open-weight models are different: anyone with GPUs can serve the published weights, and many inference providers do. Their prices for the identical model differ because their costs differ: hardware generation, utilization, batching strategy, region, and whether they serve full precision or a quantized variant. Some list low prices to win traffic and then fail to serve reliably under load.

That spread is real money, but chasing the lowest listed price by hand is risky. A cheaper provider is only cheaper if it actually serves the same weights at acceptable latency and error rates. Smouter’s value lane shops each request across a pool of providers serving the same weights, fingerprints nodes against reference weights, benchmarks them continuously, and demotes nodes that advertise low and fail to serve. The site states prices of up to 50% under OpenRouter on the same model; live per-model prices for both lanes are on the models page, for example Kimi K3 and GLM-5. The open-weight models API guide compares the families.

Worked example: what does one request actually cost?

The prices below are hypothetical round numbers chosen to make the arithmetic readable. They are not any provider’s real prices. Take a support assistant on a reasoning model priced at $1.00 per million input tokens, $0.10 per million cached input tokens, and $4.00 per million output tokens. Each request sends 10,000 input tokens, of which 8,000 are a stable system prompt and tool definitions. The model reasons for 1,500 tokens and answers in 500.

Hypothetical prices for illustration only.
Line itemTokensHypothetical rate per 1MCost without cachingCost with prefix caching
Stable prefix (input)8,000$1.00 full / $0.10 cached$0.0080$0.0008
Variable input2,000$1.00$0.0020$0.0020
Reasoning (billed as output)1,500$4.00$0.0060$0.0060
Visible answer (output)500$4.00$0.0020$0.0020
Total per request12,000$0.0180$0.0108
Per 100,000 requests$1,800$1,080

Three observations. The visible answer is the smallest line item; reasoning costs three times as much. Prefix caching cuts the total by 40% in this example without touching quality, purely by ordering the prompt correctly. And on this shape, lowering reasoning effort would save more than shortening the answer. Your real numbers will differ, but the method transfers: break a representative request into these lines, and the biggest line tells you where to work.

How do you read Smouter’s receipt headers?

Every Smouter response carries receipt headers when their values are known, so you can record the actual charge per call:

http
x-smouter-lane: value
x-smouter-cache: miss
x-smouter-charge: $0.0004
x-smouter-baseline-openrouter: $0.0031
x-smouter-saved: $0.0027
  • x-smouter-charge is what this request cost you.
  • x-smouter-baseline-openrouter is what the same request would have cost at the OpenRouter reference price.
  • x-smouter-saved is the difference between the two.
  • x-smouter-lane says whether the request ran on the value lane (sourcing pool) or verified lane (model maker’s own endpoint). See value vs verified lanes.
  • x-smouter-cache says whether the response was served from cache.
  • When they apply, x-smouter-prefix-saved and x-smouter-tokens-saved report additional savings that applied to the request, and x-smouter-warning flags a completion that hit its max_tokens ceiling.

The values above are the example from the docs, not a quote. For a streaming response, the final charge settles after the stream finishes and appears in Usage; the understand your usage article explains the per-request view, and credits and top-ups covers the prepaid ledger.

Frequently asked questions

How is LLM API pricing calculated?

Most text models charge per token at separate input and output rates, quoted per million tokens. Multiply input tokens by the input rate, output tokens (including any reasoning tokens) by the output rate, and add any cached input at its discounted rate. Token counts are returned in the usage object of each response, so the calculation can be done exactly per request.

Why do output tokens cost more than input tokens?

Input tokens are processed together in one parallel pass, while output tokens are generated one at a time, each needing its own pass through the model and holding memory while the sequence grows. Generating a token uses much more provider compute time than reading one, so output rates are typically several times higher than input rates for the same model.

Are reasoning tokens charged?

Yes. Reasoning tokens are billed at the output rate and count toward the max_tokens budget, even when the API does not return the reasoning text. On difficult prompts they can exceed the visible answer by a wide margin. Lowering the reasoning effort setting on tasks that don’t need it is one of the most direct ways to reduce cost on reasoning models.

What is the difference between cached input and response caching?

Cached input is a provider feature: a repeated prompt prefix is billed at a discounted rate, but the model still runs and you still pay for output. Response caching sits in front of the model: an identical request returns a stored answer and no model call is made. Response caching only makes sense where the same answer is acceptable, such as deterministic requests.

Why is the same open-weight model cheaper at some providers?

Open-weight models can be served by any provider with the hardware, and providers differ in GPU generation, utilization, batching, region and precision. Those cost differences show up as different prices for the same model. A lower listed price is only a real saving if the provider serves the same weights reliably, which is why health and fidelity checks matter.

Does max_tokens affect what I pay?

No. You are billed for tokens the model actually generates, not for the max_tokens ceiling. A higher ceiling costs nothing extra if the model stops early. A ceiling that is too low can cost more overall, because a reasoning model may spend the budget reasoning and return a truncated answer that you then have to retry.