How to reduce LLM API costs: a practical playbook
To reduce LLM API costs, first measure what each feature actually spends, then work through the levers in order of risk: right-size the model, trim prompts and context, cap output and reasoning budgets, cache repeated work, batch anything that can wait, and buy the same model from a cheaper provider. Finish with hard spend caps so a bug can’t burn a month’s budget overnight.
Most LLM bills are not dominated by one bad decision. They are the sum of a few defaults nobody revisited: the biggest model for every call, a system prompt that grew to four thousand tokens, a chat history that is resent in full on every turn, and identical requests paid for over and over. The good news is that each of those is fixable, and most fixes don’t touch your product logic. The order below matters: early steps are cheap and safe, later ones need more care or more infrastructure.
If the pricing vocabulary (input vs output tokens, cached input, reasoning tokens) is new, read LLM API pricing explained first. Everything below assumes you know which side of the meter each token lands on.
Step 1: How do you measure LLM API cost per feature?
You can’t optimize a single number on an invoice. You need cost per request, grouped by the thing that generated it: the feature, the endpoint, the customer, the background job. Without that, teams spend a week shrinking a prompt that accounts for 3% of spend while a forgotten nightly job quietly accounts for 40%.
- Log tokens and charge per request. Every provider returns input and output token counts in the response
usageobject. Store them next to a request id, the model id and a timestamp. - Tag every call with its origin. Use a feature name, an environment and, where relevant, a tenant id. Tags are what turn a bill into a ranked list of problems.
- Compute cost per unit of value. Cost per resolved ticket, per generated report, per active user per day. A feature that doubles in spend while usage triples is getting cheaper.
- Watch the ratio of output to input. Output tokens usually cost several times more than input, so a feature with long answers can outspend one with long prompts.
On Smouter, the x-smouter-tags header (up to 10 comma-separated tags) and a metadata object (up to 16 keys) label each request, and both are filterable in Usage. Metadata stays inside Smouter: it is never forwarded to the provider and does not affect cache matching. Every response also carries a charge receipt header, so you can log the actual cost per call instead of reconstructing it from a price sheet. The understand your usage article walks through the dashboard view.
curl https://api.smouter.ai/v1/chat/completions \
-H "Authorization: Bearer $SMOUTER_API_KEY" \
-H "Content-Type: application/json" \
-H "x-smouter-tags: production,support-summarizer" \
-d '{
"model": "<model-id>",
"metadata": {"feature": "ticket-summary", "tenant": "acme"},
"messages": [{"role": "user", "content": "Summarize this ticket."}]
}' -D - -o /dev/null | grep -i x-smouter-chargeStep 2: Are you using a bigger model than the task needs?
Right-sizing is usually the single largest lever, because prices between model tiers often differ by an order of magnitude or more while many tasks (classification, extraction, routing, short summaries, reformatting) don’t benefit from the top tier at all. The method is boring and reliable: build a small evaluation set of 50 to 200 real inputs per feature, define what a correct answer looks like, and run it against two or three cheaper models. Keep the cheapest one that passes.
Two patterns help when one model can’t serve everything. A cascade sends the request to a cheap model first and escalates only when a check fails (a validator, a confidence score, a schema). A router classifies the prompt up front and picks the tier. Both are covered in LLM model routing. Open-weight models are often the cheap tier in these setups; the open-weight models API guide compares the main families.
Smouter’s role here is optional. You can send model: "auto" to have each request resolved to one concrete model before pricing, or define your own pipeline in the dashboard and call it as model: "@route-name". You can also ignore both and name the model yourself, which is what most teams with an eval set end up doing for their hottest paths.
Step 3: How do you cut input tokens without hurting quality?
Input tokens are cheaper per token than output, but there are far more of them. Agents and chat apps are the worst offenders because they resend the whole conversation, tool schemas and retrieved documents on every turn. Places to look:
- System prompts. Long instruction blocks accrete over months. Delete rules the model already follows, and examples that no eval depends on.
- Conversation history. Keep the last few turns verbatim and summarize older ones, or drop turns that were resolved.
- Retrieval. Retrieve fewer, better chunks. Five relevant passages usually beat twenty marginal ones on both cost and accuracy.
- Tool definitions. Only attach the tools a given step can actually use. Every schema is input tokens on every call.
- Formatting overhead. Verbose JSON with long keys, repeated boilerplate and pretty-printed whitespace all count. See what is a token for how text maps to tokens.
Smouter has an optional prompt trimming and compression layer; whether it helps depends on how much redundancy your prompts carry, so measure it against your eval set like any other change.
Step 4: Should you cap max_tokens and reasoning budgets?
Capping output is about controlling what the model generates, not about the number you put in max_tokens. You are billed on tokens actually generated, not on the ceiling. Setting max_tokens to 4,000 on a call that produces 300 tokens costs the same as setting it to 400. So the real levers are:
- Ask for shorter output. Instructions like "answer in at most three sentences" or a tight JSON schema reduce generated tokens directly.
- Use structured output for extraction instead of prose with the answer buried inside.
- Lower reasoning effort on reasoning models where the task allows it. Reasoning tokens are billed as output even though you never see them, and on hard prompts they can dwarf the visible answer.
- Keep
max_tokensas a safety rail against runaway generations, not as a cost dial.
Step 5: How much can caching save on LLM calls?
There are two different things called caching, and they save money in different ways. Prompt (prefix) caching is done by the model provider: when many requests share a long identical prefix, such as a system prompt plus tool schemas, the repeated part is billed at a discounted cached-input rate. Response caching happens in front of the model: an identical request returns a stored answer and no model call happens at all. The detailed comparison is in LLM caching explained.
To benefit from prefix caching, put stable content first and variable content last: system prompt, then tools, then retrieved context, then the user turn. Anything that changes per request (a timestamp, a request id) placed near the top breaks the prefix for everything after it.
On Smouter, response caching is on by default on the value lane, is isolated per account, and applies only to deterministic requests: an explicit temperature: 0 or a pinned seed. Semantic caching is opt-in with x-smouter-cache-semantic: on. You can control TTL and namespaces per request, or skip the cache with x-smouter-cache: no-store. Retries are a hidden source of double billing: an Idempotency-Key header replays a completed response without charging again. The caching support article covers what shows up in Usage.
Step 6: Can you batch work that doesn’t need an instant answer?
Evaluations, backfills, nightly classification, document enrichment and embedding refreshes rarely need an answer in two seconds. Batch APIs accept a file of requests, process them asynchronously within a window (commonly 24 hours), and charge less than the synchronous rate. The trade is latency and operational complexity: you need to submit, poll, download and reconcile results by id. The LLM batch API guide covers the workflow.
Smouter’s Batch API takes chat-completion lines at POST /v1/batches with a 24-hour completion window and returns results as NDJSON. Completed jobs earn a discount; the rate shown in the dashboard is a ceiling, and the realized credit can be lower on thin-margin models. Batch does not work with zero-retention keys, because inputs and results have to be stored for retrieval. See batch processing.
Step 7: Can you buy the same model for less?
For open-weight models in particular, many inference providers serve the same weights, and their prices for identical models can differ substantially. Moving to the cheapest one by hand is risky: the cheapest listing may be slow, error-prone, or serving a different quantization than you expect. This lever changes nothing about your prompts or quality targets; it only changes who you pay.
This is the layer Smouter runs on every request. Each call is shopped across a benchmarked pool of providers serving the same model weights; nodes are fingerprinted against reference weights and benchmarked continuously, and nodes that advertise a low price and then fail to serve are demoted. The site states prices of up to 50% under OpenRouter on the same model. The actual saving depends on the model and the moment, and each response reports it in the x-smouter-saved header. Live per-model prices are on the models page, for example DeepSeek V4. Integration is a base URL change, covered in the OpenAI-compatible API guide.
If a request must go to the model maker’s own endpoint, for compliance or data handling, send x-smouter-lane: verified. That lane has no sourcing discount and nothing is cached; the value vs verified lanes article explains the trade-off.
Step 8: How do you stop a runaway LLM bill?
Every cost playbook ends with guardrails, because the most expensive incidents are not inefficiency, they are loops: an agent that retries forever, a cron job deployed to every pod, a leaked key. Put limits in place before you need them:
- Separate keys per service or environment, so one misbehaving job can be revoked without taking down the rest.
- Spend caps per key: monthly and daily, enforced before the request is admitted.
- Tokens-per-minute limits that turn a runaway loop into 429s instead of a bill. See rate limits and TPM caps.
- Model allowlists so a dev key can’t call the most expensive model by accident.
- Prepaid balances and alerts, so spend is bounded by what you deposited.
Smouter supports all five: per-key monthly and daily caps with a warning threshold, per-key and per-org TPM limits, model and provider allowlists, prepaid credits with low-balance emails, and optional automatic top-ups with a monthly limit. Create an API key per service to get the isolation.
Which LLM cost levers have the best effort-to-impact ratio?
| Lever | Effort | Quality risk | Typical impact | Smouter’s role |
|---|---|---|---|---|
| Measure and tag | Low | None | Indirect: tells you where the rest goes | x-smouter-tags, metadata, charge receipt per request |
| Right-size the model | Medium (needs an eval set) | Medium | Often the largest single lever | Optional auto and @route; or name models yourself |
| Trim prompts and context | Medium | Low to medium | Large for agents and RAG | Optional compression layer |
| Shape output and reasoning | Low | Low to medium | Large when answers or reasoning are long | Warning header on truncated budgets |
| Caching | Low | Low if keyed correctly | Large for repetitive traffic, near zero otherwise | Response cache on by default for deterministic requests |
| Batch async work | Medium | None | Moderate, on eligible jobs only | Batch API with completion discount |
| Buy the same model cheaper | Low (base URL change) | Low with health checks | Depends on model and provider spread | Core function, always on in the value lane |
| Spend caps and limits | Low | None | Prevents worst-case incidents | Per-key caps, TPM limits, allowlists |
What order should you do this in?
- Instrument and tag for a week. Rank features by spend.
- Put spend caps and per-service keys in place immediately; it costs nothing.
- For the top two or three features, build an eval set and test cheaper models.
- Trim prompts and reorder them for prefix caching; set deterministic parameters where answers should repeat.
- Move non-interactive jobs to batch.
- Route through a gateway that buys the chosen models at a lower price, and compare the receipts against your old invoice.
Coding agents deserve their own pass, because their context grows with every tool call; the AI coding agents cost guide covers that case, and the docs list every header mentioned here.
Frequently asked questions
What is the fastest way to reduce LLM API costs?
The fastest wins with the least risk are usually caching repeated requests, moving the same model to a cheaper provider via a base URL change, and asking for shorter output. Right-sizing the model often saves more, but it needs an evaluation set first so you know the cheaper model still does the job. Measure cost per feature before you start so you know which change actually mattered.
Does setting a lower max_tokens reduce cost?
Not directly. You are billed for tokens the model actually generates, not for the max_tokens ceiling. A lower ceiling only saves money when the model would otherwise have produced more, and on reasoning models a tight ceiling can truncate the answer after you have already paid for the reasoning. Reduce output by asking for shorter answers, using structured output, or lowering reasoning effort.
Are cheaper models good enough for production?
For many tasks, yes: classification, extraction, routing, reformatting and short summaries often show no measurable quality difference between tiers. The only reliable way to know is to test on 50 to 200 real inputs from your own traffic with a clear pass criterion. Keep the expensive model for the tasks where the eval shows it earns its price.
How does prompt caching lower LLM costs?
With provider prompt caching, a repeated prefix such as a long system prompt and tool schemas is billed at a discounted cached-input rate instead of the full input rate. Response caching goes further and skips the model call entirely for identical requests. To benefit, keep stable content at the start of the prompt and anything that changes per request at the end.
Is it worth using a batch API to save money?
For work that can wait hours, usually yes: evaluations, backfills, nightly enrichment and bulk classification. Batch APIs charge less than synchronous calls in exchange for an asynchronous completion window. The cost is operational, since you have to submit, poll and reconcile results by id. Interactive features should stay synchronous.
How do I prevent a surprise LLM bill?
Use a separate API key per service, set monthly and daily spend caps on each key, add tokens-per-minute limits so loops fail with 429s instead of spending, and restrict each key to the models it needs. A prepaid balance with low-balance alerts adds a final ceiling on what can be spent.