Cost engineering · Practical guide
Before switching models, check these three API cost controls
A practical workflow for separating urgent requests, reusable context, and unnecessary output before making an AI API buying decision.
Start with the work that can wait
A support reply needs an answer while the customer is still there. An overnight classification job may have hours to finish. Sending both through the same delivery path can hide a cost-control opportunity. Before changing the model, divide your requests by deadline: immediate, same day, or later. Then check which delivery modes the provider offers for each group. A cheaper result that misses its deadline has not completed the task.
Google's Batch Mode announcement describes an asynchronous option for nonurgent workloads. Its published offer compares batch prices with synchronous prices for the same model. That announcement is background, not a current quote for every model. Check the current model's eligibility and pricing before estimating savings. Our recommendation is to run a small representative sample first, preserve an identifier for each input, and measure completed usable results rather than submitted requests. A discount is unhelpful if a delayed result arrives after the decision that needed it.
Choose the first experiment by the work you have
- 1Delivery deadline
If a result can arrive later, check batch availability and completion limits.
- 2Repeated context
If requests share a substantial stable prefix, test cache writes and subsequent reads.
- 3Consumed output
If the application discards generated text, remove that requirement and retest quality.
Look for repeated context before shortening it
The next question is whether your requests keep sending the same substantial context. Anthropic's prompt caching article describes reusing context across calls. This is particularly relevant when a workflow repeatedly supplies a shared document or long instruction set. It does not mean that every repeated-looking request receives a cache benefit. Treat the provider's current cache rules and reported usage as the deciding evidence.
A practical investigation starts with a redacted map of your prompt: stable instructions, stable reference material, changing user input, and changing retrieved information. Measure which sections actually repeat. Try a layout that keeps shared context consistent, then inspect the provider's reported cache usage. Record both the initial setup cost and subsequent requests. Do not turn an advertised maximum saving into your budget forecast. A workload with little reusable context may gain little; a useful cache trial must include the traffic pattern you expect in production.
Decide what a useful answer needs to contain
Finally, inspect the answer your application actually consumes. OpenAI's production guidance presents token volume and token price as cost considerations. Our suggested audit is to follow one result from generation to the place where a reader or program uses it. If the application consumes three fields, ask why the prompt also requests an introduction, explanation, and repeated summary. Remove requirements that do not serve the task, while retaining information needed to interpret the result safely.
This is an editorial workflow recommendation, not a measured claim about your application. Build a small evaluation set containing ordinary inputs, awkward cases, and failures you already understand. Compare the revised output with the existing version using the same completion criteria. Record usable results, retries, total billed usage, and elapsed time. If shorter responses cause more retries or human correction, the change may increase complete task cost. Keep that failed experiment in the record instead of reporting only the requests that looked cheaper.
What has to be true before a lower bill means progress
Scroll sideways to read all columns.
| Control | Fits when | Evidence to collect | Watch for |
|---|---|---|---|
| Batch | The result can wait | Completion time, model eligibility, usable results | Results arriving after the deadline |
| Prompt caching | Substantial context repeats | Initial cache cost, read usage, total traffic | Little reuse or frequent context changes |
| Output reduction | Generated text goes unused | Accepted answers, billed usage, retries | Extra retries or human correction |
Keep one decision record
For each experiment, write down the task, provider and model, delivery mode, date of the price check, billing units, result quality criteria, and whether the deadline was met. Change one variable at a time where practical. Start with the control your workload can actually use: waiting capacity for batch, repeated context for caching, or unnecessary requested content for output reduction.
Compare the result with your existing workflow using the same inputs and acceptance criteria. Include the initial cache write, unfinished batch work, retries, and any required human correction in the account. Keep the smallest change that delivers an acceptable result within the deadline. A model comparison becomes useful once these operating conditions are explicit: you can then tell whether a lower invoice came from a different provider, a better request pattern, or less work being completed.
Sources & verification
Provider documentation supports the facts referenced below. Our explanations and recommendations are editorial interpretation.
- Google: Batch Mode in the Gemini API ↗Checked Oct 31, 2026
- Anthropic: Prompt caching with Claude ↗Checked Oct 31, 2026
- OpenAI: Production best practices ↗Checked Oct 31, 2026
Corrections: hello@aiapicost.com