An agent cost dashboard can look precise while charging the same tokens twice. One column contains total input tokens. Another contains cached input. A third contains reasoning tokens. Multiply every column by a price and add the results, and the spreadsheet produces a plausible dollar amount.
The mistake is treating every usage field as an additional billable quantity.
I would test the accounting before using that dashboard to choose a cheaper model or cut an evaluation budget. Cached input can be part of the reported input total. Reasoning can be part of the reported output total. A detailed field explains a total; it does not necessarily add to it.
The example below is synthetic. Its token counts and prices are invented teaching inputs, not a current model quote, a measured agent run or an invoice. The downloadable Python program makes no network requests.
Read the billing categories before writing the formula
OpenAI’s prompt-caching documentation says that cache-write pricing is not an additive fee: input tokens use the uncached-input, cached-input or cache-write rate. Its response examples place cached_tokens and cache_write_tokens inside input_tokens_details beneath the input total.
That contract requires a partition. For the single-write-rate case used here:
ordinary input = total input - cache reads - cache writes
Each input token goes into one of those billing buckets. Charging the entire input total at the ordinary rate and then adding cached-input and cache-write charges counts those tokens again.
The same documentation describes model-dependent caching behaviour and different write rates for different cache lifetimes. Do not copy one universal price into every request. If a request mixes differently priced write categories, preserve those categories separately. The small calculator below deliberately supports only one cache-write rate and must not receive a flattened mixture.
OpenAI’s reasoning guide describes reasoning tokens as billed output tokens and shows reasoning_tokens under output_tokens_details. For that usage contract, multiplying total output tokens by the output price already includes reasoning. Adding a second reasoning charge overstates the estimate.
These are provider-specific meanings. An adapter for another API must establish its own inclusion rules. Similar field names are insufficient evidence that two providers report the same quantities.
A small counterexample you can check by hand
Suppose a normalised record contains 10,000 input tokens. Of those, 6,000 are cache reads and 2,000 are cache writes. The remaining 2,000 use the ordinary input rate. The response contains 1,000 output tokens, including 700 reasoning tokens.
Use these hypothetical USD prices per million tokens:
| Billing bucket | Tokens charged | Invented rate | Estimated USD |
|---|---|---|---|
| Ordinary input | 2,000 | 2.00 | 0.0040 |
| Cache reads | 6,000 | 0.20 | 0.0012 |
| Cache writes | 2,000 | 2.50 | 0.0050 |
| All output, including reasoning | 1,000 | 8.00 | 0.0080 |
| Total | 0.0182 |
The faulty formula charges all 10,000 input tokens at the ordinary rate, adds cache reads and writes, charges all output, and then adds 700 reasoning tokens at the output rate. It returns USD 0.0398 for the same record.
No agent behaviour changed. The difference comes entirely from the accounting formula. An evaluation team could misread that difference as a model-cost regression, especially if a newer run uses more cached context or spends more of its output budget on reasoning.
Reasoning remains worth measuring separately. It can explain why two requests with similarly short visible answers have different output usage. Keep it as a diagnostic subset without charging it twice. Also avoid labelling output - reasoning as the exact visible-answer token count: the reasoning guide includes non-visible formatting tokens among generated tokens.
Run the calculator and its failure cases
Download token-cost-partition.py. It uses Python’s standard library, including Decimal for monetary arithmetic.
python3 token-cost-partition.py
python3 -O token-cost-partition.py
No-argument mode prints the synthetic estimate and runs twelve unit tests. Those tests check the partition, zero usage, uncached requests, small charges and reasoning-subset accounting. They also reject overlapping input counts, missing fields, unknown categories, duplicate JSON keys and malformed prices. The validation still runs under optimised Python.
To supply your own normalised teaching record, save this as usage.json:
{
"counts": {
"input": 10000,
"cached_input": 6000,
"cache_write": 2000,
"output": 1000,
"reasoning": 700
},
"usd_per_million": {
"uncached_input": "2",
"cached_input": "0.2",
"cache_write": "2.5",
"output": "8"
}
}
python3 token-cost-partition.py usage.json
These are application-owned field names, not an OpenAI request or response schema. A production adapter must map the provider’s usage into this structure only after checking the inclusion rules and pricing scope. Do not paste an API response into it and expect the program to infer the contract.
Prices are decimal strings rather than binary floating-point values. The program retains sub-cent charges instead of rounding each request to cents. Apply the provider’s actual aggregation and rounding rules during invoice reconciliation; rounding every tiny request independently can distort a batch total.
A valid input exits with status 0 and prints estimated_token_usd. Invalid input exits with status 1. Successful calculation means the supplied record satisfies this calculator’s constraints. It does not verify the provenance of the usage, the price selection or the completeness of the experiment.
Unknown usage should stop a complete-cost claim
A missing cache field and a known zero cache count are different observations. The first means the adapter lacks evidence. The second means the relevant category was checked and contains no tokens.
The fixture requires explicit values. It refuses a record where cache reads plus writes exceed total input, or reasoning exceeds total output. These checks catch impossible partitions under its declared contract. They cannot establish that an upstream collector reported every request.
For a real experiment, start with the scheduled task inventory and retain a request-level ledger. Record each attempt, its provider request identity when available, completion state and whether usage arrived. A timeout can leave the caller uncertain about server-side work. Do not silently replace missing usage with zero or drop that attempt from a cost-per-task report.
Report the subtotal for known usage alongside the number of attempts whose cost remains unknown. Withhold a complete total until those gaps are reconciled. The earlier guide to agent retries explains why a later successful attempt does not erase earlier work.
Deduplicate observations of the same provider request, but retain genuinely separate retries. Re-reading one response is not another model execution. Sending a new request can be. A ledger needs both a logical task identifier and request-level identities to preserve that distinction.
An estimate needs a price record
A decimal calculation can be exact for the wrong prices. Store the provider, model or snapshot, service tier, currency and applicable price date with the run. Preserve the source of those prices and any relevant caching configuration. A comparison that changes the model and the pricing basis at the same time needs to disclose both changes.
The calculator is intentionally narrower than a billing system. It does not handle mixed cache-write rates, audio or image-specific categories, tool charges, storage, discounts, credits, tax or negotiated rates. It accepts a normalised single-rate record and checks its arithmetic. Reject unsupported records at the adapter rather than squeezing extra categories into an ordinary-input count.
Separate the token estimate from other costs in the evaluation report. A browser session or search tool may have its own charge. Human review is labour, with its own stated rate and time measurement. Combining everything under a field called “model cost” makes later investigation unnecessarily difficult.
When comparing two agent configurations, preserve task outcomes beside cost. A cheaper run that omits required checks is not evidence of a more efficient solution. The slice-level evaluation example shows why an aggregate improvement can conceal a consequential failure.
For a broader workflow around task contracts, graders and evaluation operating costs, see AI Agent Evals. Get the book on Leanpub.
Before changing the next model budget, pass one retained usage record through your adapter and write down which fields contain other fields. Then compare the calculated subtotal with the provider’s usage and billing records. Leave any unexplained difference visible in the report.