The idea behind it
A price per million tokens is the figure most often quoted for what an AI feature will cost, and it is one input among several. What decides whether a feature pays is the cost of one completed task. That cost includes every model call the task takes, including planning steps, tool calls and checks; the retries after malformed output, failed validation or timeouts; the input sent with each call, which in retrieval and conversational designs is usually far larger than the output; any share of input billed at a cached rate; retrieval and tool charges; the minutes a person spends reviewing the output; and the tasks a person still finishes by hand after the AI fails.
The model compares that cost with what the task costs today and adds the fixed costs that do not change with volume: the platform, observability, and the evaluation runs that every prompt or model change requires. Dividing the fixed costs by the saving per task gives the break-even volume. Because cost per task is a distribution, the model runs a typical case from the median task in the pilot traces and a bad case from a task near the 90th percentile, and the budget is sized against the bad case. In the Northwind sample, model tokens are a small share of the variable cost and review time is the largest, so shortening review moves the result more than a lower token price.
What's inside
- Inputs for a typical case and a bad case: model calls per task, input and output tokens per call, retry rate, cached input share, retrieval cost, review minutes and fallback rate.
- Prices per million input, cached input and output tokens as inputs, with no vendor prices built in.
- Today's cost per task from minutes, loaded hourly cost and any per-task fees, and monthly volume.
- Results: model cost per call and per task, human review and fallback cost, variable and fully loaded cost per task, net monthly saving and break-even volume.
- Sensitivity tables for cost per task and break-even volume across price multipliers and retry rates, and cost per task from 10% to 200% of planned volume.
- A worked example for a fictional company, Northwind Logistics, with sample assumptions; the PDF guide explains each input and how to measure it.
Format: PDF (4 pages) and Excel workbook.
A look inside
Pages 1 and 2 of 4 of the PDF. Select an image to enlarge it.
Who uses it
- CIOs and heads of AI, to approve a feature on its cost per completed task rather than its token price.
- FinOps leads, to forecast AI spend from volume, calls and retries, and to set spend alerts.
- Product and engineering leads, to see which design choices drive the cost: context size, steps, caching and retries.
- Finance business partners, to test a business case's saving and break-even volume.
When to use it
- A pilot has finished and the feature is going to a production decision.
- A business case quotes the provider's token price as the cost of the feature.
- A provider changes its prices or a team proposes a different model.
- An agentic design will make several model calls per task.
- Spend on a live feature is rising faster than its volume.
How to use it
Fill it in from pilot traces and timed samples before a feature is built out, and again each quarter with production data.
- Instrument the pilot so calls per task, tokens per call, retries and cache hits come from traces and the provider's usage log.
- Time today's task and the review step on a sample of real work, and get the loaded hourly cost from finance.
- Enter prices from your contract or the provider's price page, with the date you read them.
- Fill in the typical case from the median task and the bad case from a task near the 90th percentile.
- Read the variable and fully loaded cost per task, the net monthly saving and the break-even volume for both cases.
- Use the sensitivity tables to see whether price, retries or volume moves the result, and re-run the model each quarter with production data.
You leave with: The cost of one completed task for the feature in a typical and a bad case, against today's cost, with the monthly saving, the break-even volume and the levers that move them.
Where it goes wrong
- Costing only the tokens and leaving out review time and the tasks people still finish by hand.
- Using the average task. The bad case is where an agent loops or a conversation runs long, and it sets the budget.
- Estimating calls and tokens from the design document instead of measuring them in pilot traces.
- Leaving evaluation and development traffic out of the fixed costs.
Get the model
It is free. Enter your work email and we send the download link.
Questions
How do you calculate the cost per task of an AI feature?
Multiply the model cost per call by the calls per task and by one plus the retry rate, then add retrieval and tool costs, the review cost (share reviewed times review minutes times the reviewer's hourly cost) and the fallback cost (fallback rate times today's cost per task). Add fixed monthly costs divided by volume for the fully loaded figure.
What is the break-even volume for an AI feature?
The monthly volume at which the saving per task covers the fixed costs: fixed platform plus evaluation and development cost, divided by today's cost per task minus the AI's variable cost per task. If the AI's variable cost per task is higher than today's cost, there is no break-even volume.
Why does the model include a bad case?
Cost per task varies with how many calls, retries and tokens a task needs. The typical case uses the median task from pilot traces and the bad case a task near the 90th percentile, so the budget covers the expensive tasks as well as the usual ones.