Failed attempts burn tokens too: measure LLM cost per task

Treat AI usage as an engineering cost by measuring cost per successful task: what the work spent, divided by the tasks that passed your check. Coding agents and AI features in a product both pay for the tokens every step sends and receives. The levers match too: context re-sent per step, steps per task, the model on each step, and caps that stop a runaway loop.

The tool and API behaviour below comes from each provider’s official docs, last verified on 11 October 2026. None of it was tested for this post.

This is part 22 of the series. Part 14 covered managing the context window during a task, and Part 21 covered tracing an LLM application. This part adds up what that work costs.

Count the failed attempts in cost per successful task

Cost per successful task is the total spent on a task across every attempt, divided by the tasks that passed. “Passed” means whatever your team accepts as done. For a bug fix, that might be green tests and an approved review. For a support answer, it might be a reply the customer did not escalate.

An example formula, with failed attempts counted on the cost side only:

cost per successful task =
    ( model spend for every attempt, passed or failed
    + tool and infrastructure spend for those attempts )
    / tasks that passed the acceptance check

Price per token gives the cost of one call. A task can need several calls, and much of each call can be the same context sent again.

A made-up example: a small model fixes a bug on its third try, and a larger model fixes it on its first. The small model wins on price per call. On cost per successful task, it also pays for two failed attempts and three full contexts, and the comparison can go either way.

Anthropic’s Building effective agents puts the trade plainly. “Agentic systems often trade latency and cost for better task performance, and you should consider when this tradeoff makes sense.” Some teams also add the time a person spent fixing the output. That is a choice, so write down which version you use.

To compute any of this, you need a record per task as well as per call. Example fields, with names made up for this post:

task_id         the ticket, pull request or request it belongs to
model           per call; one task can use several models
input_tokens    uncached input, per call
cache_read      input served from the prompt cache
cache_write     input written to the cache
output_tokens   including reasoning or thinking tokens
tool_calls      how many, and which tools
attempts        first try, retries, re-runs after a failed check
passed          did it pass the acceptance check?

Most of these come back in each API response. Anthropic’s prompt caching docs name input_tokens, cache_read_input_tokens and cache_creation_input_tokens. OpenAI’s prompt caching guide says to track usage.input_tokens_details.cached_tokens.

Recording them per call is tracing, which Part 21 covers. Here you group them by task.

What drives the bill: context re-sent on every step

An AI bill grows with tokens, and agent work sends the same context many times over. OpenAI’s Codex pricing page lists the inputs: “Model choice, context, reasoning, tool use, retrieval, and caching all affect usage, so prompt length alone isn’t a reliable estimate.”

Most rows quote Claude Code’s cost page, because it spells these mechanics out. They hold for any agent or feature that re-sends a conversation.

Context size is the driver you control most directly during a task. Part 14 covers it, from what fills the window to when to start clean. A sub-agent keeps your main window small, but its requests still count. Claude Code’s docs suggest giving it a smaller model.

See usage in Claude Code, Codex, Gemini CLI, Copilot and Cursor

Claude Code, Codex, Gemini CLI, Copilot and Cursor all show usage somewhere, but each coding agent counts it differently. Learn what your plan bills for before you try to lower it.

Every row comes from the linked docs, last verified on 11 October 2026; none was tested for this post. Gemini CLI’s docs also note that for unpaid tier and Google One users, Gemini CLI was replaced by Antigravity CLI on 18 June 2026.

The billing unit changes what a long agent loop costs you. GitHub’s legacy request-based billing still covers some annual Copilot Pro and Pro+ plans.

GitHub’s docs say “only the prompts you send count as premium requests”. Tool calls and other “actions Copilot takes autonomously to complete your task” don’t count. Under token billing, every one of those tool calls adds to the bill.

Look at the numbers on a schedule. The Codex pricing page suggests checking the dashboard “every week or two”. When usage runs high, it asks “whether a smaller model or tighter task scope would still produce a useful result”.

Prompt caching pays when the start of the prompt stays the same

Prompt caching lets a provider reuse the processed start of a prompt, called the prefix, when the next request begins with the same content. OpenAI’s guide says “Cache reuse requires the entire rendered prefix to match.” Change one token near the start, and everything after it misses the cache.

The two providers differ in how you turn caching on. OpenAI’s guide says “Prompt caching is enabled by default for supported OpenAI models.” Its minimum is 1,024 tokens for GPT-5.6 and later, and it describes the cached-input rate as “discounted up to 95%”.

On Anthropic’s API, you mark the cache point with cache_control, or add one top-level cache_control field for automatic caching. Its docs say “By default, the cache has a 5-minute lifetime. The cache is refreshed for no additional cost each time the cached content is used.”

A cache write costs extra. Anthropic’s pricing section says “5-minute cache write tokens are 1.25 times the base input tokens price”. A prefix you cache and never read again costs more than sending it uncached. Short prompts are skipped silently: per the same page, they “cannot be cached, even if marked with cache_control”, and no error is returned.

Anthropic’s advice on layout is “Place static content (tool definitions, system instructions, context, examples) at the beginning of your prompt.” An example request layout for an AI feature:

1. tool definitions           same on every call    (cached)
2. system instructions        same on every call    (cached)
3. reference docs, examples   same on every call    (cached)
4. conversation so far        grows each turn
5. the user's new message     changes every call

The usual ways to break that prefix are a timestamp or user name in the system prompt, and tools that change order between calls. Anthropic’s docs add that “Modifying tool definitions (names, descriptions, parameters) invalidates the entire cache”.

Coding agents cache for you, and the cache lifetime decides what a break costs. In Claude Code, the lifetime “is an hour on a subscription and drops to five minutes once you’re drawing on usage credits”; on an API key or cloud provider it is five minutes by default (docs). From v2.1.251, /usage adds a Prompt cache (main) line with the share of input served from cache and the misses (docs). Measure the hit rate from the usage fields before you assume caching is working.

Match the model, effort and speed to each step

Choose the model per step

A task has steps of different difficulty, so the model can change between them. Claude Code’s cost docs say “Sonnet handles most coding tasks well and costs less than Opus. Reserve Opus for complex architectural decisions or multi-step reasoning.” The same idea works in a product: a small model to classify a request, and a larger one only for the requests that need it.

Test a switch on cost per successful task. A cheaper model that fails your check more often can cost more per task. Run each candidate on a small set of real tasks with known good answers before you move traffic.

Lower the reasoning effort on routine steps

Reasoning is billed as output. Claude Code’s docs say “Thinking tokens are billed as output tokens, and the default budget can be tens of thousands of tokens per request depending on the model.” Claude Code has /effort, Codex sets effort through /model, and Copilot CLI’s /model covers it too. Keep high effort for planning and hard debugging, and lower it for renames and formatting.

Send work that can wait to a batch API

A batch API trades speed for price on work nobody is waiting for, such as nightly evaluations or a backlog to classify. Anthropic’s Message Batches API has “most batches finishing in less than 1 hour while reducing costs by 50%”. On the same page, batches that don’t finish within 24 hours expire. OpenAI’s Batch API offers a “50% cost discount compared to synchronous APIs”, and “Each batch completes within 24 hours (and often more quickly).”

Batches stack with caching. Anthropic’s batch docs suggest the 1-hour cache duration “when processing batches with shared context”, because batches can take longer than five minutes. Anything a user waits on, including a coding agent’s own loop, stays on the normal API.

Set budgets and alerts before the bill does

A budget turns a surprise into a notification. Set one at each level you control.

The OpenAI and Claude Code caps can overshoot. OpenAI’s spend limits guide says “recorded spend can slightly exceed the configured amount”. Claude Code’s CLI reference says of --max-budget-usd: “Spend can pass the cap, so leave headroom.”

Your own code needs caps per task as well, because a provider budget acts on the monthly total. An example config, with names and values made up for this post:

support_answer:
  model: small            # escalate to the large model only on low confidence
  max_output_tokens: 800
  max_tool_iterations: 4
  max_retries: 2          # with backoff; a retry resends the full prompt
  on_budget_exceeded: hand_to_human
nightly_ticket_summary:
  api: batch
  max_output_tokens: 400

Give every loop an on_budget_exceeded action. A loop that hits its cap should stop and report what it has done so far.

Two prompts and a checklist for AI cost

Neither prompt relies on one tool’s features. Both work in any coding agent that can read your repository.

Prompt 1: run a coding task on a budget

An example prompt for a coding agent task where you want the work scoped and capped before it starts.

Task: <describe the task>
Done means: <the check that must pass, such as a test command>

Before editing anything:
1. List the files you expect to read and change, and why.
2. Say which steps need deep reasoning and which are routine.
3. Say how you will prove the change with the narrowest test.

Then work within these limits:
- Read only the files the task needs. Search before opening whole files.
- Run the narrowest test until the final check, then run the full check.
- Filter long command output to the failing lines.
- Ask before reading large generated files, starting sub-agents or
  running long jobs.

Never print, expose or commit secret values.
Do not commit, push, deploy, run destructive commands or make external
changes unless I explicitly authorise it.
If two reasonable attempts fail, stop and report the failure rather
than repeatedly guessing.

End with the check result, the attempts used, and anything you skipped.

Compare the attempts it reports with your agent’s usage command afterwards.

Prompt 2: find the cost drivers in an AI feature

An example prompt for reviewing how a product feature calls a model API.

Task: review how <the feature or service> calls the model API.
Start from: <path or entry point>

Do not change code and do not call the model API.
Build a cost map of every model call:
1. Where is it called, which model, and what triggers it?
2. What goes into the prompt, in order? Mark each part as the same
   on every call or changing.
3. Does anything near the start change on every call, such as a
   timestamp, a user name or tools in a different order?
4. Is output capped? Is reasoning effort set?
5. How are failures retried: how many times, with what backoff?
6. Do tool or agent loops have a maximum number of iterations?
7. Which calls have no user waiting and could run as a batch?
8. Are the usage fields from each response recorded with a task or
   request ID?

Cite the file and line for each answer. Mark anything you could not
confirm as unverified.
Never print, expose or commit secret values.
Do not commit, push, deploy, run destructive commands or make external
changes unless I explicitly authorise it.
If two reasonable attempts fail, stop and report the failure rather
than repeatedly guessing.

End with the three changes most likely to lower cost per successful
task, and what each could break.

An AI cost review checklist

An example checklist, in the order of this post:

AI USAGE: COST REVIEW

□ "Done" defined by a check, for each kind of task
□ Spend recorded per task, failed attempts included
□ Usage fields saved from every response
□ Same-every-call content first, changing content last
□ Cache hit rate measured from the usage fields
□ Output capped; reasoning effort set per step
□ Retries capped, with backoff
□ Tool and agent loops capped
□ Model chosen per step, tested on real tasks
□ Work with nobody waiting sent to a batch API
□ Spend alerts on every project or workspace
□ Hard limits on experiments and development keys
□ Coding agent usage checked every week or two

Quick answers

What is cost per successful task?

Cost per successful task is the total AI spend across every attempt, divided by the number of tasks that passed an acceptance check. Failed attempts, retries and tool calls count as cost, and only tasks that passed count as output.

What makes LLM application costs grow?

Large language model (LLM) costs grow mostly from context re-sent on every request. Long sessions, tool loops, retries, sub-agents and cache misses each add requests that carry the whole conversation again. Output tokens, including reasoning or thinking tokens, are billed on top.

How does prompt caching lower LLM costs?

Prompt caching lets the provider reuse the processed start of a prompt when a request begins with the same content. Cached input is billed at a reduced rate. Keep tool definitions and instructions first, and changing content last, so the prefix matches.

When should you use a batch API for LLM requests?

A batch application programming interface (API) suits work nobody is waiting for, such as evaluations or nightly summaries. Anthropic and OpenAI both document a 50 percent discount for batches, with results within a 24-hour window.

How do you check token usage in a coding agent?

Coding agents have their own usage commands. Claude Code uses /usage, Codex CLI uses /status and /usage, Gemini CLI uses /stats model, and GitHub Copilot CLI uses /usage. Cursor shows usage in editor settings and on its usage dashboard.

More on AI coding is in the AI category.