Agent cost and latency SLOs. Price on cost per success with failures counted; early abort is the biggest margin lever

Ask “how much does this AI model cost?” and you’ll get a price per million tokens. For AI agents, that’s almost the wrong question. What you actually care about is: how much does it cost to get a job done, including all the attempts that failed?

Cost per success

Cost per success = total spend on every attempt (failures and retries included) divided by the attempts that passed.

Arize and Fireworks ran 10 models on 40 command-line tasks, 2,400 runs total.

gpt-oss-120b: $0.018 per attempt, passed 33%, so $0.054 per success.
GPT-5.5: $0.424 per attempt, passed 67%, so $0.636 per success.

The cheap model wins on cost per success by about 12x. But look at coverage: gpt-oss-120b reliably solved 8 of the 40 tasks; GPT-5.5 reliably solved 25. The cheap model looks cheap partly because it only finishes the easy ones. You need both numbers per type of task.

Variance and failures

A study of coding agents found the most expensive problem used about 7 million more tokens than the cheapest, and across four runs of the same agent on the same problem, the priciest run typically cost about twice the cheapest. Frontier models also guessed their own token use badly (correlation at most 0.39), and guessed low.

Failures are where the money goes. In a study of command-line coding agents fixing bugs, failed repairs used 39% to 343% more tokens than successful ones.

The biggest lever: give up early

BAGEN asked agents mid-run whether the task was still finishable, and stopped when the answer was no. That saved 28% to 64% of tokens on failed runs, at a cost of 1.6 to 4.2 points of success. GPT-5.2 saved 64% with a 6.6% false-abort rate; Opus saved 28% with 2.2% false aborts.

EarlyEval uses two small classifiers watching each run. Across three agent benchmarks it cut 13% to 26% of steps and up to 44% of input tokens while moving success rates about 1 to 2 points. Lowering its threshold from 0.95 to 0.75 raised step savings on SWE-bench from 26% to 63%, with a 4.1 point hit to resolve rate. That threshold is a product decision.

Latency compounds too

AWS’s Agentic AI Lens says to profile each phase’s slow tail separately, up through p99, because the phase that dominates the typical run often isn’t the one dominating the slow runs. Fan-out makes it worse: if five parallel branches each run slow 1% of the time, about 5% of requests wait on a slow branch.

The classic fix still works. In Dean and Barroso’s “The Tail at Scale,” sending a duplicate request after 10 ms cut BigTable’s 99.9th-percentile latency from 1,800 ms to 74 ms for 2% more requests.

A worked example (illustrative numbers)

Successful runs average $0.40, failed runs average $1.00, and 60% of runs pass. Cost per success is (0.6 x 0.40 + 0.4 x 1.00) / 0.6, about $1.07. If you charge $0.99 per outcome (Intercom Fin’s price, charged at most once per conversation), you lose money on every resolution.

Add an abort policy that halves failed-run spend and costs 3 points of success: (0.57 x 0.40 + 0.43 x 0.50) / 0.57, about $0.78. Same price, about 21% gross margin.

For context: HubSpot’s Customer Agent is $0.50 per resolved conversation; Sierra prices on outcomes with negotiated rates; Salesforce Agentforce has used about $0.10 per standard action. Under outcome pricing, the vendor eats every failed run.

An analogy: the fishing trip

A cheap boat rental costs $50 a day and you catch a fish one day in three. A pricey charter costs $300 and you catch something two days in three. Per fish, the cheap boat wins: $150 versus $450. But the cheap boat only catches small fish near shore. And the smartest move on either boat is knowing when the fish aren’t biting and heading home at noon instead of burning fuel until dark.

Try this

Do the worked example with your own numbers. Pick a success rate, a cost for successful attempts and a cost for failed ones (make failures pricier). Compute cost per success. Then cut failure cost in half and drop success rate by 3 points. Did cost per success go up or down? Try a version where success rate is already low (30%). You’ll find early abort matters most when failures are common.

Use it at dinner

“For AI agents, the price per token barely matters. What matters is cost per finished job, and the biggest savings come from quitting failing jobs early.”