Token prices are the wrong unit for enterprise planning. A run is the unit that maps to a business outcome, so a run is where cost control belongs.
Define an envelope for every run
An envelope has hard ceilings on each of:
- Model calls and tokens consumed.
- Tool calls, especially paid or rate-limited ones.
- Wall-clock time.
- Money, as an estimated total.
When any ceiling is hit, the run stops in a defined state: a partial result with an explanation, or an escalation to a human. A loop that cannot be stopped by a budget will eventually be stopped by an invoice.
Route by task difficulty
Not every step needs the strongest model. A routing layer assigns each step to the cheapest model that passes its evaluation:
- Classification and extraction: small, fast models.
- Planning and hard reasoning: the most capable model.
- Summaries and formatting: mid-tier.
The discipline is that routing decisions come from *evals*, not from intuition. For each step type, record which model passes at what rate and cost, and revisit when new models ship.
Other levers that usually matter more than model choice
- Cache stable prompt prefixes and repeated retrievals.
- Trim context. Send what the step needs, not the whole history.
- Batch non-urgent work.
- Stop early when the goal is reached or progress has stalled.
- Cap retries and make them smarter than "try again".
Attribute cost
Tag every call with tenant, agent, run, and step. When someone asks why the bill rose, you want to answer with a table, not a guess. Showback to the teams that own agents tends to change behavior faster than any central mandate.
Add a circuit breaker
Detect cost anomalies: a run using ten times its median, a tenant spiking overnight. Alert and, for severe cases, pause automatically. Treat it like any other production spend control.