How to control LLM API costs before the invoice arrives

Almost every team that ships an LLM feature has the same month: usage looks fine, the feature works, and the bill is three times what anyone modelled. Nothing went wrong exactly — the cost just accumulated somewhere nobody was looking.

There are only four places it accumulates, and each has a different fix.

1. Input you did not need to send

This is the largest and the most invisible, because input tokens do not feel like spending. You send a document, the model answers, the answer is good.

But you pay for every token of what you sent, and on real-world input a large share is not content. Feed a web page in raw and navigation, scripts and footers are the majority of it. Feed in a whole conversation history when only the last three turns matter and you pay for all of it, every turn, forever.

Measure before you optimise:

curl -X POST "https://twotic.dev/v1/token-counter/count"   -H "Authorization: Bearer $TWOTIC_KEY"   -H "Content-Type: application/json"   -d '{"text": "<your text>", "model": "claude-haiku-3.5"}'

# Change model and run it again to compare cost for the same input.

const price = (text: string, model = "claude-sonnet-4") =>
  fetch("https://twotic.dev/v1/token-counter/count", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.TWOTIC_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ text, model }),
  }).then((r) => r.json())

const raw = await price(rawHtml)
const clean = await price(extractedText)

console.log(raw.inputTokens, "→", clean.inputTokens)

const price = (text, model = "claude-sonnet-4") =>
  fetch("https://twotic.dev/v1/token-counter/count", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.TWOTIC_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ text, model }),
  }).then((r) => r.json())

const raw = await price(rawHtml)
const clean = await price(extractedText)

console.log(raw.inputTokens, "→", clean.inputTokens)

import os
import requests

def price(text: str, model: str = "claude-sonnet-4") -> dict:
    return requests.post(
        "https://twotic.dev/v1/token-counter/count",
        headers={"Authorization": f"Bearer {os.environ['TWOTIC_KEY']}"},
        json={"text": text, "model": model},
    ).json()

raw = price(raw_html)
clean = price(extracted_text)

print(raw["inputTokens"], "→", clean["inputTokens"])

A ten-to-one reduction on an article page is ordinary. If you are sending web content to a model without an extraction step, that is almost certainly your single biggest line item — see preparing web content for a RAG pipeline for the method.

2. The model you defaulted to

Most codebases pick a model once, early, when the task was harder than it turned out to be — and never revisit it.

The spread is not small. Between a frontier model and a fast one, the same workload can differ by an order of magnitude for output a user cannot distinguish. Classification, extraction, short summarisation and routing rarely need the top of the range.

Price the same input against several models and compare:

const models = ["claude-sonnet-4", "claude-haiku-3.5", "gpt-4o-mini"]
const quotes = await Promise.all(models.map((m) => price(prompt, m)))

for (const q of quotes) {
  console.log(q.model, q.cost.total, q.fitsInContext ? "" : "(does not fit)")
}

const models = ["claude-sonnet-4", "claude-haiku-3.5", "gpt-4o-mini"]
const quotes = await Promise.all(models.map((m) => price(prompt, m)))

for (const q of quotes) {
  console.log(q.model, q.cost.total, q.fitsInContext ? "" : "(does not fit)")
}

models = ["claude-sonnet-4", "claude-haiku-3.5", "gpt-4o-mini"]
quotes = [price(prompt, m) for m in models]

for q in quotes:
    print(q["model"], q["cost"]["total"], "" if q["fitsInContext"] else "(does not fit)")

The right shape is usually a cheap model for the bulk of calls and an expensive one for the cases that actually need it — routed by a rule you can state, not by a default nobody chose.

3. Retries and loops nobody bounded

An agent that decides its own next step will occasionally decide to keep going. A retry wrapper with no ceiling turns one failing call into fifty. Neither shows up as a spike in feature usage, which is why they survive so long.

Three guards, all cheap:

• A hard iteration cap on any agent loop, not a hopeful stopping condition.
• A retry ceiling with exponential backoff, and never retry a 4xx — it will fail identically.
• A prepaid balance rather than a metered account, so the maximum possible surprise is the amount you already added. This is why we price per request against prepaid credit: an agent cannot spend money you have not put there.

4. Context you send but never use

Long conversations grow monotonically unless something trims them. By turn thirty you are paying to resend turn one on every request.

The fix is not clever summarisation, at least not first. It is checking:

const { inputTokens, contextUsedPercent, fitsInContext } = await price(conversation)

if (contextUsedPercent > 60) {
  // Summarise the oldest turns, keep the most recent verbatim.
}
if (!fitsInContext) {
  // Do not send. This call will fail and you will still wait for it.
}

const { inputTokens, contextUsedPercent, fitsInContext } = await price(conversation)

if (contextUsedPercent > 60) {
  // Summarise the oldest turns, keep the most recent verbatim.
}
if (!fitsInContext) {
  // Do not send. This call will fail and you will still wait for it.
}

quote = price(conversation)

if quote["contextUsedPercent"] > 60:
    # Summarise the oldest turns, keep the most recent verbatim.
    ...
if not quote["fitsInContext"]:
    # Do not send. This call will fail and you will still wait for it.
    ...

A context overflow is the worst possible outcome: you pay input tokens, wait the full latency, and get an error. Checking first costs a fraction of a cent and turns it into a branch in your own code.

Put the numbers where decisions happen

The pattern that works is not a monthly cost review. It is making cost visible at the moment someone writes the code:

• Log estimated cost per request alongside latency. Both are performance.
• Assert on it in tests. If a prompt change triples token count, that should fail CI, not surface in a month.
• Show it to users where they trigger the work. An up-front estimate turns an opaque bill into an informed choice and reduces the support load that opacity creates.

What estimates can and cannot do

Any pre-call estimate is an approximation. Exact counting requires each provider's own byte-pair vocabulary — tens of megabytes, correct for one model and wrong for every other, and loading one per request makes the check more expensive than the call it protects.

A good approximation models what BPE does: words cost roughly one token plus one per four characters beyond the first four, punctuation and CJK characters tokenise individually, digits split more finely than letters. On ordinary prose that lands within a few percent.

That is the right accuracy for budgeting, model selection and context-fit checks. It is not the right accuracy for reconciling an invoice line by line, and anything claiming otherwise is overselling. Use the provider's reported usage for accounting, and estimates for decisions.

The short version

Extract before you send. Price your models against each other rather than defaulting. Bound every loop. Check context fit before the call, not after. Put the number next to the code.

Most of the saving is in the first one.

What you can build with this

A cost assertion in your test suite. A test that prices your production prompts and fails if any grows beyond a threshold. Prompt changes are usually made for quality reasons by someone not thinking about tokens, and this catches the tripling on the pull request instead of in a monthly review.

A model router. A small function that estimates the input, applies a rule about task difficulty, and picks the model. Even a crude rule beats a hardcoded default, and having it in one place means you can change your mind about the tradeoff without touching twenty call sites.

A pre-flight estimate in your own product's UI. If your users trigger expensive work, show them what it will cost before they commit. It converts a surprising bill into an informed decision, and it removes most of the support conversations that opacity creates.

A context-budget guard. Before any call, check the fit. If it will not fit, summarise or chunk instead of sending. This is a handful of lines and it eliminates an entire class of failure that otherwise costs you input tokens and latency to discover.

The first one is the highest leverage, because it keeps working after you stop paying attention.

Related reading

• Preparing web content for a RAG pipeline — the extraction step that removes most of your input cost.
• Chunking strategies for retrieval — retrieved context is input you pay for on every single call.
• Connecting an API to an AI agent with MCP — where unbounded loops actually happen, and how the connection is metered.
• Token Counter — the endpoint used throughout this guide.