LLM Cost-Performance Analysis

Model Selection Is a Budget Decision Wearing a Technical Costume

"Which model should we use" almost always gets answered as if it were purely a quality question. It isn't. Every model choice is a point on a cost-performance curve, and the frontier model that wins your internal eval by a few points can be several times more expensive per call than the option that would have shipped the feature just fine. Nobody notices at ten requests a day. Everybody notices at ten thousand.

Frontier vs Mini/Flash: What You're Actually Trading

Every major lab now ships two tiers. A frontier tier — GPT-4o-class, Claude Fable 5 or Opus 5-class, Gemini Pro — built for the hardest reasoning, the longest context, the most ambiguous instructions. And a smaller, faster tier — GPT-4o-mini-class, Gemini Flash, Claude Haiku-class — priced roughly an order of magnitude cheaper per token, with latency to match. The mini tier isn't a worse version of the frontier model in every way that matters. It's tuned for a different job: high-volume, lower-ambiguity tasks where the frontier model's extra reasoning capacity mostly goes unused. Classification, extraction, short rewrites, simple routing decisions — a mini model handles these at a quality gap that's often smaller than the price gap suggests. Multi-step reasoning, long-document synthesis, anything where getting it subtly wrong is expensive to catch later — that's where the frontier tier earns its price.

Model Routing and Cascading

Once you accept that no single model is right for every call in your system, routing follows naturally. Cascading is the specific pattern worth knowing: try the cheap model first, and only escalate to the expensive one when the cheap model's output looks unreliable. The trick is defining "looks unreliable" in a way that's cheap to check — a confidence score if the model exposes one, a schema validation failure, a length or format check, or a cheap secondary classifier trained specifically to flag "this needs the bigger model." Get that check wrong and you either escalate everything (no savings) or escalate nothing (silent quality loss), so it's worth treating as a real component with its own tests, not an afterthought bolted onto the routing function.

flowchart TD
    Req["Incoming request"] --> Cheap["Cheap/fast model
(mini or flash tier)"]
    Cheap --> Conf{"Confident?
(schema valid, score high,
task in scope)"}
    Conf -->|Yes| Return["Return result"]
    Conf -->|No| Expensive["Escalate to frontier model"]
    Expensive --> Return

    style Cheap fill:#059669,color:#fff,stroke:#047857
    style Expensive fill:#dc2626,color:#fff,stroke:#b91c1c
    style Return fill:#4f46e5,color:#fff,stroke:#4338ca
      

async function classifyTicket(ticket: string): Promise<Classification> {
  const cheap = await callModel('gpt-4o-mini', buildPrompt(ticket));

  // Escalate on structural failure or low self-reported confidence —
  // not on vibes. Define this threshold from labeled data, not intuition.
  const needsEscalation =
    !isValidSchema(cheap.output) ||
    cheap.confidence < 0.7 ||
    ticket.length > 4000; // long/ambiguous tickets go straight to the bigger model

  if (!needsEscalation) return cheap.output;

  const frontier = await callModel('gpt-4o', buildPrompt(ticket));
  return frontier.output;
}

In practice, a well-tuned cascade routes the large majority of traffic to the cheap model and reserves the expensive one for the tail that actually needs it — which is exactly the shape of most production workloads: mostly routine, occasionally hard.

Batch API Pricing

Most major providers offer an asynchronous batch endpoint: submit a large set of requests, get results back within a window (often hours rather than seconds), at a meaningful discount off synchronous pricing. This fits offline evaluation runs, bulk classification of a historical dataset, nightly enrichment jobs, anything where nobody is staring at a spinner waiting for the answer. It does not fit user-facing chat, or anything where a human is on the other end expecting a response inside a few seconds. The mistake is assuming batch pricing applies broadly when it actually applies to a narrower slice of your workload than the discount makes tempting — check which parts of your pipeline can genuinely tolerate the latency before restructuring anything around it.

Prompt Caching as a Cost Lever

If your requests share a large, stable prefix — a long system prompt, a set of few-shot examples, a document you're asking repeated questions about — prompt caching (covered in more depth in the context-windows article) discounts the repeated portion on every call after the first. It stacks cleanly with routing and cascading: cache the shared prefix once, then route the variable part of the request through whichever tier makes sense.

Worked Example: Support-Ticket Triage

Say you're triaging 10,000 support tickets a day — category, priority, routing — averaging roughly 500 input tokens and 150 output tokens per ticket. These numbers are illustrative, not current pricing; plug in your provider's actual rate when you do this for real.

The mini-only run is roughly fourteen times cheaper than frontier-only in this illustration, and the cascade lands close to it while keeping the escalation path for the tickets that are actually ambiguous — angry customers, multi-issue tickets, anything where a misrouted ticket costs more in support time than the model call ever would. Multiply the daily numbers by 30 for a monthly estimate, and notice how much the answer depends on that escalation rate. It's worth measuring on real traffic before committing to a number in a planning doc.

Key Takeaways