Token pricing, latency tiers, frontier vs economy models, caching.

Cost-Performance — Choosing Models for Your Budget

LLM cost is measured in dollars per million tokens (input + output). Frontier models (GPT-4o, Claude Sonnet, Gemini Pro) offer highest quality at ~$2-15/M tokens. Economy models (GPT-4o-mini, Claude Haiku, Gemini Flash) offer 80-90% quality at 1/10th the price. Latency: economy models are also 2-5x faster. The optimal strategy: use economy models as default, route complex tasks to frontier, implement prompt caching for repeated contexts.

Interactive Pipeline Flow

UnderstandImplementOptimise

📋 Task — classify complexity

Classify task complexity: simple (summarise, classify, extract) → economy model. Complex (multi-step reasoning, coding, nuanced analysis) → frontier model.

Architecture Diagrams

Structuring prompts to maximise cache hit rate.

Loading diagram...

Key Design Decisions

💰Economy models handle 80% of tasks at 10% cost

GPT-4o-mini and Claude Haiku handle summarisation, classification, extraction, and light reasoning at 85-90% quality for 1/10th the price. Start with economy, escalate on failure.

📦Prompt caching can cut costs by 50-90%

If your system prompt is 2000 tokens and you make 100k requests/day, caching saves 2000 tokens × 100k requests × $0.0025 = $500/day. Structure prompts with static content first.

📊Use Batch API for non-real-time workloads

Batch document processing, report generation, and evaluation runs don't need sub-second responses. Batch API (50% discount) + prompt caching can reduce costs by 70-90% for offline workloads.

Discussion