AI Wisdom reference · Patterns · 17 entries
Patterns for production AI
Compare reusable approaches by the problem they solve, when they fit, and what they cost.
17 of 17 patterns
Reliability & safety
Circuit Breaker for LLMs
Prevent cascading failures by temporarily stopping requests to a failing LLM provider and automatically recovering when it stabilises.
WHEN IT FITSAny system making more than ~100 LLM requests/day that needs high availability.
Retrieval
Context Compression
Compress conversation history or retrieved documents to fit within context window limits while preserving critical information.
WHEN IT FITSLong multi-turn conversations, multi-document RAG, agent loops with many tool outputs.
Delivery & evaluation
Evaluation-Driven Development (EDD)
Build eval suites before building AI features, then measure quality continuously — the test-driven development of AI systems.
WHEN IT FITSEvery production AI feature. Non-negotiable for customer-facing AI.
Reliability & safety
Fallback Chain
Route LLM requests through a prioritised list of models/providers, falling back to alternatives on rate limits, errors, or latency SLO violations.
WHEN IT FITSAny production system with SLA requirements and more than 1000 RPD.
Retrieval
GraphRAG
Augment vector search RAG with a knowledge graph layer to enable multi-hop reasoning over entity relationships.
WHEN IT FITSLarge, entity-rich corpora (codebases, legal documents, knowledge management) where relational queries are common.
Reliability & safety
Guardrail Sandwich
Wrap every LLM call with input validation and output validation layers to enforce safety, quality, and schema compliance.
WHEN IT FITSAny production, user-facing LLM application. Non-negotiable for regulated industries.
Agents
Human-in-the-Loop (HITL)
Insert human approval checkpoints in agent workflows for high-stakes, irreversible, or high-uncertainty actions.
WHEN IT FITSCustomer-facing agents with real-world consequences; regulated industries.
Cost & latency
Intent Router
Classify user intent first, then route to specialised handlers — cheaper models for simple tasks, expensive models for complex ones.
WHEN IT FITSMulti-purpose chatbots, customer-facing assistants, high-volume APIs.
Agents
Plan and Execute
Separate planning (powerful model) from execution (cheaper models) for complex multi-step tasks, reducing cost while maintaining quality.
WHEN IT FITSResearch tasks, code generation with multiple files, data analysis pipelines, content creation workflows.
Delivery & evaluation
Prompt Versioning
Manage prompts as code artifacts with version control, review, and rollback — treating prompt changes with the same rigour as code changes.
WHEN IT FITSAny production system where prompt quality matters. Essential for teams with more than one person editing prompts.
Agents
ReAct Agent
Interleave Thought → Action → Observation cycles so the LLM reasons before invoking tools and adapts based on results.
WHEN IT FITSAny agent that uses tools and must make sequential decisions based on intermediate results.
Agents
Reflection Pattern
Have the agent critique its own output and iteratively refine it before returning the final result.
WHEN IT FITSCode generation, long-form content, technical explanations, and any high-stakes output where quality trumps speed.
Cost & latency
Semantic Cache
Cache LLM responses by semantic similarity of the prompt rather than exact string match, reducing cost and latency for similar queries.
WHEN IT FITSHigh-volume, repetitive query patterns — chatbots, search, customer support.
Delivery & evaluation
Streaming Pipeline
Stream LLM tokens through a processing pipeline to the client, showing output progressively and enabling real-time safety checks.
WHEN IT FITSAll user-facing chat and content generation interfaces.
Reliability & safety
Tenant Isolator
Ensure strict data isolation between tenants in multi-tenant LLM applications through namespace partitioning and context boundaries.
WHEN IT FITSAny B2B SaaS with customer data in RAG pipelines; regulated industries.
Retrieval
Tiered Retrieval
Combine keyword search (BM25), semantic search (vectors), and re-ranking in a multi-stage pipeline for optimal RAG quality.
WHEN IT FITSProduction RAG systems where retrieval quality directly impacts user experience.
Cost & latency
Token Budget Gate
Enforce per-user, per-tenant, or per-request token limits to prevent runaway costs in multi-tenant LLM systems.
WHEN IT FITSAny multi-tenant or user-facing LLM application, especially SaaS products.