AI Engineering Patterns
Reusable architectural patterns for production AI systems. Each pattern includes a problem statement, solution blueprint, trade-offs, and when-to-use guidance.
17 patterns documented
Circuit Breaker for LLMs
Prevent cascading failures by temporarily stopping requests to a failing LLM provider and automatically recovering when it stabilises.
View pattern →Context Compression
Compress conversation history or retrieved documents to fit within context window limits while preserving critical information.
View pattern →Evaluation-Driven Development (EDD)
Build eval suites before building AI features, then measure quality continuously — the test-driven development of AI systems.
View pattern →Fallback Chain
Route LLM requests through a prioritised list of models/providers, falling back to alternatives on rate limits, errors, or latency SLO violations.
View pattern →GraphRAG
Augment vector search RAG with a knowledge graph layer to enable multi-hop reasoning over entity relationships.
View pattern →Guardrail Sandwich
Wrap every LLM call with input validation and output validation layers to enforce safety, quality, and schema compliance.
View pattern →Human-in-the-Loop (HITL)
Insert human approval checkpoints in agent workflows for high-stakes, irreversible, or high-uncertainty actions.
View pattern →Intent Router
Classify user intent first, then route to specialised handlers — cheaper models for simple tasks, expensive models for complex ones.
View pattern →Plan and Execute
Separate planning (powerful model) from execution (cheaper models) for complex multi-step tasks, reducing cost while maintaining quality.
View pattern →Prompt Versioning
Manage prompts as code artifacts with version control, review, and rollback — treating prompt changes with the same rigour as code changes.
View pattern →ReAct Agent
Interleave Thought → Action → Observation cycles so the LLM reasons before invoking tools and adapts based on results.
View pattern →Reflection Pattern
Have the agent critique its own output and iteratively refine it before returning the final result.
View pattern →Semantic Cache
Cache LLM responses by semantic similarity of the prompt rather than exact string match, reducing cost and latency for similar queries.
View pattern →Streaming Pipeline
Stream LLM tokens through a processing pipeline to the client, showing output progressively and enabling real-time safety checks.
View pattern →Tenant Isolator
Ensure strict data isolation between tenants in multi-tenant LLM applications through namespace partitioning and context boundaries.
View pattern →Tiered Retrieval
Combine keyword search (BM25), semantic search (vectors), and re-ranking in a multi-stage pipeline for optimal RAG quality.
View pattern →Token Budget Gate
Enforce per-user, per-tenant, or per-request token limits to prevent runaway costs in multi-tenant LLM systems.
View pattern →