AI Engineering Patterns

Solves

Prevent cascading failures by temporarily stopping requests to a failing LLM provider and automatically recovering when it stabilises.

Solves

Compress conversation history or retrieved documents to fit within context window limits while preserving critical information.

Solves

Build eval suites before building AI features, then measure quality continuously — the test-driven development of AI systems.

Solves

Route LLM requests through a prioritised list of models/providers, falling back to alternatives on rate limits, errors, or latency SLO violations.

Solves

Augment vector search RAG with a knowledge graph layer to enable multi-hop reasoning over entity relationships.

Solves

Wrap every LLM call with input validation and output validation layers to enforce safety, quality, and schema compliance.

Solves

Insert human approval checkpoints in agent workflows for high-stakes, irreversible, or high-uncertainty actions.

Solves

Classify user intent first, then route to specialised handlers — cheaper models for simple tasks, expensive models for complex ones.

Solves

Separate planning (powerful model) from execution (cheaper models) for complex multi-step tasks, reducing cost while maintaining quality.

Solves

Manage prompts as code artifacts with version control, review, and rollback — treating prompt changes with the same rigour as code changes.

Solves

Interleave Thought → Action → Observation cycles so the LLM reasons before invoking tools and adapts based on results.

Solves

Have the agent critique its own output and iteratively refine it before returning the final result.

Solves

Cache LLM responses by semantic similarity of the prompt rather than exact string match, reducing cost and latency for similar queries.

Solves

Stream LLM tokens through a processing pipeline to the client, showing output progressively and enabling real-time safety checks.

Solves

Ensure strict data isolation between tenants in multi-tenant LLM applications through namespace partitioning and context boundaries.

Solves

Combine keyword search (BM25), semantic search (vectors), and re-ranking in a multi-stage pipeline for optimal RAG quality.

Solves

Enforce per-user, per-tenant, or per-request token limits to prevent runaway costs in multi-tenant LLM systems.