Context Engineering
Treat the window as a budget and decide what goes in on every turn. Bigger windows did not solve this: stale and irrelevant context makes agents slower, costlier and less reliable.
AI Wisdom reference · Radar · 62 entries
The tools, techniques and platforms we actually run in production, each placed on a ring: Adopt, Trial, Assess or Hold. Names are open to everyone; the reasoning behind each call is part of Pro.
Last reviewed · 29 entries new or moved this review
How to read it
Each dot is numbered; select one to jump to its entry below.
Adopt is open to everyone. Pro unlocks the reasoning behind every Trial, Assess and Hold call: the trade-offs, costs and failure modes behind each ring.
Architecture, prompting and agent patterns
Treat the window as a budget and decide what goes in on every turn. Bigger windows did not solve this: stale and irrelevant context makes agents slower, costlier and less reliable.
Give an agent only the autonomy its task needs, and enforce policy in code at the tool boundary. Prompt rules can be argued with. Code cannot.
A short file of exact commands and non-obvious rules, now a shared format across coding tools. The cheapest improvement you can make to any coding agent.
Run model-written code with egress denied by default and no long-lived secrets. Permission prompts alone are not a boundary once a prompt injection is in play.
The default architecture for grounding LLM responses in private or current knowledge. Mature tooling, well-understood failure modes. Bigger context windows changed how much you retrieve, not whether you should.
Schema-constrained decoding is native across current GPT, Claude and Gemini models. Define the schema once, validate on the way in, and stop parsing prose.
LLMs calling typed external functions. Reliable across every current frontier family and the foundation for agent systems.
One choke point for every model call: auth, rate limits, semantic caching, cost attribution and provider failover. The first piece of infrastructure we add once more than one team calls an LLM.
Cache LLM responses by semantic similarity of prompts, usually at the gateway with Redis or another vector-capable store. It can cut cost and latency, but a similar question can get the wrong answer, so tune the threshold per route and scope the cache by tenant.
A versioned eval set that runs in CI on every prompt, model or retrieval change, with LLM-as-judge for open-ended answers and deterministic checks where possible. Model upgrades are now monthly; this is how you take them safely.
Validate what goes into the model and what comes out: prompt-injection screening, PII redaction, schema and policy checks on outputs. Cheap to add early, expensive to retrofit after an incident.
Model providers and AI frameworks
GPT-6 Astra for the hardest reasoning, GPT-6.1 Sol as the balanced default, GPT-6 Luna for high-volume paths. Strong ecosystem and predictable pricing.
Claude Opus 5.5 and Sonnet 5.5 lead our coding and agentic workloads; Haiku 5.5 covers fast, cheap paths. Excellent tool use and long-context behaviour.
Gemini 4 Argon for the hardest multimodal and long-context work; Gemini 3.8 Flash and 3.5 Flash-Lite suit high-throughput workloads. Argon and Flash prices both rise in the next few months, so re-run your cost sums.
OpenAI models with private networking, regional data residency and Entra ID. Model availability trails openai.com: GPT-5.6 is the current Azure deployment. Plan upgrades around Azure’s rollout, not OpenAI’s launch date.
Streaming-first AI primitives for Next.js / React with one interface across providers. Excellent DX for UI-layer AI features and tool calling.
Vector stores, observability, serving and dev tooling
The standard way to expose tools and data to agents across vendors, now stateless since the 2026-07-28 revision and hosted by the Agentic AI Foundation. Build a tool once as an MCP server and every MCP-capable client can use it. Treat servers as privileged code: scope credentials and log every call.
Vector search inside Postgres. Removes a separate vector DB at moderate scale, and runs on Neon, Supabase or Azure Database for PostgreSQL.
Managed vector database. Serverless tier removed the ops burden. Strong for production RAG at large scale.
Emit model calls, tool calls and retrieval steps as standard OTel spans (GenAI semantic conventions). Your traces then work in any backend, and you are not locked into one LLM-ops vendor. The conventions are still in Development status, so keep attribute names in one wrapper.
Open-source LLM ops: tracing, evals, prompt management. Self-hostable, and part of ClickHouse since January 2026 with its core still MIT-licensed. Our recommended observability layer for production AI.
Schema-first validation of model output: Instructor (Pydantic) in Python, Zod in TypeScript, both with retries on invalid output. The thin layer we put over native structured outputs.
Agentic coding tools that read the repo, run commands and open PRs. Useful in the hands of experienced engineers. The bottleneck has moved to review, so keep diffs small and give agents tests to run.
Where each language and IaC tool earns its place
Dominant language for AI engineering. Best ecosystem for model inference, evals, ML tooling and data pipelines.
For product and edge AI features: streaming UI, tool calling, MCP servers and the Vercel AI SDK. TypeScript is the right choice here.
Model endpoints, vector stores, gateways and keys belong in code like everything else. Reusable modules make a new AI environment a pull request, not a week of portal clicks.
Radar entries map to topics in the universe. Read the article, see the architecture, then build it in the five-mode loop.