Model Serving Quiz
Azure OpenAI + AI Search + embeddings — real-world architecture for production AI systems, including legacy data, orchestration, hybrid retrieval, cost control, and failure modes.
Model Serving — Knowledge Check
Ten questions on PagedAttention, continuous batching, speculative decoding, quantisation, KV cache, and hosting decisions.
Apply the decisions to the scenario
Each question tests one decision from Read, not a definition. The explanation names the decision it checks.
1. Three apps each call the model API directly with their own keys. Finance asks for last month’s spend by team. What is missing?
2. You plan to route 70% of questions to a small model. What do you need before turning it on?
3. After a policy update, users still get the old answer from the cache. What was missing?
4. A user’s script calls the assistant 40,000 times overnight. What limits the damage?
5. Your provider is overloaded for 20 minutes. What keeps the assistant answering?
Continue reading
All AI Engineering articles →Guardrails: Input/Output Safety in Production
Input, output, schema, tool, and cost guardrails — the layered defence system that catches LLM failures before they reach a user or downstream system.
Embeddings: From Theory to Production Choice
Convert text, images, and code into dense vectors that capture semantic meaning — the foundational primitive of modern AI systems, and the choice that dominates RAG quality.
Token Economics: Understanding and Optimizing LLM Costs
A practical guide to understanding token pricing, measuring real costs, and implementing optimization strategies — caching, prompt compression, model routing.
Discussion
Sign in to share your feedback and join the discussion.