The Demo Worked. Then Real Users Arrived.
Connecting a model to a search box takes an afternoon, and the demo is usually convincing. Production is where it changes: the bill grows several times faster than the forecast, latency swings with load, answers get quietly worse as the data grows, and when one provider has a bad hour the whole feature goes down with it.
None of that is the model's fault. It is the system around it, or the lack of one. This is the long version of how I would build that system: the request path, routing, retrieval, cost controls, observability and failure handling, with the .NET wiring included.
- You built a RAG prototype but results feel inconsistent in production
- Azure AI Search returns data, but relevance is weak
- You're unsure how to design embeddings for structured SaaS data
- You don't know how to handle millions of existing tenant records
- You have no visibility into what the system is actually doing