In this article
Problem: In multi-tenant RAG or agent systems, one tenant's data must never leak into another tenant's LLM responses.
Solution: Enforce isolation at every layer: vector store namespaces, prompt context boundaries, API key separation, and output validation.
Implementation:
- Vector store: Use collection-per-tenant or metadata filtering with tenant_id on every query
- Prompt boundary: Include tenant context only; never mix tenants in a single LLM call
- API keys: Per-tenant API keys or JWT-scoped access with tenant claims
- Output validation: Post-process LLM output to detect and redact cross-tenant data leaks
- Audit logging: Log all retrieval sources per request for compliance
Trade-Offs:
- ✔ Pro: Strong data isolation; compliance-ready (SOC 2, GDPR)
- ✔ Pro: Per-tenant usage tracking and billing
- ✖ Con: Collection-per-tenant doesn't scale beyond ~1000 tenants in most vector DBs
- ✖ Con: Metadata filtering adds query latency (~10-20ms)
When To Use: Any B2B SaaS with customer data in RAG pipelines; regulated industries. When to avoid: Single-tenant applications or internal tools with shared data.
Related Articles
designing rag systems that actually scale