Problem: In multi-tenant RAG or agent systems, one tenant's data must never leak into another tenant's LLM responses.

Solution: Enforce isolation at every layer: vector store namespaces, prompt context boundaries, API key separation, and output validation.

Implementation:

  1. Vector store: Use collection-per-tenant or metadata filtering with tenant_id on every query
  2. Prompt boundary: Include tenant context only; never mix tenants in a single LLM call
  3. API keys: Per-tenant API keys or JWT-scoped access with tenant claims
  4. Output validation: Post-process LLM output to detect and redact cross-tenant data leaks
  5. Audit logging: Log all retrieval sources per request for compliance

Trade-Offs:

  • Pro: Strong data isolation; compliance-ready (SOC 2, GDPR)
  • Pro: Per-tenant usage tracking and billing
  • Con: Collection-per-tenant doesn't scale beyond ~1000 tenants in most vector DBs
  • Con: Metadata filtering adds query latency (~10-20ms)

When To Use: Any B2B SaaS with customer data in RAG pipelines; regulated industries. When to avoid: Single-tenant applications or internal tools with shared data.