Architecture decisions, explained

Multi-Tenant Data Isolation Strategy for RAG Pipelines

As we expand from a single-tenant learning platform to supporting teams and organisations, we need to isolate content data between tenants in our RAG pipeline. A data leak where Tenant A's proprietary content appears in Tenant B's responses would be a critical trust and compliance failure.

Content Database Migration — Legacy PostgreSQL to Neon

Our platform content (topics, domains, glossary terms, patterns, decisions) had been stored in a legacy PostgreSQL setup. As our content grows and we add AI features requiring vector search and serverless edge deployments, we need to re-evaluate our database choice. Key drivers: better Vercel Edge compatibility, cost predictability, and branching support for content staging.

Testing Strategy for LLM-Powered Features

Traditional unit and integration tests are insufficient for AI features because LLM outputs are non-deterministic. We need a testing strategy that gives us confidence in AI feature quality without requiring expensive human review of every change.

Tool Integration Protocol — MCP vs Custom Tool Calling

Our AI agents need to call external tools — content search, user data lookup, web scraping, code execution. We must decide whether to build custom tool integrations using OpenAI function calling / Vercel AI SDK tools, or adopt the Model Context Protocol (MCP) as a standard integration layer.

Agent Memory Architecture — In-Context vs External Memory

Our AI agents need memory to maintain context across multi-step tasks and across user sessions. We need to decide how to implement short-term (within session) and long-term (cross-session) memory, balancing recall quality, cost, complexity, and privacy.

AI Observability Platform — LangSmith vs Phoenix vs Custom OpenTelemetry

We need an observability platform to trace LLM calls, evaluate output quality, debug RAG retrieval, and monitor production AI quality over time. Without observability, we are flying blind on LLM quality, cost, and reliability.

LLM Streaming Architecture — SSE vs WebSocket vs Polling

Our AI chat and content generation features need to stream LLM responses to users rather than waiting for full completion. We need to choose between Server-Sent Events (SSE), WebSockets, and polling for delivering streaming tokens from the LLM to the browser.

Agentic Workflow Framework — LangGraph vs Direct Implementation

We are building agentic features — multi-step AI workflows that plan, use tools, and iterate. We need to decide whether to use LangGraph (stateful graph-based orchestration), build directly on the model APIs with lightweight wrappers, or adopt another framework like AutoGen.

Embedding Model Selection

We need an embedding model for our RAG pipeline. The model determines retrieval quality, vector dimensions (affects storage), and API cost.

Orchestration Framework — LangChain vs Semantic Kernel

We need an orchestration framework for building RAG pipelines, agent workflows, and tool-calling chains. The two leading options are LangChain (Python/JS) and Semantic Kernel (C#/.NET/Python).

Primary LLM Selection — GPT-4.1 vs Claude 3.5

We need to select a primary LLM for our platform's AI features (article assistance, search, code generation). The choice affects cost, quality, and vendor dependency.

Vector Database Selection

We need a vector database for our RAG pipeline. Options range from managed services (Pinecone, Weaviate Cloud) to integrated solutions (pgvector in our existing PostgreSQL).

RAG vs Fine-Tuning for Domain Knowledge

We need to give our LLM domain-specific knowledge about AI engineering tools, patterns, and best practices. The model must provide accurate, up-to-date answers grounded in our curated content.