On this page
How it works
flowchart TD
D[Documents] --> C[Chunks]
C --> E[Embeddings and vector store]
Q[Question] --> R[Retrieve relevant chunks]
E --> R
R --> P[Question plus retrieved context]
P --> L[LLM]
L --> A[Grounded answer]The problem
We need to give our LLM domain-specific knowledge about AI engineering tools, patterns, and best practices. The model must provide accurate, up-to-date answers grounded in our curated content.
What we weighed
RAG (Retrieval-Augmented Generation)
Pros
- +No training required — content updates are instant
- +Responses are grounded in cited sources
- +Works with any foundation model
- +Lower cost for content updates
Cons
- −Retrieval quality is a bottleneck
- −Added latency for embedding + search
- −Requires vector database infrastructure
Fine-Tuning
Pros
- +Model internalises domain vocabulary and reasoning
- +No retrieval latency at inference time
- +Better for style/tone adaptation
Cons
- −Expensive to retrain on content updates
- −Risk of hallucination (no source citations)
- −Vendor lock-in to specific model families
- −Months of data curation before training
Hybrid (RAG + Fine-Tuned)
Pros
- +Best accuracy: model understands domain + retrieves specifics
- +Citations available from RAG layer
Cons
- −Highest complexity and cost
- −Overkill for our current content volume
What we chose
We chose RAG as the primary approach. Our content updates weekly, making fine-tuning impractical. RAG provides source citations which align with our editorial transparency values. We will revisit Hybrid when content volume exceeds 1000 articles.
What it committed us to
- •Must invest in chunking strategy and embedding pipeline
- •Need vector database (chose pgvector via Neon for simplicity)
- •Retrieval quality must be monitored and improved continuously
- •System prompt must instruct model to only use retrieved context