In this article
Context
We need to give our LLM domain-specific knowledge about AI engineering tools, patterns, and best practices. The model must provide accurate, up-to-date answers grounded in our curated content.
Options Evaluated
RAG (Retrieval-Augmented Generation)
Pros
- +No training required — content updates are instant
- +Responses are grounded in cited sources
- +Works with any foundation model
- +Lower cost for content updates
Cons
- −Retrieval quality is a bottleneck
- −Added latency for embedding + search
- −Requires vector database infrastructure
Fine-Tuning
Pros
- +Model internalises domain vocabulary and reasoning
- +No retrieval latency at inference time
- +Better for style/tone adaptation
Cons
- −Expensive to retrain on content updates
- −Risk of hallucination (no source citations)
- −Vendor lock-in to specific model families
- −Months of data curation before training
Hybrid (RAG + Fine-Tuned)
Pros
- +Best accuracy: model understands domain + retrieves specifics
- +Citations available from RAG layer
Cons
- −Highest complexity and cost
- −Overkill for our current content volume
Decision
We chose RAG as the primary approach. Our content updates weekly, making fine-tuning impractical. RAG provides source citations which align with our editorial transparency values. We will revisit Hybrid when content volume exceeds 1000 articles.
Consequences
- •Must invest in chunking strategy and embedding pipeline
- •Need vector database (chose pgvector via Supabase for simplicity)
- •Retrieval quality must be monitored and improved continuously
- •System prompt must instruct model to only use retrieved context
Related Articles
designing rag systems that actually scale