In this article
Problem: Pure vector RAG retrieves semantically similar chunks but cannot answer questions that require multi-hop reasoning across entities — e.g., "What are all the downstream services affected by this database migration?"
Solution: Build a knowledge graph alongside vector embeddings. At query time, use graph traversal to find related entities, then combine graph context with vector-retrieved chunks for a richer LLM context.
Implementation (Microsoft GraphRAG approach):
- Entity extraction: Run LLM over each document chunk to extract entities (names, concepts, relationships) and build a graph (Neo4j, in-memory networkx)
- Community detection: Apply Leiden algorithm to cluster related entities into communities
- Community summaries: Generate LLM summaries for each community — store these as searchable documents
- Query serving:
- Global queries (e.g., "What themes dominate this corpus?"): Retrieve community summaries, aggregate with LLM map-reduce
- Local queries (e.g., "Who is connected to entity X?"): Graph traversal + vector search for specific entity context
Trade-Offs:
- ✔ Pro: Answers complex relational queries that vector RAG cannot
- ✔ Pro: Community summaries enable high-level "what's in this corpus" questions
- ✖ Con: Expensive to build — entity extraction requires many LLM calls over all documents
- ✖ Con: Graph maintenance on document updates requires delta processing
When To Use: Large, entity-rich corpora (codebases, legal documents, knowledge management) where relational queries are common. When to avoid: Small document sets or corpora where vector RAG answers queries well.
Related Articles
designing rag systems that actually scale