LLM Model LandscapeIntermediate

Hallucination Type Taxonomy

The three kinds of hallucination, a test set that includes questions with no answer, claim-by-claim faithfulness scoring, and what lowers the rate.

By Amit UpadhyayPublished 13 Oct 202611 min

Hallucination (confabulation) is when an LLM generates plausible-sounding but incorrect information. Types: factual hallucination (wrong facts), attribution hallucination (fabricated sources), and faithfulness hallucination (answers not grounded in provided context). Detection: reference-based (compare to ground truth), RAG faithfulness (does answer follow from retrieved context?), semantic entailment. RAGAS framework measures faithfulness, answer relevancy, and context precision.

The pipeline

Each stage in order — click any step to read what it does.

Interactive Pipeline Flow

UnderstandImplementOptimise

🔍 Types — factual/faithful/attr

Hallucination types: closed-domain (answer not grounded in provided context — worst for RAG), open-domain (factually wrong statements), attribution (invents sources), consistency (contradicts itself).

Architecture diagrams

Multi-layer hallucination detection for RAG systems.

Loading diagram...

Key design decisions

The trade-offs worth knowing before you build this.

🔗RAGAS faithfulness is your primary RAG health metric

A RAGAS faithfulness score below 0.80 means 20%+ of your RAG answers contain claims not grounded in retrieved context. This is the most dangerous type of hallucination in production RAG systems.

🛡️Force citations to constrain hallucination

System prompt: "For each factual claim, cite the source sentence in brackets [1]". Models are much less likely to fabricate claims when forced to provide citations — and citations make hallucination auditable.

⚠️Monitor production hallucination rate, not just eval set

Eval set hallucination rates can be low while production rates are high (distribution shift). Sample 100 production responses weekly, run faithfulness eval. Alert on rising trend.

Next: Animate →

hallucination faithfulness grounding RAG evaluation

← PreviousGemini: Google's Multimodal AI Next →Evals for Choosing a Model

Continue reading

All LLM Model Landscape articles →

LLM Model Landscape11 min

GPT: OpenAI's Model Family in Practice

The GPT-6 tiers, what reasoning effort replaced, when to use the Responses API over Chat Completions, how the bill is calculated, and what is different on Azure.

LLM Model Landscape11 min

Claude: Anthropic's Model Family in Practice

The four Claude tiers, adaptive thinking and effort, the tool-use rules that differ from other providers, prompt caching, and where you can run it.

LLM Model Landscape11 min

Llama: Open-Weight Models for Production

The Llama lineup and licence, serving with Ollama and vLLM, GPU memory sizing, and an honest look at when a hosted API is still the better choice.

Discussion

LLM Model LandscapeIntermediate

Hallucination Type Taxonomy

The three kinds of hallucination, a test set that includes questions with no answer, claim-by-claim faithfulness scoring, and what lowers the rate.

By Amit UpadhyayPublished 13 Oct 202611 min

The pipeline

Each stage in order — click any step to read what it does.

Interactive Pipeline Flow

UnderstandImplementOptimise

🔍 Types — factual/faithful/attr

Architecture diagrams

Multi-layer hallucination detection for RAG systems.

Loading diagram...

Key design decisions

The trade-offs worth knowing before you build this.

🔗RAGAS faithfulness is your primary RAG health metric

A RAGAS faithfulness score below 0.80 means 20%+ of your RAG answers contain claims not grounded in retrieved context. This is the most dangerous type of hallucination in production RAG systems.

🛡️Force citations to constrain hallucination

⚠️Monitor production hallucination rate, not just eval set

Eval set hallucination rates can be low while production rates are high (distribution shift). Sample 100 production responses weekly, run faithfulness eval. Alert on rising trend.

Next: Animate →

hallucination faithfulness grounding RAG evaluation

← PreviousGemini: Google's Multimodal AI Next →Evals for Choosing a Model

Continue reading

All LLM Model Landscape articles →

LLM Model Landscape11 min