Your RAG system answers badly on documents full of tables, and the retrieved chunks show rows cut off from their headers. What do you change first?
Why D is the answer
Use structure-aware chunking that keeps tables and headers together
The retrieved text is already broken, so nothing downstream can repair it. Fix ingestion first, then re-measure retrieval.
In detail
- The chunks are already broken when they are retrieved.
- Nothing downstream can repair a row cut off from its header.
- Fix ingestion first, then measure retrieval again.
- Inspect real chunks before you tune anything downstream.
Use case
An illustrative scenario, not a real incident.
A finance RAG answers badly because table rows are split from their headers. Chunking that keeps each table whole brings the right values into the context, and retrieval scores rise.
See it in different ways
flowchart LR A["Documents"] --> B["Chunk by structure"] B --> C["Index with metadata"] C --> D["Filter and search"] D --> E["Rerank"] E --> F["Answer with citations"]
mindmap
root((Retrieval))
Ingestion
Structure-aware chunks
Search
Keyword
Semantic
Ranking
Reranker
Access
Filter by userflowchart LR S0["Parse with structure"] --> S1["Chunk tables whole"] --> S2["Retrieve"] --> S3["Answer"]
Key point: Fix ingestion first, because later steps cannot repair it.
Why the other options are wrong
- A. Raise the temperature on generation so the model fills in the gaps
- Temperature cannot restore lost data, and may invent values.
- B. Move to a larger model that can infer the missing headers from context
- A larger model would have to guess at what is missing.
- C. Increase the number of chunks retrieved so the headers are more likely to appear
- More chunks add noise, and the rows are still separated.