§2.3 · FOUNDATIONS · 4 MIN READ · EDITION 1.0
Debugging a wrong answer: retrieval or generation?
When an AI-supported output is wrong, the first diagnostic question is whether the system saw the right evidence at all, before asking whether it reasoned about that evidence correctly.
A wrong answer built on the wrong or incomplete source evidence is a retrieval failure; the model may have reasoned perfectly over bad input. A wrong answer built on the correct evidence is a generation failure, and a different kind of fix.
Conflating the two wastes investigation effort: teams sometimes retrain or reconfigure a model to fix what was actually a missing-document problem.
PUT THIS INTO PRACTICE
- 01Check what evidence was actually retrieved before reviewing the answer.
- 02Separate a retrieval fix from a generation fix in the incident record.
- 03Test retrieval quality independently of end-to-end answer quality.
RELATED INSTRUMENTS
CITE THIS SECTION
§2.3 · Edition 1.0 · Debugging a wrong answer: retrieval or generation?