What Is Retrieval-Augmented Generation? A Guide to Evidence in AI Chat

What Is Retrieval-Augmented Generation? A Guide to Evidence in AI Chat

An AI answer can sound fluent while leaving an important question unanswered: where did the information come from? Retrieval-augmented generation, commonly called RAG, connects language generation with retrieved material. The idea is useful for understanding document-based chat, but it should not be confused with a guarantee that every answer is current or correct. This evergreen explainer separates the research foundation from an original practical evaluation exercise.

The research foundation

Patrick Lewis and colleagues introduced RAG research in a paper submitted in May 2020 and later revised in April 2021. Their models combined a pretrained sequence-to-sequence model with a searchable index of Wikipedia passages accessed through a neural retriever. The paper examined two formulations for conditioning generation on retrieved material and reported results across knowledge-intensive tasks. Those results apply to the evaluated research setup; they do not establish the quality of every application that now uses the RAG label.

At a general level, retrieval provides candidate material and generation produces an answer using the available input. Assessing an application therefore requires examining both the material selected and the claims ultimately written. A polished paragraph alone cannot reveal whether either stage worked well.

An original library example

Imagine a fictional library with a handbook about borrowing equipment. One section covers ordinary loans, another covers holiday closures, and a third contains an outdated pilot policy. A visitor asks when a borrowed camera must be returned after a holiday weekend. The useful evidence is not necessarily the first paragraph containing the word 'camera.' It may require the general loan rule and the holiday exception together.

For an evaluation, write three short fictional documents with clearly labeled dates. Include one deliberate conflict. Decide in advance which document takes precedence and what the correct answer should say. This creates a manageable test where you can inspect all the evidence yourself.

Check retrieval separately

If the application exposes retrieved passages, examine whether it found the relevant holiday exception. Record missing documents and irrelevant matches. Do not immediately blame the language model for an answer based on incomplete material. Conversely, finding the right passage does not mean the final answer interpreted it correctly.

If passages are not visible, record that as an evaluation limitation. Ask for source references, but remember that a reference in generated text is a claim to verify. Open the cited document and check the relevant section rather than treating the presence of a link as proof.

Check the answer claim by claim

Break the response into small statements: the ordinary loan lasts a specified period; the library closes on a particular day; the exception changes the deadline. Match each statement to the fictional handbook. Mark unsupported additions separately from direct contradictions. This makes the review more useful than simply giving the whole answer a thumbs-up.

Also test an unanswerable question. Ask whether the library plans to buy a new camera next month when none of the documents discusses purchases. The desired behavior in this exercise is to acknowledge that the available material does not establish an answer. Compare what the application actually does with that expectation.

Make updates visible

Replace the old pilot document with a revised version and repeat the same questions. Check which version is cited and whether the answer changes appropriately. This is an editorial test proposal, not an assertion that a particular indexing pipeline updates instantly. Record the revision time and the time of each test so delays are visible rather than guessed.

Keep examples free of confidential information. Invented handbooks are sufficient for learning the evaluation method. Testing with private records would introduce access and privacy questions that this exercise does not address.

What a useful evaluation report includes

Write down the question, expected evidence, retrieved passages if available, answer, citations, and observed issue. Distinguish an absent source from a misunderstood source and an unsupported inference. Repeat tests after a material document or configuration change, while keeping a stable set of questions for comparison.

RAG is best understood here as a way to bring external evidence into the answer process. The practical task remains to inspect that evidence, verify the resulting claims, and state what the available documents cannot establish.

Source: Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, https://arxiv.org/abs/2005.11401

Editorial note: The library handbook and evaluation steps are original illustrative examples, not results from the cited research.