Imagine a course handbook has just been updated. You put both the old and new versions into a knowledge base and ask the AI: "What materials do I need to submit for this application?" It quickly lists them, complete with sources.
But when you open the sources, a problem appears: one passage comes from the old handbook, and another only describes the application process without listing all the required materials. The answer looks complete, but the evidence does not fully support it.
Retrieval-augmented generation, usually shortened to RAG, brings external materials into the language model's answer process. EvoClass's AI025, Practical RAG Systems: From Knowledge Base to Retrieval-Augmented Generation, unfolds this path across five lessons. What the course cares about is exactly how information travels from the original document to an answer you can verify.
The first break may occur before the material ever enters the system
In Lesson 1: RAG and Knowledge Base Construction Basics, the course first distinguishes prompting, fine-tuning and RAG, and brings document cleaning, metadata and version management into knowledge base construction.
This step is easily mistaken for preparation, but it actually determines what can be verified later. Which version a document belongs to, who it applies to, and when a clause takes effect can all affect the answer. If this information is lost at import, no matter how fluent the generation stage is, it is hard to restore reliable grounds for the answer.
Returning to that handbook, we can start with a very plain question: can the system distinguish "this is a new file" from "this is a currently valid regulation"? Publication time and applicability time are not always the same thing.
So when building a knowledge base, what needs consideration is not just storing the body text, but also preserving its identity and the conditions under which it applies.
Finer chunks do not necessarily mean more accurate retrieval
A long handbook often needs to be split into retrievable chunks. Split too small, and a chunk may keep the name of a material but lose the preceding statement of who it applies to; split too large, and the answer may drown in irrelevant content.
Lesson 2: Chunking, Embedding and Indexing discusses chunking in relation to recall, ranking, context and citation. The boundaries of a chunk affect both "whether it can be found" and "whether it can be used correctly once found."
For example, if a regulation's exception condition is written in the next paragraph, keeping only the first paragraph may make a conditional requirement look as if it applies to everyone. This kind of problem is worth catching during material preparation, rather than investigating only after the final answer goes wrong.
The course also distinguishes semantic similarity from the purpose of an answer. Two passages discussing the same topic does not mean both contain the piece of evidence the user needs.
Once relevant material is found, it still needs to be filtered
Users sometimes ask about a concept, and sometimes cite a precise number, product name or clause. The two kinds of questions place different demands on retrieval.
In Lesson 3: Hybrid Retrieval and Reranking, the course discusses the respective uses of keyword matching and semantic retrieval, and goes on to introduce reranking.
It can be understood as two rounds of selection: first, retrieve as many possibly useful candidates as possible, then judge which chunks actually contain the information needed to answer the question. Looking only at similarity scores is often not enough to complete the second judgment.
When checking handbook Q&A, failures can be broken down and recorded separately: did the valid clause never enter the candidate set, or did it enter but get ranked too low? These two situations require handling different parts of the pipeline. Only by keeping intermediate results is there a chance to turn "the answer was wrong" into a problem that can be located.
A source must also be checked for whether it supports the conclusion
Citations give readers a chance to return to the material, but the mere presence of a citation does not make an answer reliable. Further checks are needed: is this material still valid, does it correspond to this sentence, and does it cover the qualifications in it.
Lesson 4: Citations, Timeliness and Evaluation discusses evidence recall and generation faithfulness separately; Lesson 5: From Prototype to Production extends the focus to latency, cost, fault diagnosis and what to do when evidence is insufficient.
A learning task worth trying is to prepare several types of questions around the same material: ones whose answers are stated directly in the text, ones that require combining information from several places, ones that involve an older version, and ones the material does not answer at all. Check one by one what evidence the system obtained and what it said based on that evidence.
These questions are learning suggestions, not system tests already completed for this article. They remind us that the value of a knowledge base needs to be tested against real information differences, not judged from a few smooth demonstrations alone.
You can start from the AI025 course page and follow each handoff from material to answer. The next time you face an answer with sources, the question worth asking is: which statements in the answer do these sources actually support?
This article is based on EvoClass's current course materials; lesson links point to English materials. The handbook Q&A in this article is a teaching scenario; no claims are made here about the accuracy, performance or data access capabilities of any deployed system. Visitors can view currently open content; full learning access is subject to the prompts on the page.