RAG evaluation is how you check whether a retrieval-based AI system is doing its job: finding the right information and writing an answer that is faithful to it. A RAG setup has two stages, retrieval and generation, and both can fail. Evaluation measures each one separately so you know exactly where a wrong answer came from. Without that split, a broken system just feels vaguely off and nobody can say why.
Think of it like grading a student’s open-book exam. You check two things: did they open the book to the correct page, and did their written answer actually match what that page said? A confident essay built on the wrong chapter still fails. A correct quote pasted under the wrong question also fails. RAG evaluation scores a system the same way, so a polished answer that quietly cites the wrong source gets caught before a customer ever sees it.
In practice you build a test set of questions with known good answers and run them through the system. You measure whether retrieval surfaced the right material and whether the final text stayed grounded in it instead of drifting into a hallucination. Common scores include recall, which asks if the right document showed up at all, and faithfulness, which asks if every claim in the answer traces back to a real source. Run the same set after each change and the numbers tell you whether you helped or hurt.
At TopDevs we build a RAG evaluation set before we ship, so a client can see in plain numbers whether a change to the system helped or quietly made answers worse.
