# RAG — Retrieval-Augmented Generation with Chunking, Citations and Evaluation

Source: https://skills101.in/learn/ai-engineering/retrieval-augmented-generation-rag — Skills101 free tutorial (AI Engineering — Building Apps with Large Language Models). Updated 2026-11-09.

**RAG** = find the relevant passages in your documents, put them in the prompt, and tell the model to answer **only** from them. It's the most widely deployed LLM pattern in companies — support bots, policy assistants, internal search.

![Documents are split into chunks and indexed ahead of time; at question time the best chunks are retrieved and given to the LLM, which answers with citations](https://skills101.in/learn-img/ai-engineering/rag.svg)

## Case study: the official college help-desk bot

Students ask the same questions every semester: attendance, fees, hall tickets, re-evaluation. The college wants a bot that answers **from the official handbook only**, shows the section it used, and sends anything else to the office.

## 1. Chunking

Long documents are split into **chunks** (a few hundred words each), small enough to be specific, big enough to keep context. A little **overlap** stops answers being cut in half:

```python
def chunk_text(text: str, size: int = 40, overlap: int = 10) -> list[str]:
    words = text.split()
    chunks, start = [], 0
    while start < len(words):
        chunks.append(" ".join(words[start:start + size]))
        if start + size >= len(words):
            break
        start += size - overlap                      # step forward, keeping some overlap
    return chunks

page = " ".join(f"w{i}" for i in range(100))          # a 100-word page
for c in chunk_text(page):
    print(c.split()[0], "...", c.split()[-1], f"({len(c.split())} words)")
```

```text
w0 ... w39 (40 words)
w30 ... w69 (40 words)
w60 ... w99 (40 words)
```

Real systems split on **headings and paragraphs** first (a section is a natural unit) and keep metadata: document name, section number, page.

## 2. Retrieve, then generate with sources

```python
import anthropic
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity

SECTIONS = {
    "4.1 Attendance": "Students must have at least 75% attendance in each subject to appear in the end-semester exam.",
    "4.2 Medical leave": "Absence with a medical certificate submitted within 7 days can be condoned up to 10%.",
    "6.1 Fees": "The semester fee must be paid by the 10th of the first month; a late fee of Rs 500 applies after that.",
    "7.3 Hall tickets": "Hall tickets are issued two weeks before exams to students who meet the attendance rule.",
}
titles, texts = list(SECTIONS), list(SECTIONS.values())
vectorizer = TfidfVectorizer(stop_words="english")
index = vectorizer.fit_transform(texts)

def retrieve(question: str, k: int = 2, min_score: float = 0.1):
    scores = cosine_similarity(vectorizer.transform([question]), index)[0]
    ranked = scores.argsort()[::-1][:k]
    return [(titles[i], texts[i]) for i in ranked if scores[i] >= min_score]

def answer(question: str) -> str:
    sources = retrieve(question)
    if not sources:                                     # nothing relevant: don't let the model guess
        return "I can't find this in the handbook. Please contact the academic office."
    context = "\n".join(f"[{n}] ({title}) {text}" for n, (title, text) in enumerate(sources, 1))
    prompt = f"""Answer the student's question using ONLY the handbook sections below.
Cite the section numbers you used like [1]. If the sections don't contain the answer, say you don't know.

<context>
{context}
</context>

Question: {question}"""
    response = anthropic.Anthropic().messages.create(
        model="claude-opus-5-5", max_tokens=300, messages=[{"role": "user", "content": prompt}])
    used = ", ".join(t for t, _ in sources)
    return response.content[0].text + f"\n(sources: {used})"

print(answer("What attendance do I need to write the end-semester exam?"))
print(answer("Who won the college cricket tournament?"))
```

Output:

```text
You need at least 75% attendance in each subject to sit the end-semester exam [1].
(sources: 4.1 Attendance, 7.3 Hall tickets)
I can't find this in the handbook. Please contact the academic office.
```

*(The first answer's wording comes from our local stand-in API; the retrieval, the sources and the out-of-scope refusal are computed by the code above.)*

The second question is the important one: nothing relevant was retrieved, so the bot **refuses before calling the model** — no chance to invent an answer.

## 3. Measuring retrieval

If retrieval brings the wrong passages, even the best model fails. Build a small **test set** of real questions with the section that should be found, and measure **hit rate @ k**:

```python
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity

SECTIONS = {
    "4.1 Attendance": "Students must have at least 75% attendance in each subject to appear in the end-semester exam.",
    "4.2 Medical leave": "Absence with a medical certificate submitted within 7 days can be condoned up to 10%.",
    "6.1 Fees": "The semester fee must be paid by the 10th of the first month; a late fee of Rs 500 applies after that.",
    "7.3 Hall tickets": "Hall tickets are issued two weeks before exams to students who meet the attendance rule.",
}
titles = list(SECTIONS)
vectorizer = TfidfVectorizer(stop_words="english")
index = vectorizer.fit_transform(SECTIONS.values())

tests = [
    ("What attendance is needed for exams?", "4.1 Attendance"),
    ("When will I get my hall ticket?", "7.3 Hall tickets"),
    ("What is the late fee?", "6.1 Fees"),
    ("I was ill for a week, will my absence count?", "4.2 Medical leave"),
    ("Can I bunk classes and still sit the exam?", "4.1 Attendance"),
]
for k in (1, 2):
    hits = 0
    for question, expected in tests:
        scores = cosine_similarity(vectorizer.transform([question]), index)[0]
        top = [titles[i] for i in scores.argsort()[::-1][:k]]
        hits += expected in top
    print(f"hit rate @ {k}: {hits}/{len(tests)}")
```

```text
hit rate @ 1: 4/5
hit rate @ 2: 5/5
```

Misses show you what to improve: better chunking, adding synonyms, real embeddings, or hybrid search. Fix retrieval before blaming the model.

## 4. Making it production-ready

- **Citations everywhere** — users (and you) can check answers.
- **Freshness** — re-index when the handbook changes; store the version and date.
- **Access control** — only retrieve documents the user may see (staff vs students).
- **Limits** — top-k of 3–8 chunks keeps prompts small and focused.
- **Feedback** — a 👍/👎 button gives you new test questions.

## Common mistakes

- Giant chunks (whole chapters) or tiny ones (single sentences without context).
- No "not in the documents" path — the model fills gaps with guesses.
- Evaluating only the final answers, never the retrieval.
- Leaking confidential documents through retrieval without permission checks.

## Quick recap

- RAG: chunk → index → retrieve top-k → prompt with numbered sources → answer with citations.
- Refuse when nothing relevant is found; instruct the model to answer only from the context.
- Measure retrieval with a test set (hit rate @ k) and fix it first.


Practice quiz and free certificate: https://skills101.in/learn/ai-engineering/retrieval-augmented-generation-rag