labs · ai-assisted programming
Build a retrieval pipeline by hand — chunk, embed, search, ground — and then find out whether this corpus needed one at all. Both halves matter. Building infrastructure a problem does not need is a commoner and more expensive mistake than the reverse.
Open a Codespace on your own copy of the module repo.
Move into this lab and install its dependencies:
cd lectures-and-labs/week04/rag_lab
pip install -r requirements.txt
Check the install. The embedding model downloads on first use (about 90 MB) and needs no key:
python check_setup.py
Sections 3 to 5 send the retrieved text to a hosted model, and that
needs an API key. The default is the Gemini API's free tier: create
a key at aistudio.google.com/apikey
(a Google account is all it takes), copy .env.example to .env in
this folder, and put the key in LLM_API_KEY. Never commit it —
.env is gitignored and the repo's safety audit will reject it.
Sections 1 and 2 (chunking, embeddings, retrieval) need no key at all,
so you can start without one. Any OpenAI-compatible provider works:
change the base URL and model name in .env, nothing in the code.
data/ holds five short documents about programming topics. They are the
whole corpus, and their size becomes the point in section 5.
A model cannot search your documents. It can only read what you put in the prompt. So the first job is turning documents into pieces small enough to select from, and into numbers you can compare.
Work in part2_embeddings.py.
data/.Expected output
Loaded 5 documents
Produced 14 chunks (200 words, 40 overlap)
shortest: 72 words
longest: 200 words
Overlap check: chunk 1 begins "in half, eliminating the half..." -- those words also sit inside chunk 0: True
Overlap exists so that a fact sitting on a chunk boundary is not cut in half. Without it, a sentence split across two chunks may be retrievable from neither.
Split on words rather than characters — a chunk that ends mid-word embeds badly and reads worse when it reaches the prompt.
"what is a variable" and print its first five
numbers.Expected output
Embedded 14 chunks
Vector dimensionality: 384
Query vector (first 5): [0.057, 0.04, -0.052, 0.08, -0.1]
Dimensions match: True
Step 4 is not busywork. Comparing vectors of different dimensionality is meaningless, and mixing two embedding models in one index is a genuine and confusing bug — everything "works" and the results are nonsense.
Use the same model for chunks and queries, always.
Work in part3_retrieval.py.
semantic_search(query, collection, model, top_k=3) so it
returns the top_k closest chunks, each with its similarity score and
the file it came from."what is a variable"."how can my code remember a number for later" — different
words, same meaning.Expected output
Query: "what is a variable"
0.42 introduction_to_programming.txt "Introduction to Programming Programming is the..."
0.26 introduction_to_programming.txt "through a list of numbers, while..."
0.21 database_fundamentals.txt "Database Fundamentals A database is an..."
Query: "how can my code remember a number for later"
0.31 introduction_to_programming.txt "Introduction to Programming Programming is the..."
0.25 introduction_to_programming.txt "through a list of numbers, while..."
0.13 database_fundamentals.txt "Database Fundamentals A database is an..."
Step 3 is the whole justification for embeddings. The second query shares no words with the sentence that answers it ("A variable is like a labeled box that stores information"), so keyword search has nothing to match — yet it returns the same three chunks in the same order.
The scores look low, and that is normal: a similarity is not a percentage. The top chunk's preview starts "Introduction to Programming" because the definition sits a few sentences into a 200-word chunk that also covers loops and functions, and every other topic in a chunk dilutes its score (DIY 6 tests that).
If the two result sets are completely different, check you are embedding the query with the same model as the chunks.
"how do I bake sourdough".What you should have
The returned chunks and their scores, and one sentence on what a retrieval system does when nothing is relevant.
It still returns chunks. Nearest-neighbour search always has a nearest neighbour — there is no built-in notion of "nothing here is relevant".
The scores are the only signal, which is why a score threshold matters: part 3 keeps hits scoring 0.20 or more and prints how many survive. None of the sourdough hits do, while both variable questions keep their best chunks. It is also why the next section has to give the model permission to say it does not know.
Work in part4_generation.py.
Expected output
Q: What is a variable?
A: A variable is like a labeled box that stores information [source: introduction_to_programming.txt].
retrieved from: introduction_to_programming.txt, database_fundamentals.txt
Q: How do I bake sourdough?
A: I don't know - the provided context does not cover this.
retrieved from: algorithms_overview.txt, web_development_intro.txt
The wording of the first answer will vary from run to run. The
[source: …] label and the exact refusal should not.
Step 2's permission is load-bearing. "I don't know" is a rare continuation in training data, so without explicit permission the most plausible completion is a confident answer from training — which is precisely the failure grounding is supposed to prevent.
If it still answers the sourdough question, make the instruction blunter and put it after the context rather than before.
Everything so far had one setting. Now find out whether it was a good one.
Work in part5_experiments.py, recording results in results.md.
Rebuild the index at 50 words, 200 words, and 800 words per chunk — part 2 takes the size as an argument. Hold the overlap fixed, so you change one thing at a time (left to itself it shrinks with small chunks):
python part2_embeddings.py --chunk-words 50 --overlap-words 20
After each rebuild, run part3_retrieval.py and read what comes back
for its three queries.
Record for each size: was the right chunk retrieved, and how much irrelevant text came with it?
Write one sentence explaining what goes wrong at each extreme.
What you should have
A filled table in results.md:
| Chunk size | Right chunk found? | Noise | Notes |
|------------|--------------------|-------|-------|
| 50 words | | | |
| 200 words | | | |
| 800 words | | | |
At 50 words you should see chunks that have lost their own subject — a fragment saying "it must be declared before use" is useless when you cannot tell what "it" is.
At 800 you should see the right answer arriving buried in a page of unrelated text, which costs tokens and dilutes the model's attention.
There is no universally correct size. The answer depends on your documents, which is why measuring is the skill.
You have built a working retrieval pipeline. Now find out whether this problem justified one.
data/ into one string.part5_experiments.py does exactly this, with
the same grounding rules your RAG prompt uses, once your part 4 works.results.md.What you should have
## Long context vs retrieval
Whole corpus size: ................. [approx tokens]
Question asked: .................... [your question]
RAG answer: ...................... [response]
Whole-corpus answer: ............. [response]
Which was better? .................. [RAG / long context / no difference]
At what corpus size would this flip? [your reasoning]
Long context should win on this corpus, and that is the expected result, not a failure of your pipeline. Five short documents fit comfortably in a modern context window, and retrieval can only lose information that reading everything would have had.
The last line is the real question. Retrieval earns its place on scale, on cost, on freshness, and when you must cite a source. None of those apply to five files that fit in the prompt.