Prompting and Context Engineering

slide 1

Prompting and Context Engineering

how you ask, and what the model can see

slide 2

Two prompts, same task

PROMPT · A

write a function to parse a date string

PROMPT · B

Write a Python function parse_date(s: str) -> date.

Accept "2026-08-12" and "12/08/2026" (day first). Raise ValueError on anything else. No external libraries.

• Which gets the better answer, and why?

slide 3

The one idea

A prompt is not a request. It is a specification. Everything you leave unspecified, the model decides for you: silently, and from whatever was most common in its training data.

• "Better prompt" does not mean "longer prompt"
  • Length is not the variable. Two things are:
  • Decisions made: audience, format, length, tone, what to leave out
  • Information supplied: facts, data and examples the model cannot know
  • Padding ("be thorough, detailed, high quality") adds words but no decisions, so the model still fills every gap with its defaults

slide 4

Agenda

// part 1: how you ask

• Tokens, the context window, and the SPEC checklist
• Personas, step-by-step, examples: what each really does
• Why what the model can see now matters more than wording

// part 2: what the model can see

• How the model fills in what you didn't say
• Debugging a failing test, with and without the code
• Why pasting more makes answers worse, and when to restart
• From ChatGPT to agents: where prompting is now

slide 5

Tokens, the context window, and the SPEC checklist

// part 1 · section 1 of 3

What the model actually reads, and a checklist for prompts that leave nothing to guess

slide 6

What is prompting?

Prompting is writing the instructions and information you give an AI model so that it produces the result you want.

• The prompt is everything you send: your request, plus any files, errors or examples you paste in
• The model answers from its training and from what is in front of it: your prompt, plus anything its tools add, such as web search results or files it read
• A good prompt answers the questions the model would otherwise guess: what to make, for whom, in what format, and what to leave alone

slide 7

Tokens: the unit the model actually reads

• A token is a chunk of text: a word, part of a word, or punctuation

[Picture: Real token splits from OpenAI's o200k_base tokenizer: def calculate_median(numbers): is 6 tokens (def | calculate | _m | edian | (numbers | ):); the date 12/08/2026 is 6 tokens (12 | / | 08 | / | 202 | 6); KeyError: 'unit_price' is 7 tokens (Key | Error | : | ' | unit | _price | ')]

• Cost, the size limit, and what the model can remember are all counted in tokens, not words

slide 8

Pasted code and errors use more tokens than English of the same length

[Chart: tokens per 100 words, by kind of text. Tokens per 100 words (English prose: 125.2, JSON: 158.1, Python source: 191.3, Python traceback: 267.7)]

// source: counted with OpenAI's o200k_base tokenizer on samples of each kind

• Per word, code uses more tokens than English: 1.5× for Python, 2× for a traceback
• Names, file paths and symbols each split into several tokens

// OpenAI's rule of thumb: 1 token ≈ 4 characters ≈ ¾ of an English word

slide 9

The context window

// the context window: everything the model sees on one request

System prompt

Written by the assistant's makers

Standing instructions

AGENTS.md, read on every request

Conversation so far

Grows every turn, dead ends included

Pasted + tool results

Your pastes, files the agent read, web searches

free

Space left

outside

Your repo, ticket, intent

Invisible unless something puts it in

• A chat or coding assistant is a program that fills this window and sends it to the model
• The model sees only what is inside the bar. Anything outside it, like your repo or what you meant, does not exist for the model

slide 10

SPEC, a recipe for every prompt

// any checklist works; what matters is covering all four, every time

slide 11

The opening pair, decoded

PROMPT · B

Write a Python function parse_date(s: str) -> date.

Accept "2026-08-12" and "12/08/2026" (day first). Raise ValueError on anything else. No external libraries.

• S parse a date · P Python, typed signature · E two example formats · C raise an error, no libraries

Is 12/08/2026 in August or December? Prompt A leaves the model to guess. B is better because it makes that decision, not because it is longer.

// tested Sept 2026: given prompt A, three current models read it as 8 December (month first), and none asked

slide 12

Constraints and non-goals

Constraints: what the model must respect

• Python 3.12
• no new dependencies
• under 200ms
• no eval()

Non-goals: what the model must not touch

• No database schema changes

• No error handling yet

• Do not reformat the rest of the file

• Assistants often change more than you asked. Bugs hide in those extra changes, because nobody expected them, so nobody checked them

slide 13

Predict: what comes back?

PROMPT

here's my file, fix the bug in calculate_total

1

Just the one-line fix

2

The fix, plus changes you did not ask for

3

The whole file, reformatted, with type hints and error handling

Answer: none of them reliably. It depends on the model, and on the run

Tested Sept 2026: the same prompt changed 1 line with one model, 3 with another, and 11 then 4 with a third

slide 14

A vague prompt changed 1 to 11 lines; a narrow one changed 1 every time

[Chart: lines each model changed to fix a one-line bug: two runs of the vague prompt and one of the narrow prompt. vague prompt, run 1 (GLM-5.3: 1, Gemini 3.6 Flash: 3, Gemini 2.5 Flash: 11); vague prompt, run 2 (GLM-5.3: 1, Gemini 3.6 Flash: 3, Gemini 2.5 Flash: 4); narrow prompt (next slide) (GLM-5.3: 1, Gemini 3.6 Flash: 1, Gemini 2.5 Flash: 1)]

// source: tested Sept 2026 through each vendor's API, no system prompt; every reply fixed the bug

• None added type hints or error handling. The extras were a rewritten loop, new comments, or the whole file sent back

slide 15

Ask narrowly, and say what not to touch

PROMPT

Fix only the off-by-one in calculate_total.

Change nothing else: no formatting, no type hints, no error handling.

Show me a diff, not the file.

You will read a one-line change carefully, but only skim a whole file. Asking for just the fix, as a diff, is what makes a real review possible.

// tested Sept 2026: a one-line diff from all three models

slide 16

Personas, step-by-step, examples: what each really does

// part 1 · section 2 of 3

Three well-known techniques, what the evidence says about each, and when to use which

slide 17

Three techniques worth knowing

• Add-ons to a SPEC prompt: few-shot is SPEC's E, done properly
• Each is one prompt, not a slow conversation or a series of prompts

// each technique fixes one kind of problem; use it only for that problem

slide 18

Personas set tone, not accuracy

• "You are an expert" did not make answers more accurate: 162 roles, 2,410 questions, nine open models (Zheng et al., 2024)
• Re-tested in 2025 on GPT-4o, o3-mini, o4-mini and Gemini: no significant gain, and "layperson" personas made answers worse
• What a role does change: tone, audience, and what the model focuses on

Use a persona to set the viewpoint ("review this as a security auditor"), not to make answers more correct

slide 19

Chain-of-thought: a 2022 trick, now built in

[Chart: percentage points of accuracy gained by asking for step-by-step reasoning, 2022 models against 2025 reasoning models. Accuracy gained by asking for step-by-step reasoning (PaLM 540B 2022, maths: 39.0, text-davinci-002 2022, maths: 30.3, o3-mini 2025, science: 2.9, o4-mini 2025, science: 3.1, Gemini 2.5 Flash 2025, science: -3.3)]

// source: Wei et al. 2022 and Kojima et al. 2022 (GSM8K); Meincke et al., Wharton, 2025 (GPQA Diamond)

• Today's reasoning models already work through the problem before answering, so "think step by step" barely helps and can make results worse
• Still useful: ask the model to show its plan or its assumptions, so you can check them

slide 20

Predict: which technique?

You need a report as CSV: fixed header, fixed column order, no commentary. The model keeps adding a sentence of explanation above the data.

• Persona: "you are a data engineer"
• Chain-of-thought: "think step by step about the format"
• Few-shot: two complete examples of the exact output
• A longer, firmer description: "no commentary, I mean it"

slide 21

Few-shot, shown

PROMPT

One CSV line per bug report. Header: severity,component,summary.

"Login 500 when the password has an emoji" → high,auth,500 on emoji password

"Typo on the about page, and in the footer" → low,web,"typo on about page, footer"

Now: "Checkout total wrong when a coupon is applied twice"

REPLY

high,checkout,coupon applied twice miscounts total

• The two examples differ on the tricky case: the second has a comma inside a value, and shows it wrapped in quotes
• A persona or "think step by step" would not fix this: neither shows the exact format. Examples do

// if code will read the output, use the API's structured-output option (a JSON schema) instead

slide 22

Why what the model can see now matters more than wording

// part 1 · section 3 of 3

When rewording stops helping, the model is usually missing information

slide 23

Wording matters less than it did. Missing information matters most

What no wording can do: give the model a fact it was never shown

e.g. no rewording of “fix it” tells the model your data uses the key price, not unit_price

THEN · 2023

Tiny wording changes swung the results

Swapping a colon, a newline or capital letters moved one model's score by up to 76 points out of 100

So people hunted for magic phrasings

NOW

Models understand sloppy requests

Rewording the same request changes little

Polishing the wording is no longer where the gains are

So when an answer is wrong, first ask: what does the model not know?

slide 24

Context engineering

• Prompt engineering is part of it: your request is one item in the window
• The goal is not more context but the right context: only what this task needs

// Anthropic: "the smallest possible set of high-signal tokens" for the task

Context engineering is choosing what the model gets to see: which files, errors, examples and past messages go into its window, and which stay out.

slide 25

Predict: three rewrites, still wrong. Why?

SCENARIO

You asked the model to fix a bug, then rewrote the request three times, each clearer than the last. The answer is still wrong in the same way.

1

The model is not good enough: switch to a bigger one

2

Keep rewording until you find the magic phrasing

3

The model is missing a fact it cannot guess

4

Turn up the randomness for a different answer

Answer: 3. The model is missing a fact it cannot guess

If clearer wording changes nothing, wording was not the problem. Rewording adds no new facts, and a bigger model or more randomness has the same gap.

Fix: give the model what it is missing, e.g. the file, the error message or the failing test.

slide 26

Wrong answer? First ask which kind of problem

Wrong because I asked badly, or because the model doesn't know something?

You asked badly

HOW YOU CAN TELL

• The model misread what you wanted
• Each rewrite gives a different answer

WHAT TO DO

Rewrite the prompt: fill the SPEC gaps

The model lacks a fact

HOW YOU CAN TELL

• The same wrong answer every time
• The model guesses names, keys or versions

WHAT TO DO

Paste in the error, file, schema or failing test

Rule of thumb: if you can name the fact the model is missing, paste it in. If not, it's the wording.

slide 27

Ask the model to ask you questions before it writes code

PROMPT

Before writing any code, ask me up to five questions about anything you would otherwise have to guess. Then stop and wait for my answers.

• Add the line above to your prompt. The model replies with questions, not code
• Each question is a decision your prompt left out, e.g. "day first or month first?"
• You answer them, so you make those decisions instead of the model guessing

// then send your answers back: together with the request, they are a complete SPEC

slide 28

What prompting cannot fix

• The model cannot see your code, tickets or intent unless you, or its tools, put them in front of it
• The model has a training cutoff: web search can fill the gap, but only if it searches, and it may find old docs
• Exact counting and arithmetic are unreliable, unless the model runs code for them
• Ask the wrong question and the model will answer it confidently, without telling you it was the wrong question

// a good prompt directs what the model can already do; it cannot add knowledge or ability

slide 29

Up next: part 2

Part 2 answers: when you leave a gap, how does it get filled, and what, exactly, is in front of the model when it answers?

slide 30

How the model fills in what you didn't say

// part 2 · section 1 of 4

Where the model's guesses come from, and what is actually in the window when it answers

slide 31

How a gap gets filled

1. The prompt does not say which date order to use
2. Both date orders appear often in the model's training data
3. The model picks whichever order it saw more often
4. A guess that looks like a decision

• The model predicts the next token from everything in the window, and nothing else
• Where your prompt says nothing, the model picks the most common choice from its training data, not the right one for you

A vague prompt's guesses look reasonable, so the wrong ones are hard to spot. The fix: write the answer down once.

slide 32

Write the rule once, in a project file

# AGENTS.md
- Python 3.12; run the tests with `pytest -q`
- Dates are day first: 12/08/2026 is 12 August
- Money is Decimal, never float
- Never add a dependency without asking

What it is

Plain text in your project. The assistant reads it with every request.

What goes in it

Rules you keep retyping: commands, formats, limits, things to avoid.

What it is called

It depends on the tool:

AGENTS.md

CLAUDE.md

.github/copilot-instructions.md

Keep it short and current.

The model follows an outdated rule just as strictly as a correct one.

slide 33

Predict: what does the assistant see?

Your editor's assistant. A project of forty files. You ask about a function in a file you have not opened.

• The whole project: it is open in the editor
• Everything you have opened since you started the editor
• The file you are in, plus whatever the assistant decided to read first
• Nothing but your question

slide 34

Only what was put in the window

• The model sees the file you are in, any text you selected, and whatever files the assistant chose to read
• An open project is not visible to the model. It sees only what was put in the window
• Some assistants search your files first and pick the context for you
• When an answer is wrong, first check which files the assistant actually read

// if the model "doesn't know" something, that thing was not in the window

slide 35

Debugging a failing test, with and without the code

// part 2 · section 2 of 4

The same error twice: first on its own, then with the code it came from

slide 36

A second case: the failing test

• The opening pair was generation: write me a function. This is diagnosis: why is this broken?

FAILED tests/test_orders.py::test_total - KeyError: 'unit_price'

PROMPT

my test fails with KeyError: 'unit_price', fix it

• Before the next slide: is this a wording problem, or a knowledge problem?

slide 37

Predict: error only, what comes back?

• The model asks to see the function
• A .get() with a default value, so the missing key stops raising
• Both, in the same reply
• The model rewrites the test to match

slide 38

What came back

REPLY · Gemini 2.5 Flash, Sept 2026, excerpt

Use dict.get() with a default value (Recommended). This is the most Pythonic and safest way to access dictionary keys that might not exist.

• 3 of 4 replies asked to see the code, but all 4 still suggested .get() with a default of 0
• Apply that fix and the test still fails, with a different error:

assert 0 == 10.0

• The fix hid the error instead of fixing the bug

slide 39

What the model could not see

def order_total(order: dict) -> float:
    return sum(item["unit_price"] * item["qty"] for item in order["items"])

def test_total():
    order = {"items": [{"price": 2.50, "qty": 4}]}
    assert order_total(order) == 10.0

• The test data uses the key price, but the function expects unit_price. Each is fine alone; they just don't match
• No rewording of "fix it" could find this: the model was never shown this code
• The real fix is a choice only you can make: which key name should the whole codebase use?

slide 40

What to put in front of the model

// paste the real code and error, not a summary; include everything relevant and nothing else

slide 41

Why pasting more makes answers worse, and when to restart

// part 2 · section 3 of 4

Leaving things out on purpose, and starting fresh when a conversation goes in circles

slide 42

More is not better: why

• Three files pasted "just in case" · noise
• Two attempts already abandoned in this conversation · noise
• The four lines that actually decide the answer · signal
• A 300-line log, of which one line matters · noise

• The model must find the few lines that matter among everything you pasted; the more you paste, the more often it misses them
• Deciding what to leave out matters as much as deciding what to put in

slide 43

Longer input, less reliable answers

[Chart: accuracy at finding one fact as the input grows from 1K to 128K tokens. GPT-4o (1K: 98.1, 4K: 95.7, 8K: 89.2, 16K: 81.6, 32K: 69.7, 128K: 56.0); GPT-4.1 (1K: 95.6, 4K: 91.7, 8K: 87.5, 16K: 84.9, 32K: 79.8, 128K: 64.7)]

// source: NoLiMa, Adobe Research, ICML 2025 (GPT-4.1 from its repository)

• In Chroma's 2025 tests, every model answered better from a short ~300-token prompt holding just the relevant part than from the full ~113K-token conversation

// in 2023 models often missed facts in the middle of long input; that is less true now, but longer input still gives worse answers

slide 44

The thread is context too

• Every turn stays in the window: your questions, its answers, and every attempt you rejected
• The model cannot tell which ideas you have given up on. A rejected attempt still influences it as if it were current
• Correct the model, and the wrong version is still there, beside the correction
• When the window is full, assistants replace the conversation with a summary, and anything the summary leaves out is lost

// a long conversation slowly fills up with old attempts you no longer want the model to use

slide 45

Predict: the thread is going in circles

Forty minutes in, and each answer is a variation on the last one.

• Push on: it is nearly there
• Switch to a bigger model
• New conversation, carrying only the context that turned out to matter
• Raise the temperature for a fresh angle

slide 46

Start again, carry the good context

PROMPT

New conversation. Here are order_total, its test, and the fixture it uses.

A default value was already tried and is wrong: the test then fails as 0 == 10.0.

Propose the minimal fix, and state which key name you treat as canonical.

• What you carry: the code, the failing test, and what was ruled out
• What you leave behind: forty minutes of failed attempts that were still influencing every answer
• The last line forces the model to say which key name it chose, instead of choosing silently

slide 47

The best context is executable

def test_parse_date_day_first():
    assert parse_date("12/08/2026") == date(2026, 8, 12)

• These two lines give an example, a rule, and a fact the model could not know: dates here are day first
• Prose can be read two ways. A test can only pass or fail
• Write the test first, then ask for code that passes it. You stop judging code by how it looks and check whether it passes

// a test written after the code only checks what the code already does, bugs included

slide 48

Three ways an answer goes wrong

Asking "bad wording or missing information?" tells the first two apart. The third, pasting too much, is rarely suspected because it feels like being thorough.

slide 49

Common mistakes

When you ask

• Adding adjectives instead of decisions: "robust" tells the model nothing specific
• Using a technique (persona, step by step, examples) without a problem it fixes
• Accepting a large change because the fix is somewhere inside it
• Telling the model "you are an expert" to get more accurate answers

When you give the model context

• Rewording when the real problem is missing information
• Pasting far more than the question needs
• Paraphrasing the error instead of pasting it verbatim
• Assuming the model can see the project because it is open
• Staying in a long conversation full of failed attempts

slide 50

From ChatGPT to agents: where prompting is now

// part 2 · section 4 of 4

How prompting changed since 2022, and why it still matters when agents work for hours

slide 51

From one message to a plan that runs for hours

1. 2022 · ChatGPT You type one message and reword it until the answer is right
2. 2023–24 · Longer chats You paste code, errors and files into the conversation
3. 2025 · Coding agents Tools like Claude Code and Codex read your files and run the tests
4. 2026 · Agents for hours You write a plan; the agent works for hours, then you review

• The longest task an AI agent can finish alone grew from under a minute (GPT-3.5) to about 12 hours (early 2026)

// source: METR, task length agents complete half the time, measured in human working time, 2025–2026

Your job moved from choosing the right words to deciding what gets built, and checking it

slide 52

We still prompt: the plan is now the prompt

An agent that works for hours still starts from a prompt. It is just longer: a written plan, plus the standing instructions it reads every time.

• A vague plan fails the same way as a vague message, only after hours of work instead of seconds
• The rules in this lecture apply to both: make the decisions, supply the information, say what not to touch
• What changed is where your effort goes: less rewording, more writing the plan and reviewing the result

slide 53

Summary

How you ask

• A prompt is a specification: anything you leave out, the model fills with the most common choice
• Use SPEC every time; saying what not to touch keeps changes small enough to review
• Examples fix the format; personas set tone, not accuracy

What the model can see

• The model sees only what is in the window, so check what was put there
• Pasting more makes answers worse: leave out extras, and restart looping conversations
• A test is the best context: it gives an example, a rule and new information at once

Is it wrong because I asked badly, or because the model doesn't know something? No amount of rewording adds information.