A tiny language model, built from zero

Lesson 8 counted which word comes next. Lesson 9 counted words to sort commands into piles. Now you'll build a small neural network that learns its own numbers for every word, reads your whole command, and writes a whole plan one word at a time: "put the red and blue blocks away" becomes put red blue. That loop is the beating heart of every LLM, including Claude.

Going deeper: everything below is the full lesson: the idea built from nothing, a demo with the real trained model, your homework, and a challenge away from the screen.

1. The problem

Here's what our earlier models can and can't do:

• Lesson 8's bigram model looks at one previous word and counts. It can write, but it forgets everything except the last word. It could never read a seven-word command and answer it.
• Lesson 9's word counter reads the whole command, but treats every word as a separate clue. "halt" and "freeze" are as different to it as "halt" and "bin". And it picks one of four piles, so every answer is exactly one action.

One action isn't enough for a tidying robot. Real output from the bake-off: for "put the red and blue blocks away" the word counter replies The red block is in the bin. and stops. For "bin everything except the green block" it bins the green block, the one you said to leave. For "put the block away" it replies Sorry, I can't do that: 'put' needs an object instead of asking which block.

We want one model that reads the whole command and writes a plan, however long:

"put the red and blue blocks away"  →  put  red  blue  <end>
"clean up everything"               →  put  red  blue  green  <end>
"everything except green"           →  put  red  blue  <end>
"put the block away"                →  ask  <end>            (which block? it asks you)
"make me a cup of tea"              →  none  <end>           (not a robot job)
"halt"                              →  stop  <end>

The model writes from a menu of 10 words: put go find stop ask none red blue green <end>. The <end> word means "I've finished". The longest plan, put red blue green <end>, is 5 words, so it never writes more than MAX_WORDS = 5. This is the same job as Lesson 8 (predict the next word, repeat) but now the prediction depends on the whole command too.

An honest note about the first try. An earlier version of this lesson's model wrote only one action, like put red <end>. It was a cleverer pile-sorter, nothing more, and on the bake-off it scored 4/12: worse than the word counter's 5/12. A brain that can write should write what the job needs. That's why this version writes whole plans.

2. First principles

Step 1: give every word a list of numbers

Lesson 9 stored a count per word per intent. That's a big table full of separate numbers, with nothing shared. Here's a different idea: give each word its own short list of numbers, like a character card in a game. Imagine the card had columns like "how stop-ish", "how all-ish", "how red". Then "halt" and "stop" would have similar cards, and anything the model learns about one would partly carry over to the other.

The twist: we don't decide the columns. Each word starts with 12 random numbers between −0.3 and 0.3, and training nudges them until they're useful. That list is called an embedding. Here are the first 4 of the 12 numbers for "red", from the real model:

"red" before training:  [−0.14, −0.27, −0.20, −0.30, …]   (random)
"red" after training:   [−0.20, −0.76,  0.94, −2.75, …]   (learned)

Do similar words really end up with similar lists? We can measure how much two lists "point the same way" with a score from −1 (opposite) to 1 (identical direction). Real numbers from our model:

Nobody told it that "halt" means stop, or that "except" and "but" do the same job. It worked that out because those words led to the same answers in training. ("tea" and "joke" both lead to none, but each appears in only one command, so they barely moved.) That's the big idea behind embeddings, and every LLM starts with exactly this step.

Step 2: read a command by averaging

A command has several words, so several lists. We need one list for the whole thing. The simplest way: average them, position by position. Pretend the lists were only 3 numbers long:

"red"    →  [1, 0, 2]
"block"  →  [3, 2, 0]
average  →  [(1+3)/2, (0+2)/2, (2+0)/2]  =  [2, 1, 1]

Two rules. Words the model has never seen (not in its 88-word vocabulary) are skipped, because there's no list for them. And if no word is known, the result is 12 zeros. That's what read does, and you'll write it.

Step 3: also look at what you've already written

When writing the third word, the model needs to know what it has written so far. So at each step it also looks at the last two words it wrote. Those get their own embeddings (12 numbers each). At the very beginning there's nothing written, so we use a pretend word, <start>. For "put the red and blue blocks away", the four steps look at:

step   the command   2 words back   last word    → should write
1      (average)     <start>        <start>      put
2      (average)     <start>        put          red
3      (average)     put            red          blue
4      (average)     red            blue         <end>

Joined end to end, that's 12 + 12 + 12 = 36 numbers going in at every step. In Python, list + list joins two lists, so this is one line. Notice step 3: "put red" so far, and the command mentions blue, so the next word is blue. Step 4: red and blue are written, so it's time to stop.

Step 4: from 36 numbers to 10 scores

Now the neurons from Lesson 4. Each of 32 hidden neurons takes all 36 inputs, multiplies each by its own weight, adds them up, adds a bias, and squashes the result with tanh into the range −1 to 1:

hidden[j] = tanh( W1[j][0]×x[0] + W1[j][1]×x[1] + … + W1[j][35]×x[35] + b1[j] )

Then 10 output neurons, one per menu word, each take all 32 hidden values and do the same weighted sum (without the squash). Those 10 numbers are the scores. A tiny check of tanh: math.tanh(0) is 0, math.tanh(2) ≈ 0.96, math.tanh(−2) ≈ −0.96, and math.tanh(100) is 1.0. Big numbers get flattened.

Step 5: from scores to probabilities (softmax)

Scores can be any numbers, even negative. We want probabilities: all positive, adding up to 1. The recipe: raise e (about 2.718) to the power of each score, which makes everything positive and makes bigger scores much bigger, then divide each by the total. In Python that's math.exp. By hand, with three scores:

scores           2        1        0
math.exp(score)  7.389    2.718    1.000      total = 11.107
divide by total  0.665    0.245    0.090      ← adds up to 1

Now a problem. math.exp(1000) is too big for a computer and Python crashes with OverflowError: math range error. The trick: subtract the biggest score from all of them first. Scores 2, 1, 0 become 0, −1, −2:

math.exp(score)  1.000    0.368    0.135      total = 1.503
divide by total  0.665    0.245    0.090      ← exactly the same answer!

Same answer, because shifting every score by the same amount doesn't change their ratios. And now the biggest thing you ever compute is math.exp(0) = 1. No crash. Every real LLM uses this trick.

Step 6: how wrong was it? Loss, or "surprise"

Training needs one number that says how badly the model did. Ours is surprise: if the model gave the right word probability p, the surprise is −math.log(p).

Being confidently wrong is punished hardest. The grown-up name is cross-entropy loss. We add up the surprise for every word of an answer, including the <end>.

Step 7: where do the examples come from? (synthetic data)

Lesson 9 had 34 example commands. That's nowhere near enough to learn plans: it has no "red and blue" commands and no "everything except" at all. Writing hundreds of examples by hand is slow and boring. So we do what real AI labs do: make the data with a program.

Take a sentence template like "bin {x}". Fill the gap {x} with every colour, every pair of colours, every "all" phrase, and every "except" phrase, and write the right plan next to each. Real output of plan_examples(["bin {x}"], ["everything except the {c}"], random.Random(0)):

('bin the red one', ['put', 'red'])
('bin the blue one', ['put', 'blue'])
('bin the green block', ['put', 'green'])
('bin the red block and the blue block', ['put', 'red', 'blue'])
('bin the red and the green one', ['put', 'red', 'green'])
('bin the blue block and the red block', ['put', 'red', 'blue'])
...                                                   (6 pairs in all)
('bin everything', ['put', 'red', 'blue', 'green'])
...                                                   (4 "all" phrases)
('bin everything except the red', ['put', 'blue', 'green'])
('bin everything except the blue', ['put', 'red', 'green'])
('bin everything except the green', ['put', 'red', 'blue'])

One template gives 3 + 6 + 4 + 3 = 16 examples. Plans always list colours in the same order (red, blue, green), so "the blue block and the red block" is put red blue. That way there's only one right answer to learn. The rng picks between wordings like "the red block" and "the red one", so the examples don't all look identical.

The training set, from training_data():

8 put templates × (3 colours + 6 pairs + 4 "all" + 4 except × 3 colours)  = 200
Lesson 9's 34 commands, repeated 3 times                                  = 102
7 vague commands → ask, repeated 3 times                                  =  21
9 not-a-robot-job commands → none, repeated 3 times                       =  27
                                                                            350
minus every copy of a bake-off command (the robot's exam)                 −   7
                                                                    total = 343

Keep the exam out of the lessons. The 12 bake-off tasks (Lesson 19) are the robot's exam. Three of them were sneaking into the training data word for word: "put the red block in the bin" and "stop" are among Lesson 9's 34 examples (3 copies each), and the "bin {x}" template happens to make "bin everything except the green block". training_data() now removes every bake-off command, so 343 examples are left. Why it matters: a model can pass a test question it has already seen just by remembering the answer. Then the score tells you about its memory, not about whether it understands new commands. A test is only honest if the questions are new.

Why repeat the small groups 3 times? Otherwise the 200 template examples drown them out, and the model learns that almost everything is a put.

Held-out test. To check it learned the idea, test_data() builds 50 examples it never trains on, from different templates ("chuck {x} in the trash", "please put {x} in the bin", "all of them except the {c} one"), Lesson 9's 12 TEST commands, and new ask and none commands like "put it in the bin" and "what time is it".

Data matters. When this lesson was first built, the plan model learned from about 66 hand-written examples. It got only 20–40% of new plans right. The model was the same; the data was the problem. Generating examples from templates fixed it.

Step 8: learning = nudging 2,702 numbers

Where do the good numbers come from? The same trick as Lesson 3: gradient descent. For each example, work out which way to nudge each number to make the surprise a bit smaller, then nudge it a little (the learning rate is 0.05). Lesson 3 nudged 1 number. This nudges all of them:

word embeddings      88 words × 12      = 1,056
written-word embs    11 words × 12      =   132   (10 menu words + <start>)
hidden weights       32 × 36            = 1,152
hidden biases        32                 =    32
output weights       10 × 32            =   320
output biases        10                 =    10
                                  total = 2,702 parameters

Working out the nudges uses the chain rule from calculus, so train_step is written for you. Read it if you're curious: the comment softmax + cross-entropy marks the neat line where the "how wrong" for each word is just probability − 1 for the right word and probability − 0 for the others. Training goes through all 343 examples 25 times (25 epochs), in a shuffled order each time. That's 8,575 examples, and it took 10 seconds on our laptop.

Step 9: writing the answer, one word at a time

Once trained, writing is a loop. Read the command once. Then: predict, pick the most likely word, stop if it's <end>, otherwise add it and shift the "last two words" along. Always picking the top word is called greedy decoding. Here are the real probabilities for "put the red and blue blocks away":

step  looks at             top word   its probability
1     <start> <start>      put        1.0000
2     <start> put          red        0.9941   (blue 0.0057)
3     put red              blue       0.9705   (green 0.0148)
4     red blue             <end>      0.9921
→ ['put', 'red', 'blue']  →  [{'intent': 'put', 'object': 'red block'}, {'intent': 'put', 'object': 'blue block'}]

words_to_plan turns the words into a list of actions, and the robot does them one after the other. Before training, the same loop with the random starting numbers writes nothing at all: its very first pick is <end>, with only 0.1181 (green was 0.1123). Same machine, random numbers, no answer. Training is the only difference.

After training it writes exactly the right plan for all 343 training examples (100%) and for 46 of the 50 held-out ones (92%).

Step 10: now break it (this is the important part)

The 4 held-out examples it gets wrong tell you a lot:

'please put everything in the bin'      wanted put red blue green   wrote put red blue
'drive to the block'                    wanted ask                  wrote go red
'where is the block'                    wanted ask                  wrote find red
'wash the dishes'                       wanted none                 wrote find red

It forgets the third colour once, and three times it invents a colour (always red) for a command that has none.

That's a hallucination: a fluent, confident-looking answer that the input doesn't support. Our model now has words for "I don't know" (ask and none), and it uses them for commands that look like its ask and none examples: "make me a sandwich" gives none (0.9950). But it must still pick something. Give it "zzz" (no known words, so it reads 12 zeros) and it writes put red, and it nearly added blue too (<end> 0.5177, blue 0.4672).

Two more, and these matter for a robot:

• "tidy up the room" gives put red blue. It never saw "room", so it bins two blocks and forgets the green one.
• "stop the red block" gives go red (go 0.5092, find 0.4839, stop only 0.0034). "stop" is outnumbered by "the red block" in the average. You said stop and the robot drove off. A brain like this must never be trusted on its own. Next week, Lesson 10 builds a safety layer that checks every answer before the robot moves.

Is the bake-off fair? Yes: none of its 12 commands is in the training data (Step 7 removes them). So 8/12 is a real score on unseen commands, and so is the 92% on the 50 held-out ones.

Step 11: from our tiny model to Claude

Here's the honest comparison. The loop is the same. Everything else is bigger.

When Claude writes a reply, it is running Step 9: read everything, score every token, softmax, pick, append, repeat. You've now built that loop yourself.

3. Play with it

Things to try: "halt halt halt" (same as "halt": the average of identical lists is the same list). "blocks blue and red the put away" (order doesn't matter). "hello robot" and "zzz" (nothing useful known, yet a confident plan). "tidy up the room". Then try to find a command where one step is a close race.

4. Homework

Open homework/lesson15_tiny_lm.py. It imports tokenize from your Lesson 8, and TRAIN, TEST and extract_object from your Lesson 9, so those must be finished. The templates, TinyLM.__init__, train_step, make_targets, training_data, test_data, train_tiny_lm and accuracy are written for you. You write six things, in order.

4.1 softmax(scores)

Turn any list of scores into probabilities that add up to 1, using Step 5, with the subtract-the-biggest trick.

Example: softmax([2, 1, 0]) → about [0.665, 0.245, 0.090]. softmax([1000, 1000]) → [0.5, 0.5], with no crash.

4.2 TinyLM.read(self, command)

The average embedding of the command's known words (Step 2). Tokenize the command, keep only words in self.word_emb, and average their lists position by position. If no word is known, return [0.0] * E.

Example: for a model with vocabulary {"red", "block"}, read("red block") is the average of the "red" and "block" lists, and read("zzz qqq") is twelve 0.0s.

4.3 TinyLM.forward(self, ctx, prev2, prev1)

One prediction step (Steps 3–5). Join ctx with the written-word embeddings of prev2 and prev1 (from self.out_emb) into 36 numbers, x. Compute the 32 hidden values, then the 10 scores, then softmax. Return (x, hidden, probs): train_step needs all three.

Example: forward(model.read("red block"), START, START) returns an x of length 36, a hidden of length 32, and 10 probabilities adding to 1.

4.4 TinyLM.generate(self, command, max_words=MAX_WORDS)

Greedy decoding (Step 9). Read the command once. Start with prev2 = prev1 = START. Up to max_words times (5 by default): forward, take the word in OUT_WORDS with the biggest probability, stop if it's "<end>", otherwise append it and shift. Return the words, without "<end>".

Example (after training): generate("put the red and blue blocks away") → ['put', 'red', 'blue'].

4.5 words_to_plan(words)

Turn the model's words into something the robot can do: a list of action dictionaries (the same shape as Lesson 9's parse_command), or the word "ask" or "none". Return None if the words don't make sense.

["ask"]                 → "ask"
["none"]                → "none"
["stop"]                → [{"intent": "stop", "object": None}]
["go", "red"]           → [{"intent": "go", "object": "red block"}]      (go and find: exactly one colour)
["put", "red", "blue"]  → [{"intent": "put", "object": "red block"}, {"intent": "put", "object": "blue block"}]
["go", "red", "blue"]   → None   (go can't go to two places)
["put", "red", "red"]   → None   (a repeated colour)
["red", "put"], ["put", "purple"], ["put"], [], ["stop", "red"]  → None

4.6 plan_examples(put_templates, except_templates, rng)

Make the synthetic data from Step 7. For every template in put_templates, add, in this order:

1. each colour c: the template with x = rng.choice(["the c block", "the c one"]) → ["put", c];
2. each ordered pair (a, b) of different colours (6 pairs): x = rng.choice(["the a and b blocks", "the a block and the b block", "the a and the b one"]) → ["put", a, b] with the colours in COLOURS order;
3. each phrase in ALL_PHRASES: x = the phrase → ["put", "red", "blue", "green"];
4. each except template e and each colour c: x = e.format(c=c) → ["put"] + the other two colours, in COLOURS order.

Return a list of (command, plan words) pairs. Loop in exactly this order (colours in COLOURS order) so your rng choices match the checker's.

Example: plan_examples(["bin {x}"], ["everything except the {c}"], random.Random(0)) gives the 16 examples printed in Step 7, starting with ('bin the red one', ['put', 'red']) and ending with ('bin everything except the green', ['put', 'red', 'blue']).

Run it

cd homework
python3 lesson15_tiny_lm.py

What you should see (real output of the reference solution). The last line is wrong on purpose: it's a failure from Step 10. "what time is it" is right, but only just (Step 10).

343 training examples (made from templates), 50 held-out test examples
loss: first epoch 3.59 -> last epoch 0.049   (10 s)
exact plan accuracy: train 100%, held-out test 92%
  'put the green and red blocks away'          -> put red green      -> [{'intent': 'put', 'object': 'red block'}, {'intent': 'put', 'object': 'green block'}]
  'throw out all of them except the blue one'  -> put red green      -> [{'intent': 'put', 'object': 'red block'}, {'intent': 'put', 'object': 'green block'}]
  'put it in the bin'                          -> ask                -> ask
  'what time is it'                            -> none               -> none
  "where's the green one?"                     -> put                -> None

Your numbers should match exactly (the seconds will differ): the starting numbers and the shuffling come from a fixed random seed (0), so everyone's model trains the same way.

Check it

cd ..
python3 check.py 15

Six checks: softmax, read, forward, words_to_plan, plan_examples, and trains and writes plans (which trains the model, so it needs everything else, and takes a little while). Green ✓ = passed, red ✗ = read the hint, yellow … = not written yet. If the last one says not written yet (Lesson 9: write this function), it's your Lesson 9 file that's unfinished. Real hints:

• ✗ softmax: softmax([1000, 1000]) crashed or was wrong: subtract the biggest score first. You skipped the trick from Step 5.
• ✗ read: no known words -> a list of E zeros. You crashed, or returned an empty list, when nothing in the command was known.
• ✗ words_to_plan: go/find take exactly one colour. Only put may list several colours.
• ✗ plan_examples: the pair (blue, red) is still written in COLOURS order -> ['put', 'red', 'blue']. You wrote [a, b] instead of sorting into red, blue, green order.
• ✗ trains and writes plans: held-out plan accuracy should be at least 80% (got …). Usually plan_examples is subtly off, so the model learned from the wrong data.

Stretch

• A confidence alarm. Change generate (in a copy) so it gives up and returns [] if any picked word had probability below 0.7. Try the five commands in the file and "stop the red block". Which get caught? Which wrong answer slips through, and why does that matter?
  Compare with ours

  Caught: "what time is it" (none won with only 0.5250, and it was right) and "stop the red block" (go won with 0.5092, wrong). Not caught: "where's the green one?" writes put with its lowest step at 0.8647, but words_to_plan already rejects that. Over all 343 training and 50 held-out examples, 5 get caught, and only one of them, "where is the block" (0.5134), was wrong. The other 4 were right: false alarms. The other 3 wrong held-out answers slip through with their lowest step between 0.80 and 0.98. Confidence is a clue, not a guarantee.

• Teach it a new phrase. "tidy up the room" fails because "room" was never seen. Add "the room" to ALL_PHRASES (in a copy), retrain, and test it. Does it fix that command? Does it now bin everything for "tidy up the red block in the room"? Adding data fixes one thing and can break another.
• Shrink it. Set E to 2 and H to 4 and retrain. How many parameters now? Does held-out accuracy survive? Where's the smallest model that still gets at least 80%?

Stretch 2: attention (how Transformers read)

This one is bigger than the other stretches: a new idea, a new file and one new function. It's the idea at the centre of every real LLM, so it's worth the effort.

The problem with averaging

Look again at Step 2. read squashes the whole command into one average, once, before writing anything. That has two weaknesses:

1. Every word counts the same. In "put the red and blue blocks away", the word "red" is 1 of 7, so it gets 1/7 of the vote. "the", "and" and "away" get just as much.
2. It's the same at every step. When the model writes word 1 it needs the action. When it writes word 2 it needs the first colour, and for word 3 the next colour. But every step sees exactly the same average.

What we want: at each step, the model chooses which words to look at. That's attention.

The idea: ask a question, score every word

At each step, the word the model has just written asks a question. Its embedding is the query. After <start>, the question is roughly "which words tell me the action?". After "put", it's "which words tell me the first colour?". Nobody writes these questions down: training learns them, just like it learns the embeddings.

Each command word's embedding is its key. Then:

1. Score every word: how well does its key match the query? We use the dot product (multiply the two lists position by position and add up), divided by √E. A big score means "this word answers my question".
2. Softmax the scores into attention weights that add up to 1. You wrote this in 4.1.
3. The context is the weighted average: each word's embedding times its weight, added up.

If every weight is equal, that's exactly the old average. Attention just lets the weights be unequal, and different at every step.

Worked example by hand

Three words, and embeddings of just 2 numbers (so E = 2 and √E ≈ 1.414). Made-up numbers to keep the sums easy:

put  → [1, 0]       red  → [0, 2]       the  → [0.5, 0.5]
query after "put" (looking for a colour) → [0, 1]

Scores (dot product ÷ 1.414):

put:  (0×1 + 1×0)   / 1.414 = 0
red:  (0×0 + 1×2)   / 1.414 = 1.414
the:  (0×0.5 + 1×0.5) / 1.414 = 0.354

Softmax: e⁰ = 1, e^1.414 ≈ 4.113, e^0.354 ≈ 1.424. Total ≈ 6.537.

weights:  put 0.153   red 0.629   the 0.218      (they add up to 1)
context = 0.153×[1, 0] + 0.629×[0, 2] + 0.218×[0.5, 0.5] ≈ [0.262, 1.367]
plain average                                           ≈ [0.500, 0.833]

The real model: where does it look?

AttnTinyLM is your TinyLM with one change: instead of read once, it calls attend at every step, with the embedding of the word it just wrote as the query. It trains on exactly the same 343 examples. Real output, the top 3 weights at each step:

  'put the red and green blocks away'
    after '<start>' looks at  green:0.18  blocks:0.17  away:0.16   -> writes 'put' (1.00)
    after 'put'     looks at  red:0.77  and:0.07  put:0.04         -> writes 'red' (1.00)
    after 'red'     looks at  red:0.41  and:0.33  away:0.07        -> writes 'green' (1.00)
    after 'green'   looks at  the:0.27  green:0.21  away:0.13      -> writes '<end>' (1.00)
  'throw out all of them except the blue one'
    after '<start>' looks at  all:0.16  them:0.14  throw:0.13      -> writes 'put' (1.00)
    after 'put'     looks at  all:0.60  except:0.08  them:0.08     -> writes 'red' (1.00)
    after 'red'     looks at  blue:0.39  all:0.37  except:0.05     -> writes 'green' (1.00)
    after 'green'   looks at  the:0.22  one:0.17  out:0.10         -> writes '<end>' (1.00)

At the first step the weights are spread out: every word gets a similar share. After "put", one word grabs most of the weight: "red" (0.77) in the first command, "all" (0.60) in the second. The second command's third step is the clever one: to decide what comes after red, it looks hardest at "blue" (0.39), the colour you said to leave out, together with "all" (0.37), and writes green. Nobody told it to do this. Training found that it helps.

Does it help?

The file trains both models on the same data with 3 different random starts (seeds) and scores exact plans on the 50 held-out examples. Real output:

Attention is a little better on average, but not on every seed: on seed 0 they tie, and on seed 1 the averaging model wins. What attention clearly gives is steadiness: 92% on every seed, while the averaging model swings from 80% to 94% depending on its random start. That's why we run more than one seed. With seed 0, attention fixes "please put everything in the bin" (it writes all three colours) and writes stop for "stop the red block", where the averaging model wrote go red. But it makes a new mistake: for "go to the green block" it writes just go, which words_to_plan rejects.

On the robot, though, the bake-off score is the same: 8/12. The same tasks pass. On "tidy up the room" it now asks which block instead of binning two, which is safer but still a fail.

A result that changed. An earlier version of this page buried the colour among filler words ("put the red block the in bin block to") and reported attention holding on to the colour better. That was with the old, much smaller training set. With the new data, that difference turned out to be noise between seeds, so we dropped it. Even the held-out numbers above moved when we removed the 7 bake-off copies from the training data. Results from one small experiment can change when the data changes. Re-run, and use several seeds.

What attention does NOT fix

• Word order. "put the block in the red bin" → ['put', 'red'], from both models. Our attention only compares embeddings, and "red" has the same embedding wherever it sits, so shuffling the words gives exactly the same weights. "go to the blue block not the red one" gives ['go', 'blue'] from the averaging model (right, but only by luck: it can't read "not") and ['go', 'red', 'go'] from attention, which words_to_plan rejects. Real Transformers add position information to each embedding, so "red" before "block" and "red" before "bin" look different.
• Unknown words. For "where's the green one?", "where's" isn't in the vocabulary, so it has no key and can't be attended to. The attention model looks at "the green one" and writes put green, and the robot bins the green block. (The averaging model wrote just put, which was rejected. Here, being worse at writing a full plan was accidentally safer.) Attention chooses among the words it knows. It can't invent meaning for a word it has never seen.

Things to try: compare "put the red block in the bin" with "put the block in the red bin" (same weights, just in different columns). Watch the first row of "stop the red block": "stop" gets more than half the weight. Then type a long command with lots of "the"s and see what happens to the colour's share.

Your homework for this stretch

Open homework/lesson15_attention.py. It imports your finished lesson15_tiny_lm.py. AttnTinyLM (including its training, with the chain rule written out through attention) and train_attn_lm (which reuses your Lesson 15 training loop with cls=AttnTinyLM) are provided. You write one function.

attend(query, keys): query is a list of E numbers; keys is a list of word embeddings. Score each key with the dot product ÷ √E, softmax the scores, and build the weighted average. Return (context, weights). Example: attend([0.0] * E, [[1.0] * E, [0.0] * E]) gives weights [0.5, 0.5] (every score is 0) and a context of twelve 0.5s.

Run it with python3 lesson15_attention.py (it trains six models, so give it a few minutes) and you should see the outputs above. Check it with python3 check.py 15a: three checks, attend: equal scores, attend: focus and attention model learns (it must write ['put', 'red'] for "put the red block in the bin" and get at least 85% of held-out plans). Two real hints: "a query of zeros scores every word 0, so the weights are equal (0.5, 0.5)" and "the weights must add up to 1".

Want the full story? 30u30 Day 13: Attention explains queries, keys and values, and why we divide by √E.

5. Real-world challenge: be the language model

You'll play the model, with a friend or family member, and measure your own loss and your own hallucinations. You need 10 coins (or beans, or Lego bricks), paper and a pencil.

1. Round 1: next word, with bets. Your partner secretly writes a short, ordinary sentence of 6–8 words (e.g. something they'd text you). They tell you the first word. Before each next word, you split your 10 coins between up to three guesses. Then they reveal the real word.
2. Score your surprise for each word from the coins you put on the right word, using this table (it's −log(coins/10) from Step 6): 10 coins → 0, 8 → 0.22, 5 → 0.69, 3 → 1.20, 2 → 1.61, 1 → 2.30, 0 → 4.6 (a penalty, as if you'd bet 1 coin in 100). Add up the sentence. Do 3 sentences, then swap roles.
3. Notice how you bet. After "put the kettle", were you nearly sure of "on"? After "I", were your coins spread out? That's softmax: a sharp or a flat spread of probability.
4. Round 2: no "I don't know" allowed. Your partner asks 5 questions you can't possibly know the answer to, like "What's the name of my teacher's dog?" or "What did I eat on 3 March last year?" Rule: you must answer instantly, with one confident specific answer. Write down each answer and whether it was right.
5. Round 3: same game, but "I don't know" is allowed. Ask 5 new unknowable questions. Count how often you use it.

Record: your total surprise for each sentence (who was the better "model"?), your 5 forced answers from Round 2 and how many were right, and how Round 3 felt different. What it should teach you: a good language model isn't one that's always sure; it's one whose surprise is low on real text. And when you take away the "I don't know" option, you get confident nonsense, every time. That's not a bug in you, or in TinyLM. It's what happens to any system that must always answer.

6. Your Robo-Helper so far

Robo-Helper now has a second brain. Lesson 9's brain sorts commands into piles and does one thing. This one writes a plan, word by word, and words_to_plan turns it into a list of the same {"intent": ..., "object": ...} actions the rest of the robot already uses, or into "ask" (the robot asks "Which block do you mean: red, blue or green?") or "none" (it says that's not something it can do). In the simulator, brains.py's tinylm brain first runs your Lesson 8 typo fixer on the command, then runs every action through Lesson 10's safety check.

It keeps coming back:

• Week 7: Lesson 10's validate_action checks every action before the robot moves. That's exactly what a brain that turns "stop" into "go" needs.
• Week 8: Lesson 16 puts a stopwatch on this brain and compares it with a brain running far away on the internet.
• Week 9: Lesson 17 shrinks the attention model's numbers (count_weights, quantized_copy) and retrains it with a teacher's labels on new phrasings (distill_examples, passed to train_attn_lm as extra).
• Week 11: Lesson 19 runs tinylm against every other brain on the same 12 tasks.

7. Common mistakes

• Not subtracting the biggest score in softmax. Fine for small scores, then OverflowError on big ones. Checker hint: "subtract the biggest score first".
• Averaging over all tokens instead of known ones, or looking up unknown words (KeyError), or dividing by zero when none are known. Checker hint: "no known words -> a list of E zeros".
• Making x the wrong length. ctx + emb + emb joins three 12-number lists into 36. If you add them number by number you get 12, and the weights won't line up. Checker hint: "x has 3*E numbers, hidden has H".
• Forgetting to shift prev2, prev1 in generate. Then every step sees the same input and writes the same word again and again.
• Accepting nonsense in words_to_plan. ["put", "red", "red"] and ["go", "red", "blue"] must be None. Checker hints: "a repeated colour makes no sense -> None" and "go/find take exactly one colour".
• Colours in the wrong order in plan_examples. "the blue block and the red block" is ["put", "red", "blue"]. Checker hint: "the pair (blue, red) is still written in COLOURS order".
• Calling rng.choice in the wrong places. One call per colour and one per pair, nothing for the "all" and except examples. Otherwise the wordings don't match. Checker hint: "first: the red block, plan ['put', 'red']".

8. Check yourself

9. Go deeper (optional)

• 30u30 Day 13: Attention: the fix for our flat average. Each word decides how much to look at every other word.
• 30u30 Day 14: The annotated Transformer: the full machine real LLMs use, built from the same pieces you just used (embeddings, softmax, cross-entropy).
• 30u30 Day 1: RNN effectiveness: a neural network that writes text one letter at a time. Same loop, a cleverer memory.
• TensorTonic: practise writing softmax, cross-entropy and friends yourself.
• Inference engineering: what it takes to run the "predict, pick, append, repeat" loop fast for millions of people. Weeks 8 and 9 start on this.