The maths behind robots, learned by playing: ten interactive explainers, each one used to make Robo-Helper better

Linear algebra, Bayes, Markov chains, Kalman filters, A*, Fourier, momentum, reinforcement learning, Transformers and diffusion. Play with each idea in your browser, then use it on your own robot and measure the difference.

Source: Martin (@0x_meden), “Learn the math behind robots by playing with it”, 8 Nov 2026 · X post · quoting his guide “How π0, GR00T and Optimus actually think” (5 Nov 2026). All ten links checked and working on 9 Nov 2026.

Going deeper: each section links an explainer to the exact Robo-Helper lesson that uses the idea, with a tested experiment and a challenge. At the end there's a summary of the quoted guide on how π0 and GR00T are built.

1. Summary

2. How to use it with Robo-Helper

• Run the experiments: cd topics/18-robot-brain/project/solutions then python3 bonus_robot_math.py. It runs in well under a second and needs nothing installed. Your homework folder has the same file once your earlier lessons are done.
• Each experiment changes one thing and measures it. Three of them are honest surprises, which is the point: momentum can overshoot, more exploring isn't always better, and a moving average lags behind a moving robot.
• Do them in Robo-Helper order: linear algebra and Kalman/Fourier/Bayes after week 2, A* after week 4, Markov after week 5, Transformers after week 6. Diffusion is for after the course.

3. Linear algebra

Play: Immersive Math: “Immersive Linear Algebra”, a whole book with interactive figures. Try the vectors chapter and the matrices chapter: drag a vector and watch a matrix move it.

The idea: a robot sees the world from its own point of view ("2 squares ahead of me"). To use that on a map, it rotates the reading by the way it's facing, then adds where it is. Rotation is a 2×2 matrix of cosines and sines. Arms are the same idea stacked joint by joint (forward kinematics, Module 02).

In Robo-Helper: week 1 moves on a grid with fixed directions, so it never needed rotation. A real robot can face any way.

1) LINEAR ALGEBRA: the robot at (4, 3) facing 90 degrees (down the screen) sees something 2 squares ahead.
   rotate((2, 0), 90) = (0.0, 2.0)  ->  it is at room square (4, 5)

Challenge: the robot faces 180° and reads 3. Where is the thing? Predict, then check with sensor_to_room((4, 3), 180, 3). (Remember y grows downwards in our room.)

4. Probability and Bayes

Play: Seeing Theory (Brown University). Do "Basic Probability" then "Bayesian Inference": flip coins, and watch a belief update as evidence arrives.

The idea: start with a belief (every square equally likely). Each sensor reading makes some squares more likely and some less: multiply by how well each square explains the reading, then rescale so it adds up to 1. That's Bayes' rule, and it's how real robots work out where they are (localisation).

In Robo-Helper: week 2 averages noisy readings; week 5's word counter is Bayes too (the "prior" is how common each intent is).

2) BAYES: where is the cat? The robot at (1, 1) takes noisy 'distance east' readings (noise 1.0).
    1 readings: most likely square is the right one 50% of the time
    4 readings: most likely square is the right one 92% of the time
   16 readings: most likely square is the right one 100% of the time

500 trials each. Our sensor's noise is flat (anywhere within ±1), but the belief update assumes a bell curve. It still works, which is common: the model doesn't have to be exactly right to be useful.

Challenge: double the noise to 2.0. How many readings do you need now to be right 90% of the time?

5. Markov chains

Play: Setosa: Markov chains, explained visually. Change the arrows' probabilities and watch where the dot spends its time.

The idea: a Markov chain is "what happens next depends only on where you are now", with a probability for each next step. Run it for a long time and you get where it spends its time (the stationary distribution). Your week 5 next-word model is a Markov chain over words.

In Robo-Helper: the cat. Give it a favourite bed at (4, 1) and let it wander. Then let A* charge extra for squares the cat likes.

3) MARKOV CHAIN: a wandering cat with a favourite bed at (4, 1). Route to the blue block avoiding where it usually is.
   cat's favourite squares (long-run share): (4, 1) 32.9%, (5, 1) 23.1%, (3, 1) 19.8%
   shortest  route:  7 steps, expected run-ins with the cat per trip 0.828
   cat-aware route:  9 steps, expected run-ins with the cat per trip 0.060

Two extra steps cut run-ins by more than 90%. An honest note: our first try had a cat that wandered with no favourite spot. It spent its time almost evenly everywhere, so the "cat-aware" route was identical and no better. The model only helps if the cat has habits.

Challenge: try pull=0.3 (a lazier preference for the bed) and a cost of 1 + 5 * pi[s]. When does avoiding the cat stop being worth the extra steps?

6. Kalman filters

Play: Tim Babb: How a Kalman filter works, in pictures. The best-known plain explanation, built from blobs of uncertainty.

The idea: two sources of information, both imperfect: what you expect (I'm driving at 0.5 m/s, so the wall should be 5 cm closer) and what you measure (the noisy sensor). The Kalman filter blends them, trusting each in proportion to how sure it is. Every self-driving car and drone runs one (or a cousin of it).

In Robo-Helper: week 2 averaged 16 readings while standing still. That fails when the robot moves, because the old readings are out of date.

4) KALMAN FILTER: driving towards a wall at 0.5 m/s, sensor noise 1.0 m, 10 readings a second.
   raw readings         typical error 0.56 m
   average of last 16   typical error 0.54 m
   Kalman filter        typical error 0.22 m

Challenge: the filter has one knob, q (how much the robot doubts its own speed). Try 0.0001 and 1.0. Which way does it go wrong?

7. Path planning: A*

Play: Red Blob Games: Introduction to A*. Drag walls and goals and compare breadth-first search, Dijkstra and A*.

The idea: search outwards from the start, but always look next at the square that seems closest to the goal ("steps so far + a guess of steps left"). A good guess means far fewer squares looked at. Trust the guess more (weighted A*) and it searches even less, but may no longer find the shortest path.

In Robo-Helper: week 4.

5) A*: the room, start to bin. Plain search vs A* vs weighted A* (trusting the guess more).
   no heuristic (Dijkstra)  path 11 steps, squares looked at 40
   A* weight 1              path 11 steps, squares looked at 35
   A* weight 2              path 11 steps, squares looked at 13
   A* weight 5              path 11 steps, squares looked at 13

In our small room, weighted A* still finds the shortest path. That isn't guaranteed: weighted A* only promises a path at most w times the shortest. On a big map, that trade (a slightly longer path, found much faster) is what a robot with a deadline wants.

Challenge: add walls to ROOM in a copy until weighted A* with w = 5 returns a longer path than plain A*.

8. Signals: Fourier

Play: Jez Swanson: An Interactive Introduction to Fourier Transforms. Draw a shape and watch circles spinning at different speeds trace it out.

The idea: any wiggly signal is a sum of simple waves at different speeds (frequencies). The Fourier transform tells you how much of each. If a wobble comes from the wheels at a steady rhythm, it shows up as one spike, and you can remove just that spike.

In Robo-Helper: week 2's sensor. Averaging can't remove a steady rhythm well; Fourier can.

6) FOURIER: a 64-reading sensor log with a wheel wobble (8 cycles) and noise.
   strongest rhythm found at 8 cycles per log (we put it at 8)
   typical error vs the true distance: 0.44 m before, 0.11 m after removing it

This works on a log you've already recorded. A robot cleaning its readings live would use a filter that only looks backwards; the idea is the same.

Challenge: make the wobble 2 cycles instead of 8. Does the method still find it? (The slow downward drift of the true distance also lives at low frequencies.)

9. Optimisation: gradient descent with momentum

Play: Gabriel Goh: Why Momentum Really Works (Distill). Slide the step size and momentum and watch the path to the bottom of the valley.

The idea: plain gradient descent takes a step downhill each time. Momentum remembers the last steps and keeps rolling, like a heavy ball, so it speeds up along long gentle slopes. Too much momentum and it overshoots and swings.

In Robo-Helper: week 2 fits the braking distance with plain gradient descent.

7) MOMENTUM: Lesson 3's braking-distance fit. Steps until the error is below 0.001.
   learning rate 0.1, momentum 0.0: 73 steps
   learning rate 0.1, momentum 0.5: 35 steps
   learning rate 0.1, momentum 0.9: 37 steps
   learning rate 0.3, momentum 0.0: 24 steps

Momentum halves the steps at the same learning rate, but 0.9 swings and does no better than 0.5. On this easy problem, a bigger learning rate on its own does best. Momentum matters most on long, narrow valleys, which is exactly what the Distill page shows. Big networks have lots of those.

Challenge: try learning rate 0.6 without momentum. What happens, and why does the Distill page say so?

10. Reinforcement learning

Play: Andrej Karpathy: REINFORCEjs. Run the GridWorld demos and watch values spread from the goal; try the "Waterworld" deep Q-learning agent.

The idea: no teacher, only rewards. The robot tries things, remembers how good each move was in each square (its Q table), and slowly prefers the moves that led to reward. It has to explore (try new moves) and exploit (use what it knows).

In Robo-Helper: the Q-learning bonus, and DQN against A* in Module 20.

8) REINFORCEMENT LEARNING: Lesson 7's maze. How much exploring is best? (5 seeds each)
   epsilon  0.0: learned route found by 5/5 seeds (length 7), first 9-step-or-better episode: [7, 7, 7, 7, 7]
   epsilon 0.05: learned route found by 5/5 seeds (length 7), first 9-step-or-better episode: [11, 3, 9, 7, 5]
   epsilon  0.2: learned route found by 5/5 seeds (length 7), first 9-step-or-better episode: [9, 8, 7, 9, 7]
   epsilon  0.5: learned route found by 5/5 seeds (length 7), first 9-step-or-better episode: [18, 14, 23, 21, 7]

A surprise: with no random exploring at all (epsilon 0) it learns fastest. Why? Every step costs −1, and untried moves start at 0, so an untried move always looks better than one that's already cost something. The robot explores on its own ("optimism"). In a big world with distant rewards that trick isn't enough, and exploring matters.

Challenge: change the step reward from −1 to 0 in a copy of q_learn. Does epsilon 0 still work?

11. Transformers

Play: Polo Club (Georgia Tech): Transformer Explainer. A real GPT-2 running in your browser. Type a sentence, then click into the attention heads and move the temperature slider.

The idea: every word asks every other word "how relevant are you to me?" (queries times keys), turns the scores into weights with softmax, and takes a weighted mix of their information (values). Stack that many times and you have the "understanding" half of π0 and GR00T.

In Robo-Helper: you built single-head attention, with backprop by hand, in the week 6 stretch. Its measured gain: new commands 89% → 92% on average over 3 seeds, the same 8/12 on the bake-off. A real nanoGPT on the same robot data is in Module 20.

Challenge: in the explainer, set the temperature to 0.2 and then 2.0 on the same prompt. Then find where Robo-Helper's tiny LM uses softmax, and say what a temperature would do to its plans.

12. Diffusion, and how π0 and GR00T move

Play: Polo Club: Diffusion Explainer. Watch Stable Diffusion turn pure noise into a picture, step by step, and change the prompt and the guidance.

The idea: train a network to remove a little noise at a time. Then start from pure noise and run it again and again until a clean picture appears. Robots use the same trick for motion: start from a random wobble of joint positions and refine it into a smooth arm movement.

What the quoted guide says about π0 and GR00T (summary)

• The same two-part design. A vision-language model on top that understands the scene and the instruction, and a smaller "action head" below that produces motion. Per the guide, π0 starts from PaliGemma (3B) with a 300M-parameter action expert, and GR00T N1.7 uses Cosmos-Reason2-2B with a 16-layer diffusion-transformer action head that runs at about 28 Hz on a Jetson AGX Thor.
• Flow matching, a cousin of diffusion. Both action heads start from noise and refine it into a trajectory, the way an image model refines noise into a picture.
• Action chunks. π0 outputs a chunk of 50 future actions at once. Copying human demos one step at a time drifts: a small mistake leads somewhere the demos never went. Predicting a chunk is one of the main fixes.
• The real difference is the data. π0: over 10,000 hours of humans driving real robots. GR00T: 20K hours of first-person video of human hands, plus robot demos. Optimus: human video and simulation (its walking: PPO on about a billion simulated steps, per a Stanford lecture by Ashish Kumar). Tesla publishes no architecture or weights.
• Open models. openpi (π0, π0-FAST, π0.5; Apache 2.0; inference needs a GPU with more than 8 GB) and Isaac-GR00T (code Apache 2.0, weights under NVIDIA's Open Model License; inference 16 GB). LeRobot records your own demos and fine-tunes either.

These figures are as reported in the guide, which cites the π0 paper, NVIDIA's repo and the talks it links. We haven't run either model.

In Robo-Helper: no diffusion, and we're honest about why: the robot moves on a grid with A*, so there's no smooth arm motion to generate. But one idea is already there. Since the week-6 rebuild, the tiny LM writes a whole plan at once ("put red blue") instead of one action, which is an action chunk in words. It's what took the bake-off from 5/12 to 8/12.

Challenge (after the course): pick one lecture from the guide's watching order (start with Stanford AA203 lecture 15 on flow matching and action chunks), then read the openpi README and write down what you'd need to run π0 in the LIBERO simulator.

13. Playbook

1. Play first, for 10–15 minutes, before reading anything. Make a prediction, then test it on the page.
2. Find the matching Robo-Helper lesson and reread the part that uses the idea.
3. Run bonus_robot_math.py. Read the before and after numbers, and say in one sentence why they changed.
4. Do the challenge: change one number, predict what will happen, then run it.
5. Keep the honest surprises. When an idea didn't help (momentum 0.9, epsilon 0.5, a cat with no habits), write down why. That's real understanding.

14. Self-check

15. Glossary & further reading

• The ten explainers: Immersive Linear Algebra · Seeing Theory · Markov chains · Kalman filter in pictures · A* · Fourier · Momentum · REINFORCEjs · Transformer Explainer · Diffusion Explainer
• From the guide: openpi · Isaac-GR00T · LeRobot · Stanford AA203 lecture 15 (flow matching, action chunks) · Berkeley CS 285 lecture 2 (why copied behaviour drifts)
• Related: Module 22 (gradient, divergence, curl: steering a robot with a field), Module 02 (drill the maths), Module 03 (Bayes for intent), Module 06 (attention, PPO papers), Module 07 (behaviour cloning, RL), Module 18 (Robo-Helper), Module 20 (minimalRL, nanoGPT)