LLM + RoboticsModule 20 · From-scratch code · 5 of 6

tinygrad: a deep learning framework you can read

Something between micrograd and PyTorch. It waits until you need an answer, then glues your maths into as few GPU programs as it can.

Repotinygrad/tinygrad
LicenceMIT
Stars33,856 (7 Nov 2026, GitHub API)
Core code9,318 lines (the repo's own sz.py); 26,058 lines in tinygrad/ without auto-generated files
StatusActively maintained by tiny corp. Last push 7 Nov 2026. The README says it "isn't 1.0 yet".
NeedsPython ≥ 3.11, no required pip packages. Runs on CPU, Metal (Mac GPU), CUDA, AMD and more.

Tested at commit 853852d (7 Nov 2026) on a MacBook with an Apple M4. Short code excerpts are quoted with credit to the tinygrad authors (MIT).

On this page
  1. Start here (no maths needed)
  2. Summary
  3. How the code works
  4. Run it, tested
  5. Connect to Robo-Helper (Module 18)
  6. Why it matters for robotics
  7. Reality check
  8. Self-check
  9. Further reading

Start here (no maths needed)

tinygrad doesn't do your sums straight away. It writes them down as a plan, then turns the whole plan into a few fast programs for the computer's chip.

Think of it like this

You need eggs, milk and bread. You could walk to the shop three times, once for each thing. Or you could write a shopping list and go once. tinygrad is the shopping-list person. Every time you ask it to multiply or add, it only adds a line to the list. When you finally say "I need the answer now", it looks at the whole list and does it in as few trips as possible. Each "trip" is a small program, called a kernel, that runs on the chip.

How it works

  1. You write normal-looking maths: multiply these, add those, take the biggest.
  2. tinygrad doesn't calculate yet. It builds a plan (a graph of steps).
  3. When you ask for a result, it squeezes neighbouring steps together. This is called fusing.
  4. It writes real code for your chip (C for the CPU, Metal for a Mac's graphics chip) and runs it.
  5. Learning works the same way: it plans the "which way should each number move?" sums too, then runs them.

Why robots care

A robot has a small computer and a battery. It can't waste trips. Fewer, bigger programs mean faster answers and less power. comma.ai's openpilot, a driver-assistance system that runs on a small computer in a car, uses tinygrad to run its driving model.

Try it

Words to know

Tensor
A box of numbers: a list, a grid, or a stack of grids.
Lazy
Waits until the answer is really needed before doing the work.
Kernel
One small program that runs on the chip and does a chunk of the maths.
Fusing
Joining several steps into one kernel so the chip does one trip, not many.
Backend
The part that knows how to talk to one kind of chip: CPU, Mac GPU, NVIDIA GPU and so on.
Autograd
The computer works out the "which way to nudge each number" maths for you.

Going deeper: everything below is the full version. On a first read, look at the real outputs in Run it, tested and the Robo-Helper link, then come back.

1. Summary

TL;DR

tinygrad is a complete deep learning stack (tensors, autograd, a compiler, optimisers, datasets) that its README places "between PyTorch and karpathy/micrograd". Its trick is laziness: operations build a graph, and only .realize() (or reading a value) turns that graph into kernels. Because it sees the whole graph first, it can fuse many operations into one kernel. On our Mac, a matmul written as "reshape, multiply, sum" became exactly 1 kernel on the Metal GPU. Forcing the multiply to happen first made 2 kernels and a 67 MB temporary array.

2. How the code works

There are four layers. Each is a folder in tinygrad/.

  1. Tensor front end (tensor.py, mixin/). Methods like reshape, * and sum don't touch numbers. Each one wraps a UOp: a node that says "this is a multiply of those two inputs". So c = a * b just grows a graph.
  2. Autograd (mixin/gradient.py). .backward() walks the graph backwards and adds more UOps for the gradients. Gradients are also lazy, so the backward pass gets fused too.
  3. Scheduler (schedule/). When you call .realize(), it decides where a result must be stored in memory, and cuts the graph into kernels at those points. Everything between two stored results becomes one kernel.
  4. Codegen and runtime (codegen/, renderer/, runtime/). Each kernel is optimised (loop splitting, tensor cores, vector loads), then a renderer writes source code: C for the CPU (ClangRenderer), Metal Shading Language for Apple GPUs (MetalRenderer), CUDA, and others. A runtime such as ops_metal.py (282 lines) compiles it and launches it.

The whole "lazy" idea fits in one method. Nothing runs until here:

# tinygrad/tensor.py (MIT, tinygrad authors), trimmed
def realize(self, *lst:Tensor, do_update_stats=True) -> Tensor:
  """Triggers the computation needed to create these Tensor(s)."""
  to_realize = [x for x in (self,)+lst if x.uop.base.needs_storage()]
  if len(to_realize):
    linear, var_vals = Tensor.linear_with_vars(*to_realize)
    run_linear(linear, var_vals, update_stats=do_update_stats)
  return self

linear_with_vars builds the list of kernels. run_linear compiles and runs them. Reading a value with .item(), .tolist() or .numpy() calls realize for you.

The one idea that makes it click

Look at the README's matmul: (a.reshape(N,1,N) * b.T.reshape(1,N,N)).sum(axis=2). Read literally, it builds an N×N×N cube of products, then sums it. For N = 1024 that cube would be about a billion floats, 4 GB. An eager library would build it. tinygrad sees "multiply, then sum" in the same graph, so it writes one kernel that multiplies and adds inside a loop, never storing the cube. Here is the actual C it generated on the CPU backend for N = 4 (DEBUG=4 DEV=CPU):

// generated by tinygrad for N = 4, trimmed
for (int Lidx1 = 0; Lidx1 < 4; Lidx1++) {
  float4 val3 = *((float4*)(((float*)(val1))+alu0));   // one row of a
  for (int Lidx2 = 0; Lidx2 < 4; Lidx2++) {
    ...
    *(((float*)(val0))+(alu0+Lidx2)) =
      ((val3[0]*val7)+(val3[1]*val5)+(val3[2]*val6)+(val3[3]*val4));
  }
}

Multiply and add, in one loop, writing only the 4×4 answer. That is kernel fusion. The reshapes cost nothing: they only change how indexes are computed.

3. Run it, tested

All commands were run on 7 Nov 2026 on a MacBook (Apple M4, 16 GB), Python 3.14.5, in a private venv in a scratch folder. Device.DEFAULT reported METAL, so the GPU runs below are on the Mac GPU. Clang warning lines have been trimmed from the outputs.

Install, and one fix that was needed

git clone --depth 1 https://github.com/tinygrad/tinygrad.git
python3 -m venv venv
./venv/bin/pip install -e tinygrad
./venv/bin/python -c "from tinygrad import Device; print(Device.DEFAULT)"
METAL

The first run failed. On current master, Metal kernels are submitted by a small helper program that tinygrad compiles for the CPU, and by default it uses LLVM. This Mac has no Homebrew LLVM, so every realize crashed:

loading llvm failed: not found on system
AttributeError: failed to load library llvm: try setting LLVM_PATH?

The fix was to tell tinygrad to build that helper with Apple's own clang instead (the setting is in tinygrad/helpers.py). Installing LLVM with Homebrew would also work, but this needs nothing new:

export HCQ_RUNTIME_DEV=CPU:CLANG

The README's gradient example

# quick.py: the README's "Quick example comparing to PyTorch"
x = Tensor.eye(3).clone(); y = Tensor([[2.0,0,-2.0]])
z = y.matmul(x).sum(); z.backward()

$ ./venv/bin/python quick.py          # 7 Nov 2026, MacBook (Apple M4), METAL
[[2.0, 2.0, 2.0], [0.0, 0.0, 0.0], [-2.0, -2.0, -2.0]]
[[1.0, 1.0, 1.0]]

Same numbers PyTorch gives for the same code: dz/dx copies y down each column, and dz/dy is all ones because x is the identity.

The DEBUG=3 fusion demo (N = 1024)

$ DEBUG=3 ./venv/bin/python -c "from tinygrad import Tensor;
N = 1024; a, b = Tensor.empty(N, N), Tensor.empty(N, N);
(a.reshape(N, 1, N) * b.T.reshape(1, N, N)).sum(axis=2).realize()"

opened device METAL from pid:16667
scheduled     1 kernels in    25.83 ms | CACHE MISS 157a233c |     103 uops in cache
TC(0): [((1,), 1024)] [((2,), 1024)] [((0,), 1024)]
r_32_8_4_32_2_4_4_128     opts: (Opt(op=OptOps.TC, axis=0, arg=(-1, 0, 1)), Opt(op=OptOps.SPLIT, ...
opened device CPU from pid:16667
*** CPU        1 hcq_submit      arg  0 mem   0.01 GB tm    477.08us/     4.17ms
*** METAL      1 r_32_8_4_32_2_4_4_128      arg  3 mem   0.01 GB tm   3695.62us/     4.17ms (    581 GFLOPS   3|74     GB/s)

What it says:

Fused vs forced, counted

To see what laziness buys, this script runs the same maths twice at N = 256: once lazily, once forcing the big product to be stored first. GlobalCounters.kernel_count counts kernels.

# lazy.py (written for this page)
c = (a.reshape(N, 1, N) * b.T.reshape(1, N, N)).sum(axis=2)
c.realize()                                  # lazy: one plan, then run
p = (a.reshape(N, 1, N) * b.T.reshape(1, N, N)).realize()
d = p.sum(axis=2).realize()                  # forced: store p, then sum

$ ./venv/bin/python lazy.py                  # 7 Nov 2026, MacBook (Apple M4), METAL
device: METAL
before realize, c is just a plan: UOp Ops.REDUCE
lazy (fused):   kernels=1
forced eager:   kernels=2  intermediate=16,777,216 floats (67 MB)
same answer: True

$ DEV=CPU ./venv/bin/python lazy.py          # same, on the CPU backend
device: CPU
lazy (fused):   kernels=1
forced eager:   kernels=2  intermediate=16,777,216 floats (67 MB)

Train something tiny: XOR

# xor.py: 2 → 8 (tanh) → 2, Adam lr 0.05, 200 steps (written for this page)
l1, l2 = nn.Linear(2, 8), nn.Linear(8, 2)
loss = l2(l1(X).tanh()).sparse_categorical_crossentropy(Y).backward()

$ ./venv/bin/python xor.py                   # 7 Nov 2026, MacBook (Apple M4), METAL
step   0  loss 0.6987
step  50  loss 0.0020
step 100  loss 0.0006
step 150  loss 0.0004
step 200  loss 0.0003
predictions: [0, 1, 1, 0] time 5.2s

It learns XOR. 5.2 s is slow for 4 examples. Almost all of it is compiling kernels the first time and launching tiny GPU jobs. A GPU only pays off on big batches (see Reality check).

Train MNIST: examples/beautiful_mnist.py

A small convolutional network, 70 steps of batch 512, Adam, with TinyJit capturing the training step. It downloads MNIST (about 55 MB, cached in the scratch folder via XDG_CACHE_HOME, deleted afterwards).

$ XDG_CACHE_HOME=$PWD/cache ./venv/bin/python tinygrad/examples/beautiful_mnist.py
# 7 Nov 2026, MacBook (Apple M4), METAL
loss:   0.08 test_accuracy: 97.97%: 100%|███████| 70/70 [00:17<00:00,  4.01it/s]
real 28.49      # includes the download and first-time compiles

# second run, data already cached
loss:   0.10 test_accuracy: 98.24%: 100%|███████| 70/70 [00:24<00:00,  2.80it/s]
real 26.86

The README says "98% in ~5 seconds". We got 98% too, but the 70 steps took 17 s and 24 s on this laptop. The README doesn't say what hardware the 5 seconds was measured on. The two runs differ because batches are sampled at random.

4. Connect to Robo-Helper (Module 18)

Measured on 8 Nov 2026, with the first version of Robo-Helper's training data: 350 examples, which by accident included 3 of the 12 bake-off commands. On 9 Nov those were removed (343 examples). The small Module 18 brains' bake-off scores didn't change, and the tiny LM's held-out score went from 86% to 92%. Every brain here was compared on the same data, but if you re-run, expect the exact numbers to move by a few points.

Robo-Helper is standard-library Python: no numpy, no torch. That was on purpose, so you could see every number. tinygrad is the next rung up the ladder:

RungWhat computes gradientsWhat runs the maths
Lesson 4, neuron and Lesson 15, tiny LMYou, by hand (manual backprop, checked against numerical gradients)Python loops over lists
micrograd (page 1)Autograd over single numbersPython, one number at a time
tinygradAutograd over whole tensors, itself lazyGenerated kernels, fused, on CPU or GPU
PyTorchAutograd over tensorsHuge hand-written kernel libraries (optionally a compiler)

Tested: Robo-Helper's Lesson 15 brain, ported to tinygrad

We loaded the trained weights from lesson15_tiny_lm.train_tiny_lm() into tinygrad tensors. That is the averaging TinyLM that writes whole plans like "put red blue", with 10 output words and up to 5 words per plan (2,702 numbers). Then we rewrote TinyLM.generate as tensor maths. Reading the command becomes "bag of words × embedding table". The five greedy steps use one_hot and argmax, so the whole decode is one lazy graph. We checked it and timed it against the pure-Python model. The script is tg_tinylm.py, run from a scratch folder with the solutions folder on sys.path. (We ported only this plain TinyLM, not the attention or distilled brains.)

$ SOLUTIONS=…/18-robot-brain/project/solutions DEV=CPU   ./venv/bin/python -I tg_tinylm.py
$ SOLUTIONS=…/18-robot-brain/project/solutions DEV=METAL ./venv/bin/python -I tg_tinylm.py
# 8 Nov 2026, MacBook (Apple M4). Times in ms are medians. "ms/batch" = one call scoring the whole batch.

test_data(): tinygrad matches pure Python on 50/50 commands; exact-plan accuracy tinygrad 86%, pure Python 86%
4-bit copy: matches pure-Python 4-bit on 50/50; weights 2702, 10808 B at 32-bit vs 1351 B at 4-bit

           ---------- CPU backend ----------       ---------- METAL backend ---------
 batch   pure py   tinygrad   tinygrad+JIT      pure py   tinygrad   tinygrad+JIT
     1     0.383     67.318          4.864        0.370     50.540          1.842
     8     4.419     68.155          6.604        3.634     61.363          4.481
    16    10.797    110.808          7.104        9.795     65.238          6.094
    32    21.630     74.419          9.066       20.056     60.634          4.555
    64    39.679    103.012          7.947       37.400     59.585          5.162
   512   303.821     86.144          8.482      283.167     64.383          4.834
  4096  2172.644     95.716         28.492     1980.240     70.155          5.946

timed single command: 'chuck the red block in the trash' -> ['put', 'red']
one decision: pure Python p50 0.481 ms p95 0.749 ms | tinygrad+JIT p50 4.525 ms p95 6.658 ms   (CPU)
one decision: pure Python p50 0.463 ms p95 0.568 ms | tinygrad+JIT p50 1.975 ms p95 3.102 ms   (METAL)

What the numbers say, honestly:

When does this matter on a robot? Robo-Helper hears one command every few seconds. A half-millisecond pure-Python brain is already fast enough, and the tinygrad version would make each decision slower. Batching pays off when a robot must score many things at once: every candidate route, every object in a camera frame, a fleet of robots sharing one brain, or replaying thousands of logged commands to test a new model (as in the Lesson 19 bake-off). It is the Lesson 16 lesson again: measure first, per decision and per batch.

Next-step exercise

Take the forward pass of your Lesson 4 neuron and write it in tinygrad: weights as a Tensor, the same inputs, the same loss. Call loss.backward() and print w.grad.tolist(). Check that the numbers match your hand-derived gradients for the same weights. Then run it with DEBUG=2 and count the kernels for one forward and backward step. Can you guess, before running, which steps will fuse?

5. Why it matters for robotics

6. Reality check

7. Self-check

What does "lazy" mean in tinygrad, and what makes the work actually happen?

Operations only build a graph of UOps. The work happens when you call .realize(), or read a value with .item(), .tolist() or .numpy().

Why did the README matmul become 1 kernel instead of 2?

The multiply and the sum were in the same graph, and nothing needed the N×N×N products stored. So tinygrad wrote one kernel that multiplies and adds in a loop. Forcing .realize() on the product made it 2 kernels and a 67 MB temporary (at N = 256).

What is a backend, and roughly how much must one support?

The code that talks to one kind of chip: it renders kernels to that chip's language and runs them. The README says about 25 low-level ops.

Why did 4 XOR examples take 5.2 seconds on a GPU?

Most of the time is compiling each new kernel and launching tiny GPU jobs. The maths itself is trivial. GPUs pay off on big batches.

Where does tinygrad sit between your Lesson 15 language model and PyTorch?

Lesson 15 does backprop by hand in plain Python. micrograd automates backprop for single numbers. tinygrad automates it for tensors and compiles fused kernels. PyTorch does the same at full scale with large hand-tuned kernel libraries.

8. Further reading