AI training and inference, in nine videos: watch guides for Jia-Bin Huang's playlist
Clear, animated explainers on how foundation models are trained and served: memory and parallelism, fine-tuning, RL for reasoning, distillation, optimisers, speculative decoding and serving. Each video gets a watch guide that maps its chapters into the rest of this course.
Start here (no maths needed)
Nine short films that explain how giant AI models are trained, and how they answer millions of people quickly.
Think of it like this
Building a huge AI is like building a skyscraper. You need lots of workers sharing the job (many computers working together), smart tools (the "optimiser" that decides how to improve each step), ways to adjust the building later without rebuilding it (fine-tuning), and a good front desk to serve visitors fast (serving). Each video explains one of these jobs with animations.
How it works
- Pick a video. They're 7 to 42 minutes long.
- Open its watch guide here first, and read the Start-here box.
- Watch with the guide open. The chapter table tells you what to look for.
- Answer the self-check questions afterwards.
Why robots care
Robot brains are trained and run with exactly these tools. Fine-tuning (video 3) is how a general robot model gets adapted to your robot. Shrinking big models (video 5) and fast serving (videos 4 and 9) decide whether the robot reacts in time.
Try it
Start with the shortest video, number 7 on optimisers (about 7 minutes). Then tick it off below.
Words to know
- Training
- Teaching the model from data.
- Inference / serving
- Using the trained model to answer people.
- GPU
- The chip that does the maths.
- Optimiser
- The rule for how to nudge the model to get better each step.
Going deeper: about 3 hours 50 minutes of video in total. The guides link into modules 04–11 wherever the course already goes deeper.
Progress
The nine videos
7:19 · Gradient descent, momentum, RMSProp, Adam, AdamW.
17:52 · Orthogonalised updates via Newton–Schulz, and the attention-logit fix used at scale.
23:05 · Number formats, mixed precision, data parallelism, ZeRO stages 1–3, FSDP.
40:27 · Data, tensor, context, pipeline and expert parallelism; Llama 3's 16,384-GPU layout.
21:26 · LoRA, initialisation, LoRA+, QLoRA, VeRA, DoRA.
23:31 · Policy gradients, baselines, advantages, actor-critic, TRPO, PPO, GRPO, Dr. GRPO.
30:34 · Soft targets, exposure bias, KL direction, self-distillation.
24:34 · Draft and verify, rejection sampling, draft trees, Medusa, MTP, EAGLE 1–3, DFlash.
41:42 · Scheduler, KV cache, PagedAttention, RadixAttention, structured outputs, CUDA graphs, DeepSeek-V3 on 8 GPUs.
How it fits the course
| Videos | Pairs with |
|---|---|
| 01, 02 | 08 Stage 06 distributed training (ZeRO calculator), 06 Day 24 GPipe |
| 03 | 08 Stage 05, 09 quantisation |
| 05, 06 | 07 post-training, 04 distillation, 11 Raschka & Lambert |
| 07, 08 | 08 Stage 01, 05 (Kimi K3 uses Per-Head Muon) |
| 04, 09 | All of Module 09, inference engineering |