LLM + RoboticsModule 12 · Video course

AI training and inference, in nine videos: watch guides for Jia-Bin Huang's playlist

Clear, animated explainers on how foundation models are trained and served: memory and parallelism, fine-tuning, RL for reasoning, distillation, optimisers, speculative decoding and serving. Each video gets a watch guide that maps its chapters into the rest of this course.

Source: Jia-Bin Huang, “AI Training and Inference” playlist on YouTube, described as covering fundamental training and inference techniques for foundation models (LLMs, VLMs). Video details saved in sources/12-jiabin-huang-playlist.json. The guides are written from each video's chapter list and references plus the original papers. Videos © Jia-Bin Huang.

Start here (no maths needed)

Nine short films that explain how giant AI models are trained, and how they answer millions of people quickly.

Think of it like this

Building a huge AI is like building a skyscraper. You need lots of workers sharing the job (many computers working together), smart tools (the "optimiser" that decides how to improve each step), ways to adjust the building later without rebuilding it (fine-tuning), and a good front desk to serve visitors fast (serving). Each video explains one of these jobs with animations.

How it works

  1. Pick a video. They're 7 to 42 minutes long.
  2. Open its watch guide here first, and read the Start-here box.
  3. Watch with the guide open. The chapter table tells you what to look for.
  4. Answer the self-check questions afterwards.

Why robots care

Robot brains are trained and run with exactly these tools. Fine-tuning (video 3) is how a general robot model gets adapted to your robot. Shrinking big models (video 5) and fast serving (videos 4 and 9) decide whether the robot reacts in time.

Try it

Start with the shortest video, number 7 on optimisers (about 7 minutes). Then tick it off below.

Words to know

Training
Teaching the model from data.
Inference / serving
Using the trained model to answer people.
GPU
The chip that does the maths.
Optimiser
The rule for how to nudge the model to get better each step.

Going deeper: about 3 hours 50 minutes of video in total. The guides link into modules 04–11 wherever the course already goes deeper.

Progress

Watched
0 / 9
Next up
Total runtime
3h 50m

The nine videos

Listed in a suggested learning order: optimisers → memory and parallelism → adaptation → RL and distillation → inference. The playlist itself lists the newest first.

07

7:19 · Gradient descent, momentum, RMSProp, Adam, AdamW.

08

17:52 · Orthogonalised updates via Newton–Schulz, and the attention-logit fix used at scale.

01

23:05 · Number formats, mixed precision, data parallelism, ZeRO stages 1–3, FSDP.

02

40:27 · Data, tensor, context, pipeline and expert parallelism; Llama 3's 16,384-GPU layout.

03

21:26 · LoRA, initialisation, LoRA+, QLoRA, VeRA, DoRA.

06

23:31 · Policy gradients, baselines, advantages, actor-critic, TRPO, PPO, GRPO, Dr. GRPO.

05

30:34 · Soft targets, exposure bias, KL direction, self-distillation.

04

24:34 · Draft and verify, rejection sampling, draft trees, Medusa, MTP, EAGLE 1–3, DFlash.

09

41:42 · Scheduler, KV cache, PagedAttention, RadixAttention, structured outputs, CUDA graphs, DeepSeek-V3 on 8 GPUs.

How it fits the course

VideosPairs with
01, 0208 Stage 06 distributed training (ZeRO calculator), 06 Day 24 GPipe
0308 Stage 05, 09 quantisation
05, 0607 post-training, 04 distillation, 11 Raschka & Lambert
07, 0808 Stage 01, 05 (Kimi K3 uses Per-Head Muon)
04, 09All of Module 09, inference engineering