Optimizers and learning-rate schedulers (step 7/7) · training loops, backprop, optimizers, and schedulers

Checkpoint

One last thing before we move on. Same surface as a write step — but the lesson doesn't complete until this passes.

Implement lr_at(t) — linear warmup from 0 to PEAK over the first WARMUP steps, then cosine decay PEAK * 0.5 * (1 + cos(pi * progress)) over the rest. Print f"step {t:>4}: lr={lr_at(t):.4f}" for t in (0, 50, 100, 550, 1000), then two property checks: "peak at warmup end:" (lr_at(100) == PEAK) and "near zero at the end:" (lr_at(1000) < 1e-9). The lr as a plan, not a constant.

Checkpoint

One last thing before we move on. Same surface as a write step — but the lesson doesn't complete until this passes.

full-screen editor opens — close anytime to keep reading.

Optimizers and learning-rate schedulers — step 7 of 7