skip to content
promptdojo
_
[ save your spot ]
[ follow on x ]
Gradient descent by hand — step 2 of 7
ch 43 · training loops, backprop, optimizers, and schedulers
2/7
promptdojo
_
›
phase 06 · ml systems
›
ch 43 · training loops, backprop, optimizers, and schedulers
lesson 2 of 5 · gradient descent by hand
step 2 of 7 in this lesson
Three steps into training, the loss prints NaN. What's the FIRST move?
1
Add gradient clipping
2
Halve the learning rate and re-run — cheapest experiment you own
3
Switch from Adam to a different optimizer
4
Train on more data
check
Show hint 1
ch 43 · training loops, backprop, optimizers, and schedulers
2/7
promptdojo
_
›
phase 06 · ml systems
›
ch 43 · training loops, backprop, optimizers, and schedulers
lesson 2 of 5 · gradient descent by hand
step 2 of 7 in this lesson
Three steps into training, the loss prints NaN. What's the FIRST move?
1
Add gradient clipping
2
Halve the learning rate and re-run — cheapest experiment you own
3
Switch from Adam to a different optimizer
4
Train on more data
check
Show hint 1
park a thought