This descent on a simple parabola ends at w in the HUNDREDS — each step overshoots the minimum by more than the last, the textbook divergence personality (in real training: loss to inf/NaN in a few steps). Nothing is wrong with the gradient. Fix the one knob.
The break is on line 7 — but read the whole snippet first.