Optimizers and learning-rate schedulers (step 4/7) · training loops, backprop, optimizers, and schedulers

promptdojo_

Step decay raises the decay factor to the power of the epoch number, so the lr shrinks geometrically. Fill the blank with the operator that raises decay to the epoch.

Operator:

Step decay raises the decay factor to the power of the epoch number, so the lr shrinks geometrically. Fill the blank with the operator that raises decay to the epoch.

Operator:

Optimizers and learning-rate schedulers — step 4 of 7