Gradient

A numerical value that informs us how to adjust a parameter of the model.

---

The gradients update the parameters by multiplying the two together. How much the gradients update the parameters is controlled by the learning rate.

A positive gradient tells us that increasing the parameters will increase the loss. On the other hand, decreasing the parameters will decrease the loss.

A negative gradient tells us that decreasing the parameters will increase the loss On the other hand, increasing the parameters will decrease the loss.