What is gradient descent?

What is gradient descent?

• A. An optimization method that minimizes loss by updating parameters in the direction of the negative gradient.
• B. A technique to increase model complexity by adding more parameters.
• C. A method to randomly initialize weights before training.
• D. A metric to evaluate model accuracy.

Answer: (A)

Gradient descent is an optimization method used to minimize a loss function by updating model parameters in the direction opposite to the gradient of the loss. At each step, you compute how the loss changes with small moves in parameter space and adjust parameters by a learning rate times the negative gradient. Repeating this reduces the loss and moves you toward a minimum, with variants like batch, stochastic, or mini-batch depending on how the gradient is computed. This approach is central to training many models, including neural networks, where backpropagation provides the gradient across layers. It isn’t about making the model more complex, it isn’t about randomly initializing weights, and it isn’t a metric of accuracy.

Gradient descent is an optimization method used to minimize a loss function by updating model parameters in the direction opposite to the gradient of the loss. At each step, you compute how the loss changes with small moves in parameter space and adjust parameters by a learning rate times the negative gradient. Repeating this reduces the loss and moves you toward a minimum, with variants like batch, stochastic, or mini-batch depending on how the gradient is computed. This approach is central to training many models, including neural networks, where backpropagation provides the gradient across layers. It isn’t about making the model more complex, it isn’t about randomly initializing weights, and it isn’t a metric of accuracy.