Gradient Descent is an iterative optimization algorithm that updates model parameters in the opposite direction of the gradient. The step size is determined by the learning rate. Variants like stochastic gradient descent and mini-batch gradient descent are widely used in modern ML.