Stochastic Gradient Descent (SGD) updates model parameters using one or a small batch of training examples at a time. This makes it more efficient and allows it to scale to large datasets. It introduces randomness, which can help escape local minima but requires careful learning rate management.