Vanishing Gradients occur when gradients become extremely small, halting updates to earlier network layers. This was a significant problem in training deep RNNs until LSTM/GRU were introduced. Proper weight initialization and activation functions like ReLU also help mitigate this issue.