A Support Vector Machine is a powerful classifier (or regressor) that finds an optimal hyperplane or set of hyperplanes in a high-dimensional space. It aims to maximize the margin between classes, improving generalization. Kernel functions allow SVMs to handle nonlinear decision boundaries.