Overfitting.

Overfitting occurs when a machine learning model learns the training data so thoroughly that it captures noise and specific patterns rather than the underlying relationships that would help it generalize. This results in excellent performance on training data but poor performance on validation or test sets. Overfitting is one of the most fundamental challenges in machine learning, affecting everything from simple linear models to complex neural networks and large language models.

The phenomenon happens when a model has too much capacity relative to the amount of training data, allowing it to memorize individual examples rather than learning generalizable patterns. Common signs include a large gap between training and validation accuracy, or performance that degrades when moving from training to test data. Regularization techniques like dropout, weight decay, and early stopping help prevent overfitting by constraining model complexity or limiting training duration.

In practice, overfitting represents a critical trade-off between model expressiveness and generalization ability. While some degree of fitting to training data is necessary for learning, excessive overfitting leads to models that fail in real-world deployment. Modern techniques like cross-validation, regularization, and careful dataset splitting help practitioners identify and mitigate overfitting, though it remains an ongoing concern especially as models grow larger and more complex.