Backpropagation.

Backpropagation is the fundamental algorithm that enables neural networks to learn by calculating how much each parameter contributed to the overall error. It works by computing gradients of the loss function with respect to every weight and bias in the network, providing the direction and magnitude needed to update parameters during training. This process is essential for all modern deep learning, from simple feedforward networks to complex transformer architectures like those powering large language models.

The algorithm operates by applying the mathematical chain rule in reverse, starting from the output layer and propagating error signals backward through each layer to the input. At each layer, it calculates partial derivatives that indicate how sensitive the loss function is to changes in that layer's parameters. The 'backward pass' complements the 'forward pass' where data flows through the network to produce predictions, creating a complete training cycle that iteratively improves the model's performance.

While backpropagation is computationally intensive and requires storing intermediate values during the forward pass, it remains the most practical method for training neural networks at scale. Modern implementations use automatic differentiation frameworks that handle the complex gradient calculations automatically, but the underlying principles remain unchanged. Common misconceptions include thinking backpropagation is the learning algorithm itself, when it's actually just the gradient computation method used by optimization algorithms like SGD or Adam to update parameters.