- Learning happens in the context window during inference, not through weight updates or fine-tuning.
- Performance typically improves from zero-shot to few-shot, with larger models showing stronger zero-shot capabilities.
- Researchers debate whether ICL works through implicit gradient descent in forward pass or sophisticated pattern matching.
- ICL is limited by context window size and cannot provide persistent learning across different interactions.
- Performance depends on example quality and order, with misleading examples degrading results.
The Surprising Ability
When GPTGPTGenerative Pre-trained Transformer — the model architecture and family name behind OpenAI's most famous models, from GPT-2 to GPT-5.Learn more →-3 was released in 2020, one of its most striking properties was few-shotFew-ShotA prompting technique where you include a small number of input-output examples in the prompt to demonstrate the desired task format and behavior to the model.Learn more → learning: show it three examples of translating English to French, and it would correctly translate the fourth — without any training, weight updates, or fine-tuningFine-tuningThe process of further training a pre-trained model on a smaller, task-specific dataset to specialize its behavior for a particular use case.Learn more →. The model was never explicitly trained to do translation; it learned from the pattern in context.
This is called In-Context LearningIn-Context LearningThe ability of an LLM to learn from examples provided directly in the prompt, without any weight updates or training — just from seeing patterns in context.Learn more → (ICL) because the 'learning' happens in the context windowContext WindowThe maximum amount of text (measured in tokens) that a model can read and reason over in a single interaction, including your prompt, any documents, and previous conversation history.Learn more →, not in the model weights. It's a form of meta-learning — the pre-training process teaches the model to learn from examples at inferenceInferenceThe process of running a trained AI model to generate outputs — what happens when you send a prompt and receive a response.Learn more → time, not just to produce text. GPT-3's ICL capability was the primary reason it became an immediate landmark. Zero-ShotZero-ShotThe ability of a model to perform a task correctly from just a description, with no examples — demonstrating that it has generalized the concept from training.Learn more → and Few-Shot are the two main forms of ICL, with Few-Shot generally outperforming Zero-Shot on harder tasks.
Zero-Shot, One-Shot, and Few-Shot
ICL comes in three strengths: zero-shot (task description only, no examples), one-shotOne-ShotA prompting technique where exactly one example is provided to demonstrate the desired task format before presenting the actual query.Learn more → (one example), and few-shot (2–10 examples). Performance typically improves from zero to few-shot, though the gap depends on task difficulty and model size. Larger, more capable models show stronger zero-shot performance because they've seen more task patterns during training.
For new, unusual tasks that the model may not have encountered during training, few-shot prompting provides essential disambiguation. The examples show the model the exact format, style, and level of detail you need, compensating for the model's prior toward its training distribution. For common tasks (summarization, translation, Q&A), zero-shot usually suffices.
Why ICL Works: Competing Theories
The mechanism of ICL remains an active research debate. One view: models perform implicit gradient descentGradient DescentAn optimization algorithm that iteratively adjusts model parameters by moving in the direction opposite to the gradient of the loss function.Learn more → in their forward pass — the transformerTransformerThe neural network architecture that underpins virtually all modern LLMs, introduced in 2017, built around self-attention mechanisms that process entire sequences in parallel.Learn more →'s attention mechanismAttention MechanismA technique that allows each token in a sequence to 'pay attention' to all other tokens, enabling the model to understand context and relationships across long distances.Learn more → implements a learning algorithm that adjusts internal representations based on examples. Evidence: ICL performance scales similarly to learning curves from explicit optimization.
An alternative view: ICL is sophisticated pattern matching. The model recognizes that the current context resembles training examples where it learned task X, and activates learned task-X behavior. Evidence: ICL is sensitive to example order and the specific wording of demonstrations in ways that differ from explicit optimization. Both mechanisms likely contribute.
Limitations of In-Context Learning
ICL is constrained by the context window: you can only fit so many examples. For tasks requiring extensive demonstration libraries, fine-tuning is more practical. ICL is also sensitive to example quality and order — including misleading examples degrades performance, and earlier examples in the context have less influence than recent ones for some tasks.
ICL doesn't update model weights, so nothing 'learned' in one context carries to the next. For persistent behavioral changes across all interactions (consistent output format, domain-specific terminology, behavioral constraints), fine-tuning is necessary. ICL is best thought of as dynamic configuration at inference time, not persistent learning.