Test-Time Compute.

Test-time compute refers to the practice of allocating more computational resources during inference to enhance the quality of model outputs, rather than relying solely on the model's initial response. This approach represents a fundamental shift from traditional inference patterns where models generate responses in a single forward pass. By investing additional computation at test time, models can explore multiple reasoning paths, verify their work, or generate and evaluate multiple candidate responses before selecting the best one.

The core mechanism involves techniques such as chain-of-thought reasoning where models explicitly work through problems step-by-step, sampling multiple responses and selecting the highest-quality output, or implementing search algorithms that explore different solution paths. Unlike training-time compute which improves the base model's capabilities, test-time compute allows the same model to achieve better performance on individual queries by thinking longer or more carefully. This can involve iterative refinement, self-verification, or even having the model debate with itself to reach better conclusions.

Test-time compute creates a trade-off between response quality and latency, as better outputs come at the cost of increased inference time and computational expense. This approach has become particularly important for reasoning-heavy tasks like mathematics, coding, and complex analysis where the additional thinking time can dramatically improve accuracy. However, it requires careful consideration of cost-benefit ratios, as the computational overhead can be substantial, and not all tasks benefit equally from extended reasoning time.