Self-Consistency.

Self-consistency is a decoding strategy that enhances the reliability of large language models by generating multiple reasoning paths for the same problem and taking a majority vote among the outputs. Rather than relying on a single chain-of-thought response, this approach leverages the principle that correct reasoning processes are more likely to converge on the same answer. The technique addresses the inherent randomness in language model sampling by treating reasoning as an ensemble problem, significantly improving performance on mathematical, logical, and commonsense reasoning tasks.

The process works by prompting a model multiple times with the same chain-of-thought question, using different sampling parameters or random seeds to generate diverse reasoning paths. Each response follows a step-by-step reasoning process, but the specific intermediate steps and explanations may vary due to the stochastic nature of text generation. After collecting multiple responses, the system identifies the most frequently occurring final answer across all attempts. This majority voting mechanism filters out inconsistent or incorrect reasoning paths that might occur in any single generation, capitalizing on the model's tendency to arrive at correct conclusions through multiple valid reasoning routes.

Self-consistency typically requires 5-40 samples to achieve optimal results, creating a trade-off between accuracy gains and computational cost since inference time scales linearly with the number of samples. The technique is most effective on problems with discrete, verifiable answers rather than open-ended creative tasks where multiple valid responses exist. While self-consistency can dramatically improve performance on reasoning benchmarks, it may not help with tasks where the model consistently lacks the underlying knowledge or reasoning capability, as it cannot correct systematic errors that appear across all generated paths.