What chain-of-thought is
Chain-of-thought prompting encourages the model to write out its intermediate reasoning steps before giving a final answer. This dramatically improves accuracy on tasks that require arithmetic, logical deduction, or multi-step planningPlanningAn agent's decomposition of a goal into an ordered set of steps or subgoals before or while acting to achieve complex objectives systematically.Learn more → — tasks where jumping straight to an answer leads to errors.
The insight is that transformers process tokens sequentially. When forced to write reasoning steps, each step is visible to the model when it generates the next one, which allows it to self-correct and carry forward intermediate results accurately.
The simplest approach: 'think step by step'
The easiest way to trigger CoT is to append 'Think step by step' or 'Let's think through this carefully' to your promptPromptThe input text sent to a language model — the question, instruction, or context that triggers a response.Learn more →. This alone improves performance on arithmetic and logic tasks by 20–50% in most evaluations.
For even better results on specific problem types, use a more targeted instruction: 'First, identify all the relevant information. Then, determine what the question is asking. Finally, work through the calculation step by step and double-check your arithmetic.'
Few-shot CoT with examples
Providing examples of good reasoning chains (few-shotFew-ShotA prompting technique where you include a small number of input-output examples in the prompt to demonstrate the desired task format and behavior to the model.Learn more → CoT) further improves performance. Structure each example as: Question Reasoning (step by step) Answer. Use 2–4 examples that cover different cases you expect in production.
The examples teach the model the reasoning format you want — not just that it should reason, but how. This is particularly valuable for domain-specific tasks where the reasoning structure is non-obvious.
When CoT helps and when it doesn't
CoT significantly helps: multi-step arithmetic, logic puzzles, planning tasks, debugging complex code, legal or medical reasoning chains, and any task where the model must track multiple variables or constraints.
CoT does not help and may hurt: simple classification, factual recall, or sentiment analysis. For these tasks, asking the model to reason first can introduce unnecessary hedging and slow down responses. Use CoT selectively.
Zero-shot CoT for production
In production, you usually want the reasoning to happen silently and only the final answer to be returned. Use a two-call approach: first call with 'Think step by step' and extract the reasoning; second call with 'Given this reasoning: [reasoning], what is the final answer?' to extract a clean response.
Alternatively, use structured outputs: ask the model to return `{"reasoning": "...", "answer": "..."}` as JSON, then display only the answer in your UI while logging the reasoning for debugging.