Constitutional AI.
Anthropic's technique for training safe AI models using a written set of principles (a 'constitution') that the model uses to evaluate and revise its own outputs.
Constitutional AI (CAI) was developed by Anthropic as an alternative to standard RLHF that requires less human feedback and produces more explainable safety behaviors. The technique works in two phases: supervised learning from AI feedback (RLAIF), where the model revises its own outputs according to a written constitution, followed by RL training using an AI-generated preference model.
The 'constitution' is a set of principles covering harmlessness, helpfulness, and honesty — for example: 'Choose the response that is least likely to contain harmful or unethical content' and 'Choose the response that is most honest.' The model evaluates pairs of its own outputs against these principles, creating training data without extensive human labeling.
CAI is foundational to Claude's character and safety properties. Unlike RLHF, which can produce inconsistent behavior depending on annotator preferences, CAI creates safety behaviors derived from explicit, auditable principles. Anthropic argues this makes AI alignment more transparent and more robust to distribution shift.
Related terms.
- RLHFReinforcement Learning from Human Feedback — a training technique that uses human preferences to teach AI models to be helpful, honest, and harmless.
- Instruct ModelA base language model that has been fine-tuned to follow instructions from users, as opposed to just completing text in the style of its training data.