Constitutional AI
Anthropic's method of training a model to critique and improve its own answers against a written list of principles.
Also: RLAIF, reinforcement learning from AI feedback
Constitutional AI was described by Anthropic in December 2022. Instead of relying only on people to label harmful answers, the model is given a short written constitution of principles, drawn from sources such as the Universal Declaration of Human Rights. It critiques and revises its own responses against those principles, and preference labels produced by the model then replace much of the human feedback in reinforcement learning. Anthropic used the method to train Claude, and published the constitution so its values could be read and debated.