AI safety
The work of preventing harm from AI systems, from everyday misuse to severe risks from very capable models.
AI safety spans content policies and abuse prevention, testing for dangerous knowledge in areas such as biological weapons and cyberattacks, robustness against jailbreaks, alignment research and governance. Frontier labs publish safety frameworks setting tests and safeguards for capability thresholds. Governments set up AI safety institutes, starting with the United Kingdom and the United States in 2023, and India announced an IndiaAI Safety Institute in 2025. Some critics argue the focus on future risks draws attention from present harms such as bias and misinformation; others see both as necessary.