Alignment
Making AI systems pursue the goals and values their designers and users intend, and avoid harmful behaviour.
Also: AI alignment
Alignment research asks how to make sure a model does what people actually want, including in situations its trainers did not foresee. In practice it includes instruction tuning, RLHF, constitutional methods and written behaviour specifications. Longer-term concerns include models that pursue goals deceptively or resist correction, which some researchers study with deliberately constructed test scenarios. Views differ widely on how hard the problem is and how urgent, from those who see it as ordinary engineering to those who see it as a central risk.