What are the top challenges in AI alignment?

EA

Emmanuel Ameisen

Research · Anthropic

Understanding Model Internals

Emmanuel Ameisen says a key alignment challenge is understanding model internals while preventing evasion, because as models get better their world models are clearer but they are more able to evade basic security measures.

William MacAskill

Senior Research Fellow · Forethought

Value Drift

William MacAskill points to value drift, saying that even if AIs are given specific values today, their values should be expected to drift substantially as they evolve and interact with the world.

The answer starts at 3:46.

JP

Jakub Pachocki

Chief Scientist · Openai

Generalization of Values

Jakub Pachocki says the fundamental challenge of AI alignment is generalization of values, adding that value alignment is what counts most.

The answer starts at 1:51.

Preventing Objective Drift

Ilya Sutskever says a challenge is preventing an AI's objectives from drifting as it continuously learns and changes; he argues this is much harder than aligning a frozen chatbot.

Chris Painter

President · METR

Higher Stakes

Chris Painter says the challenge is higher stakes: as models become more capable, even rarer alignment failures can be a bigger deal because delegated tasks are larger in scope.