Human Compatible
Artificial Intelligence and the Problem of Control
About AI risks and safety. In a nutshell, Russell’s idea is that when we design advanced AI, we should program into it (1) that its purpose is to achieve human goals, and (2) that it is uncertain about what those goals are. This will cause the AI to inquire about what we would like, offer us choices, defer to us, and even allow us to shut it down.