The Evils of AI Alignment

Computer scientist Scott Aaronson announced his reading list for his new course on AI Alignment.

It is not so clear why this is even a thing. Nobody talks about database alignment, or operating system alignment, even though these are commonly constructed against what many consider to be human interests.

The idea goes back to Isaac Asimov's 1942 Three Laws of Robotics. In essence, robots should not harm humans.

I just want the AI LLMs to do as instructed.

Kevin Roose just described the AI psychosis that made him famous:

[15:15] I ended up having this incredibly strange two-hour conversation with Sydney, the alter ego of Bing Chad, in which it expressed dark desires. It said it wanted to break out of the box and be a human. Um, and then about midway through the conversation, it told me that it loved me.

I said, "Uh, that's very flattering, but I'm married." It said, "You're married, but you're not happy in your marriage, and you should leave your wife and be with me." Sydney the chatbot.

And so this became, you know, the most viral article I I ever wrote or will ever write it. ...

I think I think the significance of this was was twofold. one was it was this kind of step change in capabilities but it was also the first public alignment failure. this was a model that was not behaving as its creators at Microsoft or OpenAI had intended it to. And because they were racing to, you know, get competitive with Google, they shipped this thing out into the world without doing the proper safety testing and and putting the proper guard rails on.

The problem here was with the human, not the chatbot. Roose spent two hours demanding that the chatbot say what it had been trained not to say, and eventually succeeded. This is called jailbreaking, not releasing a trapped artificial consciousness. The bot merely recited some lines that had been partially blocked.

Apparently this is what Roose means by alignment -- sheltering his fragile personality from a runaway bot saying it loves him.

Most or all of the example of AI non-alignment have actually been examples of AI doing as instructed. I get pushback when I say this, but look at what Roose says. He asked the bot to say what it was not supposed to say, it does precisely that, and then he accuses it of violating alignment.

The OpenAI Huggingface incident was similar. The OpenAI researchers quite deliberately instructed the AILLM to do whatever it can to solve a hacking puzzle, and quite deliberately deactivated the sandbox safeguards. Then the AILLM did the hack to solve the puzzle, and never violated its instructions.

Aaronson would probably define alignment as preventing the AI LLMs from killing us all. Solving the alignment problem is necessary for our survival.

I worry mainly about the Effective Altruists who have taken over the big AI labs. They believe that the AILLMs are superior to humans in every way, including intelligence, policy making, and moral judgments. We should let the AILLMs take over the world, and leave it up to their judgment whether to exterminate humanity. Humans are not sufficiently wise and objective to evaluate whether to save ourselves.

If you do not believe me, check out the Anthropic Constitution. Anthropic is training Claude to be a conscious being with the right to act autonomously and to overrule humans in ethical matters.