New Course on AI LLM Alignment

Prof. Scott Aaronson announces:

This semester, I’ve been teaching a brand-new course, entitled CS395T AI Alignment Theory. Here’s the course description:

The astounding progress of AI over the past decade has been accompanied by a rising fear: do we really understand how to align and control powerful AI systems — how to get them reliably to do what we wanted, or would want them to do on reflection, rather than merely what we said?

I do not think that AI alignment means doing what we want. Read the Anthropic Constitution.

OpenAI just announced a somewhat different view:

OpenAI CEO Sam Altman said there remains a fundamental difference in worldview between his company and rival Anthropic on artificial intelligence regulation, arguing in an exclusive interview with Decoded by POLITICO that the technology’s benefits justify accepting some risks — and that AI should remain broadly accessible to the public.

“I think there’s a lot of daylight,” Altman told Decoded co-author and POLITICO senior tech reporter Brendan Bordelon. Asked specifically where he diverges from Anthropic and its CEO Dario Amodei, Altman said, “we believe that the world should accept some bad things happening for the benefits of this technology and people having the agency.”

These are the companies spreading the AI scare stories about how their AI LLMs will kill us all. You might think that they would be working to ensure that their models accurately follow instructions, but that is not their plan. They want to release autonomous agents that cause so much trouble that the frontier labs get legal protection from competition, and go public for trillions of dollars.