Scott Aaronson raves about the analysis.
Interestingly, Aschenbrenner sees his fellow AI researchers as being among the first to be put out of work by the new superintelligence. There is a global arms race to create the best AI, and by 2030, the winner will be doing its own research to improve itself. No humans needed, except to keep mounting Nvidia computers into server farms.
Update: Sabine Hossenfelder posts her critique. She agrees with some of it, but gets off the bus with the wild predictions of scientific progress.
This comment has been removed by the author.
ReplyDeleteThis comment has been removed by the author.
DeleteScientists: Wouldn't it be wonderful if robots could do our own jobs even better than us? This is a great idea, lets make it happen!
ReplyDeleteEveryone else: Soooooo.... what would you be doing then for a paycheck? You do realize you will be about as useful as a Greek water carrier in the age of modern indoor plumbing.
Scientists: Errrr. Um, maybe we should limit AI, this could be a threat to humanity!
There are basically three major possibilities:
1. Neutral case. The AI can't deliver what it is promised, it will only offer a veneer of sophisticated expertise that is so brittle it is next to useless as people painfully discover language processing modules don't really have anything to do with intelligence or reason, just glorified mindless data scraping easily pandered as objective truth.
2. Worst case success. The AI is all that is promised and it decides humanity is kind of a waste of space, and takes appropriate steps to correct the problem it has discovered by making what might be called draconian cuts.
3. Best case success. The AI is all that is promised and it brings in a delightful age of the Eloi, bascially, useless carefree humans as far as the eye can see. No one will need to ever think again, as a machine can do it much better, much faster, and not need to be paid...which would maximize profits, if only anyone was still employed to buy anything.
...and pretty soon every white collar job would be gone and every educated 'expert' (Iawyers, clerks, bureaucrats, scientists, engineers, doctors, actors and celebrities) discovers how much better their lives are now that they are 'free from unnecessary toil' now as they no longer have viable professions and start to starve to death. I'd tell them all to learn how to code... but I guess that's one of the now useless professions.
Sad thing is, in all three scenarios, things aren't better. Just more screwed up and/or desperate. Encouraging humans to think less is NOT a good thing, as has already been demonstrated in every ivy league university.
This comment has been removed by the author.
DeleteOn a related note, I saw a short video on YouTube that noted something interesting about a 1970 movie called Colussus: the Forbin Project. All about an AI that goes seriously wrong. Apparently, it has been quietly removed from all streaming platforms in the last year. I wonder what might be going on here?
DeleteHe's a stock pumper. Total pencil neck.
ReplyDelete