The False Promise of Chomskyism

Noam Chomsky is maybe the world's leading intellectual, with huge academic and political followings:

Avram Noam Chomsky[a] (born December 7, 1928) is an American public intellectual: a linguist, philosopher, cognitive scientist, historian,[b] social critic, and political activist. Sometimes called "the father of modern linguistics",[c] Chomsky is also a major figure in analytic philosophy and one of the founders of the field of cognitive science. He is a Laureate Professor of Linguistics at the University of Arizona and an Institute Professor Emeritus at the Massachusetts Institute of Technology (MIT), and is the author of more than 150 books on topics such as linguistics, war, and politics. Ideologically, he aligns with anarcho-syndicalism and libertarian socialism.

He just weighed in the new large language model (LLM) fad in in the NY Times. He has been arguing all his life that language is innately human, and will never be mastered by computers. So has ChatGPT proved him wrong? Of course not. He doubles down on his theories.

Scott Aaronson now works for OpenAI, and has leapt to its defense:

to my mind, Chomsky was nasty and vicious utterly without justification, in the pages of the New York Times no less, in attacking one of the major engineering accomplishments of our time ...

I meant is that there exists a large “old guard” in AI and NLP research that ...

(1) has been bizarrely insulting, dismissive, and hostile toward the mind-boggling engineering achievement represented by large language models,
(2) failed for 60 years to produce any comparable artifact,
(3) idolizes Chomsky and was probably influenced by him more than any other person, and
(4) displays the trademark Chomskyan tactic of “retreat from the empirical,” where even when something succeeds, the success can be breezily dismissed as an unimportant “epiphenomenon” (but the failures are not epiphenomena). ...

On deeper reflection, I probably don’t need to spend emotional energy refuting people like Chomsky, who believe that Large Language Models are just a laughable fad rather than a step-change in how humans can and will use technology, any more than I would’ve needed to spend it refuting those who said the same about the World Wide Web in 1993. Yes, they’re wrong, and yes, despite being wrong they’re self-certain, hostile, and smug, and yes I can see this, and yes it angers me. But the world is going to make the argument for me.

Aaronson views ChatGPT as being like the nerds who get bullied in high school.

Chomsky makes a point out of the ambiguity of t his sentence:

John is too stubborn to talk to.

I tried this on Bing Chat and on some children, to see how they interpret it. I asked them what it means, along with questions like: Is this saying something about John's ability or willingness to talk?

The answers I got from humans were not much different than from Bing Chat. Where there are differences, I cannot be sure who is right. My conclusion is that the sentence is ambiguous and should be corrected by copy editor. It is not an example of good grammar.

In my opinion, the LLMs have mastered English grammar in a way that Chomsky and other naysayers have long said was impossible. They have also captured a vast amount of knowledge and put it in a usable form.

Does Bing Chat actually understand what it is saying? Sabine Hossenfelder says that is like the Feynman remark about nobody understanding quantum mechanics. Sure, lots of people understand it well enough to apply to textbook examples. If you want a deeper metaphysical understanding rivaling your understanding of everyday macroscopic objects, then it is debatable whether such an understanding exists.

In case you are still wondering what the excitement is about, just try it. Bing chat is as much better than Google search, as Google search was better than Alta Vista.