But not everyone recognized it. Prominent MIT professors wrote a 1969 book, Perceptrons: An Introduction to Computational Geometry, trashing the concept.
Reports by the New York Times and statements by Rosenblatt claimed that neural nets would soon be able to see images, beat humans at chess, and reproduce. ...The book claimed to prove that perceptrons were hopelessly limited, as they could not even approximate a simple xor function.This book is the center of a long-standing controversy in the study of artificial intelligence. It is claimed that pessimistic predictions made by the authors were responsible for a change in the direction of research in AI, concentrating efforts on so-called "symbolic" systems, a line of research that petered out and contributed to the so-called AI winter of the 1980s, when AI's promise was not realized.
The authors doubled down in 1988 updated edition, saying "little of significance [has] changed since 1969". At the time there was a revival of interest in neural nets, but the authors said they would fail to scale up.
We now know that those neural nets of the 1940s, 50s, and 80s do scale up, and perform spectacularly well. The bold predictions of the perceptron inventor must have been seen as wildly optimistic at the time, but they turned out to be correct.
I post this as an example of how smart people can be wrong about what is possible, and how brilliant research could be sidetracked.
Rosenblatt made a hardware computer perceptron in 1960, and that is now in the Smithsonian Museum. What no one knew until about 2012 was that video game chips would be the key to scaling up neural nets. [date corrected]
Here is a recent video that tries to relate these old ideas about neural nets to current research.
This comment has been removed by the author.
ReplyDeleteThis comment has been removed by the author.
DeleteThis comment has been removed by the author.
DeleteThis comment has been removed by the author.
DeleteThis comment has been removed by the author.
DeleteThis comment has been removed by the author.
DeleteRobert, you mentioned folks didn't know something until 1912.... did you mean to say 2012? because microchips didn't exist in 1912.
ReplyDeleteRoger,
ReplyDeleteNeural nets being used now are digitally simulated, and have tremendous amounts of RAM memory at their disposal they simply didn't have until very recently. This is the reason they were deemed impractical in the 60s, because they had to be hand wired adn were quite small, and later impractical in the 80s when computer memory was simply not available at the scale required to simulate very many neural layers.
Presently, RAM memory prices are well over 500% higher due to this present demand for memory manufacturing to support the AI data centers. Neural nets also have a lot of problems you are either unaware of, or are simply glossing over. Most of what is going on is the LLM models, which are just complicated word predictors, they have no self awareness of what they are even doing with words, and generate results that are frequently just slop and plain wrong.
Sam Altman, the primary person behind this AI mania is a psychotic liar. He has no idea how he is going to justify hundreds of billions of dollars being borrowed to then be invested in AI, other than "We'll just ask the AI what to do next". This is NOT a plan.This is an economic atomic time bomb never seen before.
No, neural nets did not have to be hand wired in the 1960s. They were being simulated.
ReplyDeleteThe AI LLMs are not just word predictors. They have now surpassed the intelligence of every human on Earth.
Sam Altman is not the primary man behind this AI mania. If he were to quit today, there would be no noticeable change in AI progress. -Roger
You do not know what you are talking about.
DeleteNeural nets of the 60s had such incredibly limited processing capability and memory, RAM was measured in duel digit K, long term storage was cards and magnetic tape. It was deemed unfeasible, and computer languages like LIST were considered far more promising avenues to pursue machine intelligence. Both my parents worked for the federal government through subcontractors at GRC and TRW working on various programs trying to employ large scale computers of the time to do things involving machine intelligence. The nets didn't scale with the resources they had available, and LIST quickly turned into an endless query laundry list that included everything BUT intelligence. They tried something like this approach again in the 80s with Cyc, and that didn't fare any better, it merely became yet another overwrought laundry list machine.
The very people who have designed and worked on the LLMs have openly said it isn't what people think it is, nor is it intelligent or self aware. There is something else entirely going on with the AI mania, it isn't good, and it isn't really AI.
Altman has openly said his plan to repay hundreds of billions is 'ask the AI how to do it'. THIS IS NOT A PLAN. He also is blathering about post scarcity, which is just socialist star trek goo dependent upon imaginary machines called 'replicators' to be possible.
The fact you don't know the computer hardware architecture limitations of the 60s, or understand how an LLM works, or what constitutes a general AI demonstrates that you probably shouldn't be discussing this topic without doing a hell of a lot more research before presenting yourself as knowledgeable.
CFT, my comments are correct. That computer language is LISP, not LIST. It is true that the MIT professors thought that LISP AI was more feasible than neural nets, and my post above explains that.
ReplyDeleteThere are about 100,000 people working on AI LLMs, and millions more who understand how they work. Yes, the AI is intelligent and nearly everyone agrees to that. Altman may have some funny ideas, but the field will go forward with or without him. -Roger
You are right about LISP, when I wrote that I just kept telling myself that it sounded wrong in my head since 1.) LISP is about lists, lots of lists, and 2.) It sounds like a speech impediment, and I didn't think that was something folks would use for a language designed to create artificial intelligence.
ReplyDeleteYou are wrong about LLMs.
Are LLMs Really Intelligent? by Hadi Alsibassi
The Illuision of Intelligence by Damien Kopp
Debunking LLM Intelligence:What's really happening under the Hood by Frederic Jacquet
A few examples is not proof of course, but many people deep into the reeds and weeds of how the models function acknowledge we aren't to general AI yet. The folks I list above talk about how the AI models work. LLMs are statistical word predictors, they have no self awareness or consciousness. They present the intended illusion of self awareness because of how they were programmed to handle certain linguistic prompts.
If you are having problems identifying the AI boom as a boom, and that spending trillions on something that has demonstrated no ability to pay for itself is a good idea, then I think you need to seriously reconsider the sheer economics and pull out a book about ROI and risk assessment. We are not long out of the Mortgage crisis of 2007 which almost destroyed the economy. A shame that people are so eager to make the same kind of mistakes so soon.
The entire AI boom is financially leveraged far beyond what it can ever produce even within just the limitations of the computer hardware it is composed of, as the life expectancy of the equipment has been financed over a period of time it simply won't be able to continue to function. Components which have a life expectancy of around two years are being amortized up to six. The projections for profit sound impressive, but they don't begin to even remotely approach the expenditure. For every dollar they earn, they are losing more than two, you can't possibly pay back a loan like this. When the bust happens (and it's already beginning to happen), it's going to take a hell of a lot of the economy with it.
No, the authors of those article do not understand intelligence, or how LLMs work. You say "LLMs are statistical word predictors", but that is just the "P" in ChatGPT. One article even denies that high-IQ people have more intelligence than low-IQ.
ReplyDeleteYes, the AI boom is a boom.
The GPU chips get superseded in about two years, but the five-year-old chips are still being fully used. Six-year depreciation is not crazy.
While many of the AI investments are likely to fail, some the AI companies are very profitable. There is definitely a lot of money to be made with AI LLMs.
Name one LLM that is generating profit, and by PROFIT I don't mean investor capital brought in by means of IPOs or convincing the government to throw money at them in a bullshit AI arms race, or merely looking at income without factoring in the servicing of debt. Profit is what comes AFTER your operating expenses, not before, so regardless of income, profit requires you to have subtracted expenses which include the servicing of your debt
ReplyDeleteName one PROFITABLE LLM Roger. Impress me with your superior understanding on this matter.
Google, Microsoft, Amazon, Nvidia, Anthropic, Palantir, Micron, Samsung, Apple, AMD, Broadcom, SpaceX, Meta, and others are all making billions of dollars from AI LLMs. Some of those are reinvesting their profits in a risky manner. Only Palantir is heavily dependent on government spending.
ReplyDeleteA financial circle jerk is not making billions. If you can't exactly report your earnings in $$$, you don't have any. If I say I give you a gazillion dollars of my services for wadazinga of your services in some perverse swap, we aren't making money, we are bullshitting and hoping no one notices before we con others into investing in our companies.
DeleteLet us put a knot in this for now, and revisit this 'profits in a risky manner' in another six or seven months.
I've seen this same kind of financial maneuvering before, from behind a retail counter in the 90s for the dot com bomb and the fall of the most useless valuable company that ever lived (AOL), and later from inside a bank during the mortgage crisis. Same bullshit with experts swearing up and down how things are fine, while they are packing their bags getting ready to leave town.