The 1969 Book that Killed AI Research

The neural nets at the core of AI were invented in 1943, and popularized in 1957-60 as the Perceptron. It had the basic idea that was later developed into today's AI LLMs.

But not everyone recognized it. Prominent MIT professors wrote a 1969 book, Perceptrons: An Introduction to Computational Geometry, trashing the concept.

Reports by the New York Times and statements by Rosenblatt claimed that neural nets would soon be able to see images, beat humans at chess, and reproduce. ...

This book is the center of a long-standing controversy in the study of artificial intelligence. It is claimed that pessimistic predictions made by the authors were responsible for a change in the direction of research in AI, concentrating efforts on so-called "symbolic" systems, a line of research that petered out and contributed to the so-called AI winter of the 1980s, when AI's promise was not realized.

The book claimed to prove that perceptrons were hopelessly limited, as they could not even approximate a simple xor function.

The authors doubled down in 1988 updated edition, saying "little of significance [has] changed since 1969". At the time there was a revival of interest in neural nets, but the authors said they would fail to scale up.

We now know that those neural nets of the 1940s, 50s, and 80s do scale up, and perform spectacularly well. The bold predictions of the perceptron inventor must have been seen as wildly optimistic at the time, but they turned out to be correct.

I post this as an example of how smart people can be wrong about what is possible, and how brilliant research could be sidetracked.

Rosenblatt made a hardware computer perceptron in 1960, and that is now in the Smithsonian Museum. What no one knew until about 1912 was that video game chips would be the key to scaling up neural nets.

Here is a recent video that tries to relate these old ideas about neural nets to current research.

Schrödinger was an Indeterminist

Conventional wisdom is that classical physics is determinist, while quantum physics is not.

New paper:

Was Schrödinger ever a determinist?
Flavio Del Santo, Nicolas Gisin

Erwin Schrödinger is often portrayed as a reactionary who resisted the indeterminism introduced by quantum mechanics, but on closer inspection his views on determinism prove to be more complex and more radical.

Schrödinger argued, correctly in my opinion, that neither classical nor quantum physics is deterministic.

(i) The first argument challenges the mathematical idealization of determinacy in the initial conditions. ...

(ii) The second argument for indeterminism concerns discontinuity and irreversibility. ...

(iii) In his final argument, Schrödinger links quantum indeterminacy to the abandonment of the concept of trajectory and to the essential role of discontinuity. Remarkably, he can be seen here as anticipating the notion of “information” in physics (he does not use the term “information”, which was formalized only in 1948 by C. Shannon), suggesting that nature may be understood as a discrete sequence of yes–no answers, representable as strings of 0s and 1s.

None of this should be surprising or controversial, except for the notoriety of those with contrary views.

Einstein was a determinist.

The many-worlds advocates, like Sean M. Carroll, are determinists.

Sabine Hossenfelder is a superdeterminist.

The Bohm pilot wave theorists, like Tim Maudlin, are determinists.

It is philosophically possible to be a determinist, just like the rest of the existential crisis iceberg is possible, but silly. If you believe in free will, or in scientific and experimental choices, then determinism is wrong.

Here is a new comment with a typical modern scoff about free will, rejecting any suggestion that it could be possible:

OMG

“It is our free will that enables it to be so.”

1) your brain is nothing more than a collection of atoms

2) the atoms in your brain are no different from the ones in any other clump, and they just move around in the only way that’s compatible with the state of all the other atoms in the universe and the laws of physics.

3) No matter the details of the physics, there is only cause and effect, with some amount of randomness (assuming it’s even a thing), and there’s absolutely no room for your ego to magically unshackle itself from this simple ground truth.

4) the concepts of freedom and will you’re invoking stem from the notion of counterfactuals – because alternative versions of cause and effect chains are conjured in our mind (like an AI hallucination), and you believe those could have happened, therefore, at every moment, we must have the ability to pick among those possible future paths arbitrarily, before even knowing what those possibilities are, or could have been. As if, when imagining the counterfactuals after the fact, you’re able to signal the past version of yourself to alter its “choice”, and this would repeat in a loop until it stabilizes. But even in this far fetched theory of retroactive influence, everything that happens is still a chain of inescapable causes and effects, going in a time loop. And the stable version is left with no freedom of will whatsoever.

The earlier commenter replied:

I simply don’t agree. If you can explain what causes an electron spin measurement to be either up or down, and thereby abolish superposition, then maybe I am mistaken. Otherwise I can base my decisions on the outcome of the measurement and not possible for any entity to predict my resulting actions a priori. The cause of my actions is the measurement result that is random with no discernible cause. The cause and effect chain starts only after measurement and not before.

I agree my brain is made from atoms but don’t agree it is just atoms as I look at collections of atoms that form my environment. My two dogs are collections of atoms but I realize one of them is not even bright enough to collapse the wave function. 🙂

How would one prove unequivocally there is no free will. Maybe if there were some digital simulation of me that perfectly predicted my actions then I would have to accept my sense of free will is illusory.. The future is perfectly predictable.

My point is that I can take steps to render a simulation inaccurate by consciously taking actions dependent on some measurement that cannot be known before the fact.

In lieu of a perfect simulation of me I will continue to believe that I have free will and that my future isn’t fully determined in a mechanistic way. If you have some predictive model of me we can conduct the experiment now to see if in fact I am perfectly predictable. I am ready and eager to participate. If you don’t then the claim that there is no free will is an unsupported belief while my belief I do have free will is supported by the failed experiment.

Free will is mostly a philosophical issue, but a few things can be said.

Everyone, except possibly for schizophrenics and stubborn philosophers, experiences free will and lives his life accordingly.

The above argument that science precludes free will is entirely fallacious.

Rejecting free will leads to nihilism and dispair.

Making a freely willed decision is somewhat like measuring an electron spin. For all the successes of quantum mechanics, we do not have a good theory for how such a measurement is determined. At best we can give probabilities. Some have even suggested that human consciousness is involved, it is so mysterious.

I am not saying that I have a physical explanation for how free will works. I do not. But it is also not the case that anyone has a deterministic explanation for how quantum measurements work.

The Bogus Dunning-Kruger Effect

Dr. Bee has a new video questioning the Dunning–Kruger effect. She gives papers on the subject 10/10 on the BS meter.

The dumber you are, the more certain you are that you aren't. That's the Dunning-Kruger effect, and it's been used to explain everything from bad politicians to the state of social media. Now psychologists have gone back to the biggest dataset we have and come out with the opposite conclusion. But there's a reason both sides can look at the same dots and see the opposite thing.

I agree with her. Somehow this obscure has entered public consciousness, and people commonly cite confidence as evidence of incompetence, according to this effect.

The whole subject appears to be an inside joke. Anyone citing the Dunning-Kruger effect is likely an idiot.

I have criticized the effect as bogus here, here, and here. The published effect is really just a misunderstanding of regression to the mean.

No Born Rule in Many Worlds Theory

There are dozens of papers on trying to reconcile the Born rule with the many-worlds interpretation of quantum mechanics. At first this seem bizarre. The Born rule is the main empirical prediction of QM, and any interpretation would have to obey it. How else could it be an interpretation?

The position I have repeatedly taken on this blog is that many-worlds is not an interpretation, and is wholly incompatible with the concept of probability, whether expressed by the Born rule or anything else.

Now a new paper agrees with me, and says it well:

Against Many Worlds
Emily Adlam, Jacob A. Barandes

Any viable interpretation of quantum theory needs to account for the Born rule, from which the theory gets its probabilistic empirical predictions. In this paper, we give an overview of possible approaches to this problem in the context of the Many Worlds interpretation. We argue that, for structural reasons, none of them can possibly succeed. More precisely, we argue that the Many Worlds interpretation must obtain the Born rule by proceeding either axiomatically, deductively, or inductively, and that all three of these approaches run into general, fundamental obstructions.

I previously posted a video explanation by Barandes.

You cannot just postulate probabilities for the many-worlds branches of the wave function. It is essential to many-worlds that the branches emerge as a result of decoherence. The branches cannot be assumed to have probabilities, and in fact no one has been able to make sense out of some branches being more likely than others.

The issue goes right to the core of what we mean by probability. For this, the paper recites a textbook probability problem for comparison.

A more fundamental obstruction is that merely having multiple things is not enough, by itself, to imply a conception of probability. To see why, consider, for example, a jar containing 43 red marbles among a total of 100 marbles. On the one hand, it would be entirely reasonable to say that the fraction of red marbles was 43%. On the other hand, it would not yet be reasonable to call this fraction a probability, because, at this point in the thought experiment, the probability would not be a probability of anything. It would not make sense to say that there was a 43% probability simpliciter.

Of course, if we were to stipulate that an agent were picking a marble ‘at random’ from the jar, under some suitable notion of ‘at random’ and with the further assumption that the marbles were ‘well-mixed,’ then we would be in a reasonable position to assert that there was a probability of 43% of the agent picking a red marble. But absent such a picking process, or the introduction of some other selection process to serve a similar functional role in the argument, there would be no probability, because, again, there would simply be nothing for the probability to be a probability of.

It is worth dwelling for a moment more on what premises are needed for a ‘picking process’ to legitimate the move from 43% qua fraction to 43% qua probability. Assigning a picking probability of 43% amounts to adopting a ‘principle of indifference’ (Eva 2019) assuming that all the marbles are ‘equally likely’ to be picked. And the philosophical literature on indifference principles makes clear that they are justified and successful only when they reflect relevant features of the process by which an outcome is selected, such as the symmetries of the picking process (van Fraassen 1989, Shackel 2007).

In particular, assigning a probability of 43% makes sense if the agent responsible for picking a marble is doing so by some process that is blind to color, meaning that there is a symmetry with respect to color – hence our invocation of terms like ‘at random’ and ‘well-mixed.’ Obviously if the agent is picking marbles in some way that is biased toward red marbles, then it would not make sense to assign a probability of 43%.

Just as you cannot abstractly assign probabilities to a jar of marbles, you cannot assign them to a wave function either. You have to have some notion of a measurement, and that is what the many-worlds folks refuse to do.

I doubt that any of these arguments are new. Many worlds theory is wholly incompatible with a scientific outlook, and is a crackpot belief.

The only way to understand many-worlds is as a complete rejection making scientific predictions, and a complete rejection of probability. That is, to accept it you have to say that no predictions can be made, and that probability theory does not make sense.

In Sean M. Carroll's recent podcast, he rambles in support of many-worlds theory:

I have heard you describe more unlikely worlds, thinner branches as not mattering as much. I don't know what your ontology of mattering is here, but it brought is it broadly speaking the fact that there are just more of the likely versions and therefore we should care about those more? If so, would you consider this a fundamentally utilitarian approach to mattering?

Well, uh it's not that there are more likely versions. ...

That's that's the entire trickiness of many worlds is that you can't get the counting that you need to get the Born rule right just from counting the numbers of worlds. The worlds don't count equally. They count as much as the amplitude squared. So 99% for this one, 1% for that one.

So I use the word mattering just to be indicative of whatever matters to you. So it might be the probability that you find yourself in such a world. It might be how much energy such a world has. It might be the utilitarian uh utility function that you have for such a world. Whatever you think is assigned to the world mattering goes along with the amplitude squared in many worlds. I think that's a good motto to keep in mind to get you through figuring out how to think about this scenario.

In quantum mechanics, the amplitude squared gives the probability. In many worlds, you cannot count the worlds or get more likely worlds. What you get is that some worlds subjectively matter more than others. You can think of the mattering as being based on probability or energy or utility function.

Really? This is all nonsense. There is nothing scientific about some worlds mattering more than others.

Superdeterminism makes Science Impossible

Superdeterminism has very few adherents, like Sabine Hossenfelder. I would ignore it completely, as too nutty to discuss, except that she has become one of the most prominent expositors of Physics.

Here is a new paper, discussing one of the main problems:

This is one of the longest-standing objections to (naïve) superdeterminist theories, often referred to as the Tobacco Company Syndrome (Hossenfelder and Palmer, 2020). Fundamentally, it is an epistemological concern. In Maudlin’s formulation (2019), the objection draws an analogy with a tobacco industry apologist who first invokes a common cause to deny that smoking produces cancer, and then, when confronted with randomised animal trials, insists that the randomisation procedure itself was somehow biased in favour of placing cancer-prone subjects in the experimental group, a move Maudlin regards as wholly unscientific. It is not difficult to see the risks to scientific practice if this objection were to remain unresolved within superdeterminism. Such a denial of Statistical Independence would imply, for instance, the impossibility of discussing isolated systems or experimental random errors. Moreover, it would pose a significant risk to the empirical coherence of the theory, as defined by Barrett (1996).

That is right, if you believe in superdeterminism then it is impossible to do an experiment to prove that tobacco smoking causes cancer. It is also impossible to do any other experiment, or reach any scientific conclusions.

Many-worlds theory has similar problems. It makes all scientific work impossible.

There is a Frame for Defining Now

From a recent video explaining relativity:

There is no way to draw a single 17:33 consistent line across spacetime that all observers agree separates has happened from hasn't happened yet. The failure parade. Some physicists argued this was a mathematical convenience with no ontological weight. But no experiment has ever found a preferred frame that would let now become absolute again.

Not correct. Such a frame is given by the Cosmic microwave background.

This video is AI generated, so should not be taken too seriously. It is a common error, so I point it out.

Relativitism Corrupted Humanities and Social Sciences

This blog focuses on the hard sciences and objective reality, mainly Physics. I expect physicists to be hard-headed scientists, and I am disappointed when they are not.

Outside the hard sciences, the academic world is a wasteland. They are concerned with relativitism, not relativity.

A bunch of experts were asked to write a report on the decline of humanities and social sciences. Objectivity has been abandoned. Political considerations have overwhelmed serious work.

Here is a summary by a biology professor. One example:

In 2023, the AAA canceled an accepted panel at its annual conference, titled “Let’s Talk About Sex, Baby: Why Biological Sex Remains a Necessary Analytic Category in Anthropology.” The AAA said at the time that the panel would have harmed members’ “safety and dignity,” and that its premise contradicted “settled science.” The Vanderbilt report cited this incident to argue that when politically charged questions “are treated as settled by the scholarly community, the result is an illegitimate suppression of dissent.”