The Bogus Dunning-Kruger Effect

Dr. Bee has a new video questioning the Dunning–Kruger effect. She gives papers on the subject 10/10 on the BS meter.

The dumber you are, the more certain you are that you aren't. That's the Dunning-Kruger effect, and it's been used to explain everything from bad politicians to the state of social media. Now psychologists have gone back to the biggest dataset we have and come out with the opposite conclusion. But there's a reason both sides can look at the same dots and see the opposite thing.

I agree with her. Somehow this obscure has entered public consciousness, and people commonly cite confidence as evidence of incompetence, according to this effect.

The whole subject appears to be an inside joke. Anyone citing the Dunning-Kruger effect is likely an idiot.

I have criticized the effect as bogus here, here, and here. The published effect is really just a misunderstanding of regression to the mean.

No Born Rule in Many Worlds Theory

There are dozens of papers on trying to reconcile the Born rule with the many-worlds interpretation of quantum mechanics. At first this seem bizarre. The Born rule is the main empirical prediction of QM, and any interpretation would have to obey it. How else could it be an interpretation?

The position I have repeatedly taken on this blog is that many-worlds is not an interpretation, and is wholly incompatible with the concept of probability, whether expressed by the Born rule or anything else.

Now a new paper agrees with me, and says it well:

Against Many Worlds
Emily Adlam, Jacob A. Barandes

Any viable interpretation of quantum theory needs to account for the Born rule, from which the theory gets its probabilistic empirical predictions. In this paper, we give an overview of possible approaches to this problem in the context of the Many Worlds interpretation. We argue that, for structural reasons, none of them can possibly succeed. More precisely, we argue that the Many Worlds interpretation must obtain the Born rule by proceeding either axiomatically, deductively, or inductively, and that all three of these approaches run into general, fundamental obstructions.

I previously posted a video explanation by Barandes.

You cannot just postulate probabilities for the many-worlds branches of the wave function. It is essential to many-worlds that the branches emerge as a result of decoherence. The branches cannot be assumed to have probabilities, and in fact no one has been able to make sense out of some branches being more likely than others.

The issue goes right to the core of what we mean by probability. For this, the paper recites a textbook probability problem for comparison.

A more fundamental obstruction is that merely having multiple things is not enough, by itself, to imply a conception of probability. To see why, consider, for example, a jar containing 43 red marbles among a total of 100 marbles. On the one hand, it would be entirely reasonable to say that the fraction of red marbles was 43%. On the other hand, it would not yet be reasonable to call this fraction a probability, because, at this point in the thought experiment, the probability would not be a probability of anything. It would not make sense to say that there was a 43% probability simpliciter.

Of course, if we were to stipulate that an agent were picking a marble ‘at random’ from the jar, under some suitable notion of ‘at random’ and with the further assumption that the marbles were ‘well-mixed,’ then we would be in a reasonable position to assert that there was a probability of 43% of the agent picking a red marble. But absent such a picking process, or the introduction of some other selection process to serve a similar functional role in the argument, there would be no probability, because, again, there would simply be nothing for the probability to be a probability of.

It is worth dwelling for a moment more on what premises are needed for a ‘picking process’ to legitimate the move from 43% qua fraction to 43% qua probability. Assigning a picking probability of 43% amounts to adopting a ‘principle of indifference’ (Eva 2019) assuming that all the marbles are ‘equally likely’ to be picked. And the philosophical literature on indifference principles makes clear that they are justified and successful only when they reflect relevant features of the process by which an outcome is selected, such as the symmetries of the picking process (van Fraassen 1989, Shackel 2007).

In particular, assigning a probability of 43% makes sense if the agent responsible for picking a marble is doing so by some process that is blind to color, meaning that there is a symmetry with respect to color – hence our invocation of terms like ‘at random’ and ‘well-mixed.’ Obviously if the agent is picking marbles in some way that is biased toward red marbles, then it would not make sense to assign a probability of 43%.

Just as you cannot abstractly assign probabilities to a jar of marbles, you cannot assign them to a wave function either. You have to have some notion of a measurement, and that is what the many-worlds folks refuse to do.

I doubt that any of these arguments are new. Many worlds theory is wholly incompatible with a scientific outlook, and is a crackpot belief.

The only way to understand many-worlds is as a complete rejection making scientific predictions, and a complete rejection of probability. That is, to accept it you have to say that no predictions can be made, and that probability theory does not make sense.

In Sean M. Carroll's recent podcast, he rambles in support of many-worlds theory:

I have heard you describe more unlikely worlds, thinner branches as not mattering as much. I don't know what your ontology of mattering is here, but it brought is it broadly speaking the fact that there are just more of the likely versions and therefore we should care about those more? If so, would you consider this a fundamentally utilitarian approach to mattering?

Well, uh it's not that there are more likely versions. ...

That's that's the entire trickiness of many worlds is that you can't get the counting that you need to get the Born rule right just from counting the numbers of worlds. The worlds don't count equally. They count as much as the amplitude squared. So 99% for this one, 1% for that one.

So I use the word mattering just to be indicative of whatever matters to you. So it might be the probability that you find yourself in such a world. It might be how much energy such a world has. It might be the utilitarian uh utility function that you have for such a world. Whatever you think is assigned to the world mattering goes along with the amplitude squared in many worlds. I think that's a good motto to keep in mind to get you through figuring out how to think about this scenario.

In quantum mechanics, the amplitude squared gives the probability. In many worlds, you cannot count the worlds or get more likely worlds. What you get is that some worlds subjectively matter more than others. You can think of the mattering as being based on probability or energy or utility function.

Really? This is all nonsense. There is nothing scientific about some worlds mattering more than others.

Superdeterminism makes Science Impossible

Superdeterminism has very few adherents, like Sabine Hossenfelder. I would ignore it completely, as too nutty to discuss, except that she has become one of the most prominent expositors of Physics.

Here is a new paper, discussing one of the main problems:

This is one of the longest-standing objections to (naïve) superdeterminist theories, often referred to as the Tobacco Company Syndrome (Hossenfelder and Palmer, 2020). Fundamentally, it is an epistemological concern. In Maudlin’s formulation (2019), the objection draws an analogy with a tobacco industry apologist who first invokes a common cause to deny that smoking produces cancer, and then, when confronted with randomised animal trials, insists that the randomisation procedure itself was somehow biased in favour of placing cancer-prone subjects in the experimental group, a move Maudlin regards as wholly unscientific. It is not difficult to see the risks to scientific practice if this objection were to remain unresolved within superdeterminism. Such a denial of Statistical Independence would imply, for instance, the impossibility of discussing isolated systems or experimental random errors. Moreover, it would pose a significant risk to the empirical coherence of the theory, as defined by Barrett (1996).

That is right, if you believe in superdeterminism then it is impossible to do an experiment to prove that tobacco smoking causes cancer. It is also impossible to do any other experiment, or reach any scientific conclusions.

Many-worlds theory has similar problems. It makes all scientific work impossible.

There is a Frame for Defining Now

From a recent video explaining relativity:

There is no way to draw a single 17:33 consistent line across spacetime that all observers agree separates has happened from hasn't happened yet. The failure parade. Some physicists argued this was a mathematical convenience with no ontological weight. But no experiment has ever found a preferred frame that would let now become absolute again.

Not correct. Such a frame is given by the Cosmic microwave background.

This video is AI generated, so should not be taken too seriously. It is a common error, so I point it out.

Relativitism Corrupted Humanities and Social Sciences

This blog focuses on the hard sciences and objective reality, mainly Physics. I expect physicists to be hard-headed scientists, and I am disappointed when they are not.

Outside the hard sciences, the academic world is a wasteland. They are concerned with relativitism, not relativity.

A bunch of experts were asked to write a report on the decline of humanities and social sciences. Objectivity has been abandoned. Political considerations have overwhelmed serious work.

Here is a summary by a biology professor. One example:

In 2023, the AAA canceled an accepted panel at its annual conference, titled “Let’s Talk About Sex, Baby: Why Biological Sex Remains a Necessary Analytic Category in Anthropology.” The AAA said at the time that the panel would have harmed members’ “safety and dignity,” and that its premise contradicted “settled science.” The Vanderbilt report cited this incident to argue that when politically charged questions “are treated as settled by the scholarly community, the result is an illegitimate suppression of dissent.”

Greene on the Most Mind-Boggling Result

Brian Greene answers questions:

And I'd say the most mind-boggling result of quantum 1:44:38 mechanics is the result of John Bell, John Stewart Bell, ...

But he was not willing to accept quantum entanglement at face value. He wanted to know if a version of quantum mechanics might be constructed that would be that would be local that wouldn't have these spooky non-local influences.

and he was trying to realize a dream really of Albert Einstein who thought that might be possible to come up with a new theory that embraces the same predictions as quantum mechanics but doesn't have this weird spooky action and he worked very hard thought he was coming very close to formulating that theory but every time he got to the near to the end it fell apart.

so finally he said to himself maybe this is a fool's errand maybe it's impossible to have a theory that agrees with the predictions of quantum mechanics and yet somehow excises spooky action non-local influences and he proved that that is the case a beautiful theorem John Bell proved a theorem which then because of the experimental data being such as to confirm the version that I just described that any quantum mechanically uh viable theory that agrees with the predictions of the standard approach is necessarily non-local. And so that to me is the most stunning implication. We necessarily live in a nonlocal universe. Yeah.

In fact, as a small teaser, I'll mention I'm actually writing a new book on quantum mechanics.

It is going to be a horrible book.

Quantum mechanics does not have any spooky non-local influences. Bell only proved that it differs from a type of local classical theories. It left open the possibility that a spooky classical theory might agree with QM, but that is not what Einstein was looking for.

Greene is famous for promoting string theory and multiverse theories. He now admits that these are speculative and unproven. Next he is going to promote quantum spookiness.

Meanwhile, Dr. Bee is writing a similar book and ventures into spookiness in her latest video on You probably misunderstand the double slit experiment. She is mostly correct, but goes off the rails when she says it proves either superdeterminism or spookiness:

4:37 But if the particle went through the other slit, you never interacted with it. You didn't touch the particle. It just knew you were trying to. The only way that you can avoid this faster than light update is that the particle knows whether you'll measure it before you've made the decision whether to measure it or not. This is what's called super determinism. And this is what's so weird about the double slit.

The error is that she keeps talking about particles, as if each has a definite position and momentum even when not observed. They do not. They are really waves, and waves always make interference patterns when going throught a double-slit, or even a single slit. There is no need to invoke spookiness or superdeterminism.

She refers to another quantum oddity. If you fire an electron at a double slit, it is really a wave that goes through both slits. If you put detectors on the slits, you only find one electron. If you observe the electron in one slit, you can be sure that the other detector will not see it. Conversely, if a detector says no electron went through one slit, then the other detector will see it in the other slit.

She describes this as very strange:

You make a measurement at one slit and the wave function on the other slit 4:22 changes immediately. And note that this is the case even if the particle does not trigger the detector. If it doesn't go into the detector, you know it must have gone through the other slit. But if the particle went through the other slit, you never interacted with it. You didn't touch the particle. It just knew you were trying to.

It is not so odd. Suppose I put my car key in my pocket, and I forget which of my two pockets. I can reach into one pocket, find it empty, and immediately deduce that my key is in the other pocket. I am finding my key, without interacting with it. It is not spooky or superdeterministic. The key did not know that I was looking for it.

It is peculiar to QM that measurements give eigenvalues. The theory says that the (unmeasured) electron goes partially through each slit. It could even tunnel through a barrier. But you never observe half an electron or an electron being in two places at once. Those are the real quantum mysteries. But the spookiness and superdeterminism are just nonsense.

Update: Greene has posted a long interview of Sean M. Carroll.

New Paper Calls Quantum Computing a Big Failure

Dr. Quantum Supremacy responds to a new paper:

A week ago, a philosopher named Amit Hagar put out a preprint entitled The NISQ Trap: Eight Years of Demonstrations the Hardware was Built to Lose. Here’s the abstract:

With a single clear exception, every NISQ-era flagship demonstration of ‘quantum advantage’ has, within eighteen months of its announcement, been classically reproduced, shown to rest on classically tractable structure, or closed by a simulability theorem.

Nevertheless, the basic thesis — that quantum supremacy in the NISQ (Noisy Intermediate Scale Quantum computing) era has been a failure, or even an example of pathological science — seems surprisingly widely shared, along with the opposite thesis that quantum computing already gives oodles of useful advantages for optimization and finance. ...

You’re right that, if you just look at the commercial value right now versus money spent to date, quantum computing looks like a colossal waste — as did classical computing before it scaled, as did nuclear power and heavier-than-air flight and every other nontrivial new technology.

Nobody invested a nickel in those other technologies, until someone had a proof of concept. Quantum computing is different in that it has attracted many billions of investment dollars, without a proof of concept, or a business plan on how it would ever make money.

This response is very weak. Go ahead and read it and decide for yourself. Someone publishes a paper saying the whole field has been a failure for 20 years, he admits that the view is widely shared, but says new technologies take time.

And of course he falls back on the argument that quantum computers have not been proved impossible:

Yes, the “real” goal remains, as it’s been since the 1990s, to build a scalable fault-tolerant quantum computer — and I’m glad that Hagar (unlike, say, Gil Kalai) never suggests that we’ve learned anything to rule that goal out. In the meantime, an intermediate goal would be to use NISQ devices to do physics and chemistry simulations that are commercially useful, or that help solve important scientific problems. The point of quantum supremacy experiments, you might say, is that by demonstrating the reality of quantum speedup about as clearly as it can be demonstrated with current hardware, they let us cleanly turn our attention to those more ambitious goals.

Hagar does not even talk about whether quantum computers are possible. His point is that they are empirical failures.

Update: Aaronson says:

For me, though, the real game always has been to learn the truth or falsehood of the Extended Church-Turing Thesis: that is, can all of nature be simulated on a standard Turing machine with polynomial overhead, or is quantum mechanics a counterexample to this? And from that standpoint, it seems obvious to me that the burden of proof lies with those who think the Extended Church-Turing Thesis still holds in a quantum-mechanical universe. Why does it hold? What astonishing new principle causes it to hold, despite the exponentially many canceling terms that generically arise in a path integral?

I get enraged when (some) QC skeptics seem not even to notice these profound questions, and content themselves to poke holes in this or that specific quantum fault-tolerance proposal and then call it a day. That seems to me to reflect a staggering incuriosity about the way the world actually is.

I am probably one of those enraging him. I agree that the big question is whether quantum computers can outdo Turing machines. Since he is a leading expert and he regards this as still an open question, then I accept that it is unsettled.

I disagree about the burden of proof. I say that a new technology should show that it works. He says the burden of proof is to show that it does not work.

The profound question is to understand why it does not work. I am somewhat incurious about this, because I am not sure any explanation will be fully satisfactory. Do we understand why F=ma, or how gravity workds, or why nothing goes faster than light, or why perpetual motion machines are impossible? Yes, I know the textbooks have explanations, but they do not stop people from theorizing about them, so apparently the explanations are not totally convincing.

That Extended Church-Turing Thesis was widely accepted for 50 years, including by very smart physicists who understood quantum mechanics. If quantum supremacy were possible, I would think that someone would have demonstrated it by now.

Update: Gil Kalai Says:

I also tend to agree with Amit’s conclusion that the quantum supremacy claims are either false or inconclusive. ...

For more than two decades, Scott has argued that the burden of proof lies on the shoulders of the skeptics. I suggest that we split the burden of proof as follows:

The burden of showing that the engineering obstacles to high-quality quantum error correction and quantum fault tolerance are not far beyond reach lies with the proponents of quantum computing. They still have a long way to go before providing convincing evidence that these goals are feasible and can be achieved in the foreseeable future.