Where exactly does probability enter the theory?

Peter Woit writes:

A central question of the interpretation of quantum mechanics is that of “where exactly does probability enter the theory?”. The simple question that has been bothering me is that of why one can’t just take as answer the same place as in the classical theory: in one’s lack of precise knowledge about the initial state.

Lee Smolin says he is writing a book, and there are 3 options: (1) orthodox quantum mechanics, (2) many-worlds, (3) hidden variable theories, like pilot waves. All attempts at (2) have failed, so he says "My personal view is that option 3) is the only way forward for physics."

This is a pretty crazy opinion. No one has been able to makes sense out of probabilities in a many-worlds theory, and Bell test experiments have ruled out all sensible hidden variable theories.

Lubos Motl posts a rant against them, as usual:

Quantum mechanics was born 93 years ago but it's still normal for people who essentially or literally claim to be theoretical physicists to admit that they misunderstand even the most basic questions about the field. As a kid, I was shocked that people could have doubted heliocentrism and other things pretty much a century after these things were convincingly justified. But in recent years, I saw it would be totally unfair to dismiss those folks as medieval morons. The "modern morons" (or perhaps "postmodern morons") keep on overlooking and denying the basic scientific discoveries for a century, too! And this centennial delay is arguably more embarrassing today because there exist faster tools to spread the knowledge than the tools in the Middle Ages.

Lumo is mostly right, but it is possible to blame uncertainties on lack of knowledge of the initial state. It is theoretically possible that if you had perfect knowledge about a radioactive nucleus, then you would know when it would decay.

However it is also true that measurements are not going to give you that knowledge, based on what we know about quantum mechanics. This is what makes determinism more of a philosophical question than a scientific one.

I agree with Lumo that deriving the Born rule is silly. The Born rule is part of quantum theory. Deriving it from something equivalent might please some theorists, but really is just a mathematical exercise with no scientific significance.

This question about the origin of probabilities only makes sense to those who view probably as the essential thing that makes quantum mechanics different from classical mechanics. I do not have that view. Probabilities enter into all of science. It is hard to imagine any scientific theory that can be tested without some resort to a probabilistic analysis. So I don't think that the appearance of probability requires any special explanation. How else would any theory work?

It is very strange that respectable physicists can have such bizarre views about things that were settled about a century ago. I agree with Lumo about that.