Foundation of probability theory

When physicists talk about chance, probability, and determinism, they are nearly always hopelessly confused. See, for example, the recent Delft experiment claiming to prove that nature is random.

Whether electron have some truly random behavior is a bit like talking about whether humans have free will. It certainly appears so, and there is even a relation between the concepts, and quantum mechanics leaves the possibilities open.

Randomness is tricky to define. Mathematicians have thought very carefully about the issues, and reached a XXc consensus on how to formulate it. So it is best to look at what they say.

UCLA math professor Terry Tao is teaching a class on probability theory, and this is from his introductory notes:

By default, mathematical reasoning is understood to take place in a deterministic mathematical universe. In such a universe, any given mathematical statement S (that is to say, a sentence with no free variables) is either true or false, with no intermediate truth value available. Similarly, any deterministic variable x can take on only one specific value at a time.

However, for a variety of reasons, both within pure mathematics and in the applications of mathematics to other disciplines, it is often desirable to have a rigorous mathematical framework in which one can discuss non-deterministic statements and variables – that is to say, statements which are not always true or always false, but in some intermediate state, or variables that do not take one particular value or another with definite certainty, but are again in some intermediate state. In probability theory, which is by far the most widely adopted mathematical framework to formally capture the concept of non-determinism, non-deterministic statements are referred to as events, and non-deterministic variables are referred to as random variables. In the standard foundations of probability theory, as laid out by Kolmogorov, we can then model these events and random variables by introducing a sample space (which will be given the structure of a probability space) to capture all the ambient sources of randomness; events are then modeled as measurable subsets of this sample space, and random variables are modeled as measurable functions on this sample space. (We will briefly discuss a more abstract way to set up probability theory, as well as other frameworks to capture non-determinism than classical probability theory, at the end of this set of notes; however, the rest of the course will be concerned exclusively with classical probability theory using the orthodox Kolmogorov models.)

Note carefully that sample spaces (and their attendant structures) will be used to model probabilistic concepts, rather than to actually be the concepts themselves. This distinction (a mathematical analogue of the map-territory distinction in philosophy) actually is implicit in much of modern mathematics, when we make a distinction between an abstract version of a mathematical object, and a concrete representation (or model) of that object.

Note his mention of the map-territory distinction, that I have emphasized here many times. Randomness is described by ordinary mathematical construction that look non-random. All of the formulas and theorems about randomness can be derived without any belief in true randomness. It is not clear that there is any such thing as true randomness. Randomness is a matter of interpretation.

Quantum mechanics often predicts probabilities, and physicists often say that this means that nature is random. But as you can see, mathematicians deal with probability and random variables in completely deterministic models, so probability formulas do not imply true randomness.

The Schroedinger equation is deterministic and time-reversible. And yet quantum mechanics is usually understood as indeterministic and irreversible. Is this a problem? No, not really. As you can see, mathematicians are always using deterministic formulas to model probability. These issues drive physicists like Sean M. Carroll to believe in the many-worlds interpretation (MWI), but that conclusion is entirely mistaken.