How Kaluza-Klein ruined Physics

There is a new European paper on Albert Einstein and the fifth dimension. A new interpretation of the papers published in 1927. The fifth dimension is Kaluza–Klein theory.

This is one of those subjects where theoretical physicists have crazy ideas about how things ought to be.

In 1919, the German mathematician Theodor Kaluza developed a theory that maintained all the formalism of Riemannian geometry but extended the geometry's reach by proposing the possibility that Nature in fact utilized a five-dimensional spacetime, with electromagnetism appearing as a natural consequence of the unseen fifth dimension (the same idea was actually proposed by the Finnish physicist Gunnar Nordström in 1914, but was ignored). Kaluza communicated his idea to Einstein in the form of a draft paper, who was initially very enthusiastic about the concept of electromagnetism springing from the fifth dimension. But despite promises to assist Kaluza in publishing, Einstein sat on the idea for another two years before he finally recommended Kaluza's work for publication. ...

Consequently, in 1926 the Swedish mathematician Oskar Klein reexamined Kaluza's theory and made several important improvements ... Since that time, theories involving extra hidden (or compactified) dimensions have become known as Kaluza-Klein theories.

Big shot physicists continue to write papers about the idea today.

What makes these theories attractive is that gravity can be formulated as a geometric theory of curvature of 4-dimensional spacetime, and electromagnetism as a geometric theory of curvature of a circle bundle over spacetime. The circle bundle can be viewed as a 5th dimension to spacetime, or as a tiny circle at each spacetime point. See here for an explanation.

Einstein was strangely antagonistic to this geometric view. So are some modern physicists, like Steve Weinberg. The physics textbooks rarely mention it.

The Kaluza-Klein theories would be great if they emphasized this geometric view. But they don't. Instead, the starting point for those theories is that the geometric view is defective because there is no coupling between electromagnetism and gravity.

According to unified field theory dogma, as accepted by Einstein and most modern theoretical physicists, all forces should be unified in the way that Maxwell unified electricity and magnetism and light waves. It is impossible to understand magnetism without electricity, because they are both just manifestations of the same thing.

But with electromagnetism and gravity, you can learn them separately. When solving a problem, you can compute the electromagnetic effect, and then the gravity effect, and add them. There is a unified geometrical description of both theories, but there is no coupling, and you cannot pretend that they are the same.

It is like belief in God. You can believe in one god, or in multiple gods. If you believe in multiple gods, you don't see the work of one god necessarily has anything to do with the work of another. But try telling that to a believer in one god, and he will stubbornly insist that one god is responsible for everything, no matter what you say.

As it is with unified field theory. You just cannot convince a modern theoretical physicist that the fundamental forces are uncoupled. They will make the most extravagant assumptions, such as 6 extra Calabi-Yau dimensions, to justify their unified field theory preferences.

I used to think that unified field theory meant putting all the forces under a common geometric mathematical formalism. But the Standard Model does that, and the unified field theorists reject it.

The above paper quotes Einstein in 1921:

A theory in which the gravitational field and the electromagnetic field do not enter as logically distinct structures would be much preferable. H. Weyl, and recently Th. Kaluza, have put forward ingenious ideas along this direction; but concerning them, I am convinced that they do not bring us nearer to the true solution of the fundamental problem. I shall not go into this further [...]

The paper does not even mention the fact that you do get a very nice geometric theory if you are willing to accept the fields as "logically distinct structures".

That nice geometric theory turned out to be essential for the Standard Model. I cannot figure out who deserves credit for it. My best guess is Weyl, but it could have been Nordström or someone else. I also cannot figure out how Einstein convinced everyone that the forces have to be coupled in order to be "the true solution of the fundamental problem."

I think that historians should recognize Kaluza-Klein theory as a point where Physics went down a wrong path. They took a very good idea, the geometrization of the fundamental forces, with a very bad idea, a belief that all forces are coupled, and got a dead-end theory in 1921.

It appears that for a whole century, no one had the good sense to separate out the good idea from the bad idea.