How physicists discovered the math of gauge theories

We have seen that if you are looking for theories obeying a locality axiom, and if you adopt a geometrical view, you are inevitably led to metric theories like general relativity, and gauge theories like electromagnetism. Those are the simplest theories obeying the axioms.

Formally, metric theories and gauge theories are very similar. Both satisfy the locality and geometry axioms. Both define the fields as the curvature of a connection on a bundle. The metric theories use the tangent bundle, while the gauge theories use an external group. Both use tensor calculus to codify the symmetries.

The Standard Model of particle physics is a gauge theory, with the group being U(2)xSU(3) instead of U(1), and explains the strong, weak, and electromagnetic interactions.

Pure gauge theories predict massless particles like the photon. The Standard Model also has a scalar Higgs field that breaks some of the symmetries and allows particles to have mass.

The curious thing, to me, is why it took so long to figure out that gauge theories were the key to constructing local field theories.
Newton published his theory of gravity in 1682, but was unhappy about the action-at-a-distance. He would have preferred a local field theory, but he could not figure out how to do it. Maxwell figured it out for electromagnetism in 1865, based on experiments of Faraday and others.

Nobody figured out the symmetries to Maxwell’s equations until Lorentz and Poincare concocted theories to explain the Michelson-Morley experiment,in 1892-1905. Poincare showed in 1905 that a relativistic field theory for gravity resolves the action-at-a-distance paradoxes of Newtonian gravity. After Minkowski stressed the importance of the metric, the geometry, and tensor analysis to special relativity in 1908, Nordstrom, Grossmann, Einstein, and Hilbert figured out how to make gravity a local geometrical theory. Hermann Weyl combined gravity and electromagnetism into what he called a “gauge” theory in 1919.

Poincare turned electromagnetism into a geometric theory in 1905. He put a non-Euclidean geometry on spacetime, with its metric and symmetry group, and used the 4-vector potential to prove the covariance of Maxwell’s equations. But he did not notice that his potential was just the connection on a line bundle.

Electromagnetism was shown to be a renormalizable quantum field theory by Feynman, Schwinger and others in the 1940s. ‘tHooft showed that all gauge theories were renormalizable in 1970. Only after 1970 did physicists decide that gauge theories were the fundamental key to quantum field theory, and the Standard Model was constructed in the 1970s. They picked SU(3) for the strong interaction because particles had already been found that closely matched representations of SU(3).

It seems to me that all of relativity (special and general) and gauge theory (electromagnetism and the Standard Model) could have derived mathematically from general principles, with little or no reference to experiment. It could have happened centuries ago.

Perhaps the mathematical sophistication was not there. Characterizing geometries in terms of symmetry transformations and their invariants was described in Klein's Erlangen Program, 1872. Newton did not use a vector notation. Vector notation did not become popular until about 1890. Tensor analysis was developed after that. A modern understanding of manifolds and fiber bundles was not published until about 1950.

Hermann Weyl was a brilliant mathematician who surely had an intuitive understanding of all these things in about 1920. He could have worked out the details, if he had understood how essential they were to modern physics. Why didn’t he?

Even Einstein, who was supposedly the big advocate of deriving physics from first principles, never seems to have noticed that relativity could be derived from geometry and causality. Geometry probably would not have been one of his principles, as he never really accepted that relativity is a geometrical theory.

I am still trying to figure out who was the first to say, in print, that relativity and electromagnetism are the inevitable consequences of the geometrical and locality axioms. This seems to have been obvious in the 1970s. But who said it first?

Is there even a physics textbook today that explains this argument? You can find some of it mentioned in Wikipedia articles, such as Covariant formulation of classical electromagnetism, Maxwell's equations in curved spacetime - Geometric formulation, and Mathematical descriptions of the electromagnetic field - Classical electrodynamics as the curvature of a line bundle. These articles mention that electromagnetics fields can be formulated as the curvature of a line bundle, and that this is an elegant formulation. But they do not explain how general considerations of geometry and locality lead to the formulation.

David Morrison writes in a recent paper:

In the late 1960s and early 1970s, Yang got acquainted with James Simons, ... Simons identified the relevant mathematics as the mathematical theory of connections on fiber bundles ... Simons communicated these newly uncovered connections with physics to Isadore Singer at MIT ... It is likely that similar observations were made independently by others.

I attended Singer’s seminars in the 1970s, so I can confirm that it was a big revelation to him that physicists were constructing a standard model based on connections on fiber bundles, a subject where he was a leading authority. He certainly had the belief that mathematicians and physicists did not know they were studying the same thing under different names, and he knew a lot of those mathematicians and physicists.

String theorists like to believe that useful physical theories can be derived from first principles. Based on the above, I have to say that it is possible, and it could have happened with relativity. But it has never happened in the history of science.

If there were ever an example where a theory might have been developed from first principles, it would be relativity and electromagnetism. But even in that case, it appears that a modern geometric view of the theory was only obtained many decades later.