In the 1960s Friedrichs met Heisenberg and used the occasion to express to him the deep gratitude of mathematicians for having created quantum mechanics, which gave birth to the beautiful theory of operators on Hilbert space. Heisenberg allowed that this was so; Friedrichs then added that the mathematicians have, in some measure, returned the favor. Heisenberg looked noncommittal, so Friedrichs pointed out that it was a mathematician, von Neumann, who clarified the difference between a self-adjoint operator and one that is merely symmetric. "What's the difference," said Heisenberg.There is the difference between a physicist, and a mathematical physicist.
- story from Peter Lax, Functional Analysis (slightly edited for length)
To a physicist, an observable is a symmetric operator, because those are the ones that give real values, and only real values are observed. To von Neumann, an observable is a self-adjoint operator on a Hilbert space, where some additional technical requirements are needed in order to prove the spectral theorem.
I am not trying to say that Heisenberg was stupid. But it is striking that a world-famous physicist could get a Nobel Prize for using operators as observables, and still be oblivious to the formal mathematical definition found in textbooks. We cannot expect physicists to understand mathematical subtleties.