Who can really work on interpreting quantum mechanics and the standard model of particle physics?

Question

Starting from the answers to this question, even people not familiar with the interpretations of quantum mechanics can understand that physicists are not interested in substance metaphysics. This is understandable, as the scientific method requires experimentally verifiable predictions. But the result is a quite unsatisfying lack of answers to the "why" and "how" questions inherent to our nature.

I was attracted to science from an early age, because I had seen that countless key questions could be answered unequivocally. I was initially surprised to discover the limits in understanding we had hit with QM. But I was actually annoyed to see that the people most qualified to provide answers that could help make sense of the quantum weirdness had given up for decades. I now understand the reasoning, but will never accept that "x just is and doesn't make sense" or that "y can never be understood". It sounds too much like religion's cheap cop-out that our minds can't hope to understand God.

So who does the task of tackling these hard questions fall to? Scientists who dabble in philosophy and aren't afraid to publish unscientific theories? Philosophers who dabble in physics or read popularized versions of experimental results and interpretations of QM? Or has the academic community overall decided to give up on substance metaphysics? So we leave it to journalists to take interpretations like multiple worlds and present them as reasonable explanations of how the universe works? (I know the math for MWI works out, but I personally find it a ridiculous theory, that took Occam's razor and used it to shear sheep)

Answer 1

I agree with Conifold; here are my perspectives.

The general position of physicists on the "x just is and doesn't make sense" issue (also known as the shut up and calculate paradigm) is not that we have given up all hope of making sense of QM's predictions, but that 1) QM is under no obligation to "make sense" to even an extremely clever human mind which occupies a macroscopic, non-QM world, and 2) at the present moment we have no good candidates in hand for a brand new interpretation or mathematical conception of QM which "makes sense" (this is known as the foundational problem of quantum mechanics).

What constitutes a "quite unsatisfying lack of answers" is entirely a matter of opinion and in this context, those best qualified by actual experience to offer opinions on that are the practitioners themselves i.e., those who write down the equations and mathematically solve them for a living. If you can't do the math, whatever you might think about whether or not the "lack of answers" is in fact real and whether or not that is quite unsatisfying isn't relevant. I furnish here an analogy:

Imagine you have a new conception of the meaning and significance of the classic ancient Greek plays. Now, anyone who wants so can come up with any interpretation of those plays they wish, without justifying that interpretation to anyone at all. But if someone from the community of ancient Greek scholars then asks you if you 1) know how to read ancient Greek and 2) have used that skill to read the plays in their original form and you answer "no" to both questions, the chance that you will add anything of value to the body of knowledge on ancient Greek plays is so small that no one in the community is going to spend their time reading your thesis- especially when there is no shortage of unsolved problems in the field to work on.

Answer 2

It is clear based on the discussions and links in the comments that the task has been picked up by non-physicists, with several modern attempts at providing different interpretations. Very few (if any) of the authors are able to do the math themselves, but they seem to be consulting the people who can, to ensure that their proposals are compatible with observations. Some of the interpretations may be quite difficult to grasp and others may be unsettling, but I was very happy to learn that work is ongoing and that we haven't given up. Special thanks to @Conifold for his pointers.

Answer 3

There are physicists who approach the problem to satisfy their ontological needs. The are not content with just shutting up and calculating. David Bohm is one of them. If the by him repicked up version of de Broglie's pilot wave was taken seriously we would have been left with something less vacuuous than the now standard Copenhagen view. History decided not to take this path though and we are stuck with numerous interpretations to account for the collapse of the wave function. The many worlds being one of these outlandish attempts.

The hidden variable approach by Bohm is a viable version but because so little people pursue it it is not yet fully fledged. The main problem is agreement with special relativity but this problem can be surmounted by choosing the right hidden variables. It is often said that the Bell experiment contradicts the existence of these variables but it merely rules out local hidden variables and yiu can imagine that these are exactly the ones not needed. My country fellow man and Noble prize winner van 't Hooft tries to figure out a deterministic approach too. He is not happy shutting up. Or just to calculate, though his Noble Prize was awarded for his showing of the renormalizability of certain field theories. Of course hidden variables have their drawbacks too. They are hidden! But they at least offer an explanation devoid of empty probability.

So there are not just journalists trying to look for the meta stuff. There are also many physicists who consider the MWI to be the ultimate theory but they put to many emphasis on unitarity and serve us the most science fictional idea ever.

Answer 4

So who does the task of tackling these hard questions fall to? Scientists who dabble in philosophy and aren't afraid to publish unscientific theories? Philosophers who dabble in physics or read popularized versions of experimental results and interpretations of QM? Or has the academic community overall decided to give up on substance metaphysics?

An important fact to consider before answering the question are the goals of science and philosophy (which includes metaphysics).

Science seeks for empirical truth. Thermodynamic laws can be verified to a reasonable value of certainty just with simple tools as a thermometer, from a scientific standpoint. Now, consider this: temperature is a feeling, a simple metaphysical fact, which in physical (scientific) terms is quite difficult to approach. But that's not a problem of science. Science solves the issue just by adding an additional law (which, sorry, should have been there from the start, so, let's call it zeroth law!) saying that thermal equilibrium between systems is a transitive relation. Now, what the hell is that? You want to be precise and dig into that? Impossible. Any dictionary (even technical ones) is just a circular set of crossed-references. But that's not a problem of science. Science seeks empirical truth, not final truths.

The discipline that seeks for deeper truths is philosophy. Philosophically, yes, there are issues in thermodynamics, but not only there or in QM. There are far more deep problems. For example, just to start, the central axis of rationality: induction, which C.D. Broad said to be "the glory of science but the scandal of philosophy". QM, in such regard, is just a minor issue. How can QM be important, when reason itself has a broken gear?

To your punctual observation: let's say that science focus mostly on the object (see scientific realism and metaphysical positivism), and not on the subject. The subject would be essentially a metaphysical problem. QM proved impossible to assess only from the perspective of the object, without considering the subject.

So, by asking "who should work on QM?" it is already assumed that this is an issue demanding specialists (for example, either the philosophical or either the scientific community), and NOT generalists, which is a fallacy. In order to work on any issue, all the necessary background must be applied, that is, scientists and philosophers.

And here a problem raises: while philosophers are, since s.XVII, concerned with the issues of science and philosophy, and trying to enter into the scientific debate with a large disadvantage, scientists have taken a path of apparent independence of philosophy. Consider Richard Feynman saying that "philosophy of science is as useful to scientists as ornithology is to birds"... while he himself was one of the most remarkable philosophers of QM. Such is a quite important issue that needs an urgent solution. But no solution appears in the horizon for now. And the problem is moreover growing:

The distance between philosophy and science is not decreasing but getting larger with time. From my personal perspective, the fallacy is mostly on the scientific side: scientists believing that pure science, independent of the subject and its metaphysical issues, is possible. Scientists believing that science without foundations, without goals, without essence is possible. Making science nowadays is just publishing observations performed using the scientific method (which extent and application domain, in addition, are largely debatable).

On the other hand, philosophers have not supplied the necessary knowledge to produce scientific knowledge that would embed the elementary metaphysical principles. Immanuel Kant prepared the background to such goal: the formalization of metaphysics. Kant (successfully, IMHO) proved that metaphysics can reach the level of a science (which I interpret as getting a sufficient systematized formal background, as in any scientific field). But it seems that such task never got any follow-up. From a personal standpoint, I precisely work on a book about such subject. But the task is huge, it is not of academic interest, and it even sounds strange: who would want to formalize metaphysics, all that stuff about God? (such is the current understanding of metaphysics... and it is radically wrong: clearly, metaphysics has a bad reputation, mainly between men of science).

Answer 5

Even people not familiar with interpretation of quantum mechanics can understand physicists are not interested in substance metaphysics.

Your assertion appears to be contradicted by what physicists themselves have said:

In an article published by the Philosophical Transactions of the Royal Society A, Jaeger points out:

Heisenberg offered an interpretation of the quantum state which made qualitatative use of an earlier notion, dunamis, of Aristotle by referrimg to it by its Latin name potentia ... signifying the systems objective capacity to be found later to possess a property in actuality. For such an actualisation, Heisenberg required a measurement to have taken place ...

The Isha Upanishad states:

The Brahman forms everything that is living or non-living ... The wise man knows that all beings are identical with his own self and his self is the self of all beings.

Schrodinger was inspired by this to name

Atman = Brahman

As the second Schrodinger equation and said:

There is obviously only one alternative. Namely the unification of minds or consciousnesses. Their multiplicity is only apparent. In truth, there is only one mind. This is the doctrine of the Upanishads.

These are both substance metaphysics. You may have heard of Heisenbergs Uncertainty principle and Schrodingers equation. Perhaps a little more research on you behalf might be required before coming forward with such point blank assertions?