What would a paper proposing a new theory to explain something with several lines of evidence look like?

Question

As an example, suppose I come up with a theory that removes the need for dark matter. Dark matter is a hypothetical substance with many different lines of evidence for its existence, but it hasn't been directly detected, so it's conceivable that it doesn't exist. However, there are several independent lines of evidence indicating it exists, and it's not easy to explain all of them within one new theory/model.

Which of the following is more appropriate?

1. The paper I write must simultaneously address every line of evidence. In other words, my new paper must explain all the observations. A single one is not enough.
2. The paper I write must explain one of the observations. Then it can end with "We leave the explanation of the other lines of evidence to future work", and that's enough for this paper (New Theory Paper: Part #1).

Option #1 seems like it requires a formidably large amount of work, taking at least several years to complete. There'd also be a high failure rate, so it would not be easy to even embark on such an endeavor. On the other hand, with option #2, it seems like a natural question the reviewer will ask is "What about the other lines of evidence?" and to conclusively answer that would require doing option #1. Furthermore, I would also expect option #2 to lead to New Theory Paper: Part #2 which says "We tried to explain ___ with our theory, but we couldn't get it to work, therefore the theory is probably wrong", and I'm not aware of any papers of this kind.

If option #1 is more appropriate, are there any examples of such papers? If option #2 is more appropriate, what does one say when the reviewer asks "What about the other lines of evidence"? What about when one later discovers that the theory cannot explain a different line of evidence?

Answer 1

A lot depends on the purpose and claims of the paper. If you claim to fully solve a problem, you'd better address the key lines of evidence and appropriate consistency criteria for a theory. (Note the restriction from all to key, as some observations are less certain or carry less weight than others.) But there is no need to make this a forced dichotomy between addressing all or only one of the key lines of evidence; as an author you should strive to write good, accurate papers that address what you can. Certainly, full agreement with all key lines of evidence need not be demonstrated in the initial paper, rather it may be enough to sketch a promising path forward (which may require expertise you don't have). In general, promising ideas are worth exploring, and are thus worth publishing. On the other hand, if you propose a model of dark matter that can never explain the observed galaxy rotation curves, your model just does not stand much chance as an explanation for our universe. I can still imagine cases where it would be valuable as a toy model or to provide constraints on future models, but any such paper should spell out this restriction.

This is all research, so much is always in flux or unclear. It is thus much more common to write papers addressing some specific properties and to treat them as pieces in a much larger jigsaw puzzle than it is to claim to have arrived at a true resolution. Genuine scientific progress is typically messy and hard to capture in a single paper. There are always future questions to be addressed. This is especially the case if your paper (like many of the most important theoretical physics papers) makes novel predictions.

Consider Steven Weinberg's famous A Model of Leptons paper, Phys. Rev. Lett. 19, 1264 (1967). This short letter brought together many ideas, and unified the electromagnetic and weak interactions. Weinberg did not declare this the theory of electroweak interactions, he just called it a model. And he flagged that the question of whether the model was renormalizable would need further investigation. That question was addressed affirmatively in 1971 by Gerard 't Hooft. Following experimental confirmation of weak neutral currents and the development of quantum chromodynamics, the remarkably successful standard model of particle physics was formed.

Option #1 seems like it requires a formidably large amount of work, taking at least several years to complete. There'd also be a high failure rate, so it would not be easy to even embark on such an endeavor.

Well, yes, nature provides many constraints on what kind of physical theories are useful for describing our universe. This is a feature, not a bug. However, we also don't want to stay trapped in local but not global minima in "theory space", so appropriately disclosed speculative excursions in various directions can be useful. Hence, as I outlined above, papers rarely go to the extreme of your option #1. However, it is a common criticism that the current funding climate and publish-or-perish culture systematically discourage the type of long-term and risk-taking efforts that are needed for truly groundbreaking work. Instead, more and more papers are produced, most incremental or of dubious value. After all, that's what the system rewards.

On the other hand, with option #2, it seems like a natural question the reviewer will ask is "What about the other lines of evidence?" and to conclusively answer that would require doing option #1.

It is a reasonable question to ask, but reasonable reviewers and editors also understand that some aspects will be left for future work. Again, if you can demonstrate your approach is promising also regarding aspects not addressed head-on in your paper your work is likely to be taken more seriously. If there are strong reasons to think your approach will never work in these regards, it might not be worth publishing.

Furthermore, I would also expect option #2 to lead to New Theory Paper: Part #2 which says "We tried to explain ___ with our theory, but we couldn't get it to work, therefore the theory is probably wrong", and I'm not aware of any papers of this kind.

I don't have an example of a paper like that off the top of my head. Of course, there is a bias against publishing purely negative results, with the possible exception of comment papers by other authors. Many other things can occur too: perhaps an observational paper comes out noting disagreement with the theory, or perhaps a new paper is written by the original author(s), highlighting some issue and proposing a tweak to resolve it or an entirely different model. But sometimes a dead end is a dead end and there are no more papers. The citation record can be a helpful heuristic for figuring out the usefulness of specific papers.

Answer 2

A published academic paper merely needs to add some novel contribution that is of value to the profession, and conducted in a scholarly way. When putting forward a new theory, it is possible to present a theory that is not fully worked out, so long as the contribution is sufficient to be of genuine value. A paper that presents a partially-worked theory that does not address all lines of evidence might still be of sufficient value to warrant publication. If there are lines of evidence that count against the theory then it would be expected that you would disclose these and draw them to the reader's attention, even if you don't deal with them.

Rather than the two options you present, I would recommend you go with option #3: Set out your theory and disclose all relevant lines of evidence on the problem, then state which lines of evidence are consistent with your theory and which lines of evidence you are (presently) unable to explain or which potentially count as evidence against your theory. You should at least mention all the relevant lines of evidence (and give citation to the relevant literature) so that the reader is given full information on the problem, and so that the reviewers are aware that you have considered the full problem. You can restrict your analysis to only some parts of the evidence if you wish, but it might be worth giving some brief remarks on part of the evidence you don't analyse --- e.g., are these something you think could potentially be explained in future, or do they flat-out contradict your theory?

Your goal in this kind of paper would be to make your reader familiar with the full problem, and all relevant lines of evidence, and then present your partial theory and show analysis of some of these lines of evidence. Be open about the fact that the theory is still partial and does not yet include analysis of all these lines of evidence, and be open about any lines of evidence that you think might be problematic to the theory.

Answer 3

Let us continue on the example of dark matter. There has so far been no direct evidence of its existence, but it is an assumption that allows to explain certain indirect observations that cannot be explained otherwise. If you have a model without dark matter, it would be necessary that it can explain these observations (if not all, then certainly the main ones) and that there are no inconsistencies with generally accepted theories that have already been confirmed by observation.

Answer 4

Here's an Option 3: state a new or unexplained observation that a new theory explains, but an existing theory doesn't. (If you could only write one paper, a new theory's consistency with existing evidence would almost certainly end up in an appendix or supplementary section.)

This most clearly states how your new theory would be different from the status quo, why it exists, and why it would be exciting and interesting for other people to pursue it.

After all, most people don't introduce relativity by saying "Einstein's groundbreaking realizations were that, at small speeds and low spacetime curvature, classical mechanics are correct."