What channels do people use to challenge or provide criticism for newly developed theories?

Question

I currently not in academia, but I am an undergraduate alumnus with no graduate experience.

What are the normal processes for academics to challenge each other’s theories or findings, or to support them.

Answer 1

The primary way to refute a theory is to do the research necessary and publish a paper (or papers) with a (more) correct theory. the primary way to support a theory is to (positively) cite it in future work that embraces it, especially to extend it.

The theory of the aether was refuted when Einstein published the theory of relativity. But it took years for confirmation of Einstein's ideas to become firm - it is still happening, actually.

Every paper that cites another is supporting the earlier work.

It isn't my field, but I've heard that the ideas of Freud are slowly being replaced in psychiatry.

This is the normal process of science, actually. Old ideas are replaced by newer and more supportable ideas. Think Galileo.

---

Minor channels, are discussion groups and arguments at conferences. But the "refutations" need to be backed up, not just claimed.

Answer 2

If the theory has not been published, usually one does not bother challenging it.

If the theory was published, most journals allow other authors to submit a "comment" or "matters arising" which can criticise it. This rarely occurs. Usually the comment is peer reviewed.

Answer 3

As stated in the answer by Buffy, the major road to criticise published results is to publish something yourself. There is a problem with this, which is that the academic publication process is slow, and may be held up even further if supporters of the criticised work recommend rejection when they are reviewers of critical papers. In particular, reviewers and editors often demand a better original theory or results than the one criticised. This can be counterproductive, as this requires new research which may be more difficult and harder to do than correctly finding problems in somebody's work. There is a handfull of journals publishing plain but well justified criticism; the majority, according to my experience, will demand additional original work, which obviously is a problem if the criticised work is wrong and will be used and cited as if true in the meantime! (I have occasionally criticised related work as a side remark in a published paper the main aim of which wasn't to criticise that work.)

Another major avenue for criticism is open to only a few, which is being a peer reviewer. You need to be invited, so this is not a way to criticise whatever you find wrong, but rather to criticise something specific you were asked to assess. However, if you establish yourself in science, you will be asked to do more peer reviews than you can do, and also young and relatively unknown scientists are asked (because much peer review is needed), so being a peer reviewer isn't very exclusive, also it is often effective as you can ask the authors to repair certain issue or recommend to reject the paper, but of course it's not a path to criticise stuff you are not invited to review. (Occasionally journals also publish discussion papers where either a chosen view or everyone interested is invited to write discussions.)

It is important to have in mind that human psychology is important also in science, and hardly anybody likes to be criticised in public. One should think that scientifically minded people are actually happy to learn from having errors pointed out by others, but careers and funding depend on this, so it is often not that easy.

My first channel for criticism will always be to contact the criticised authors directly. Most journals allow the authors to publish correction notes if they themselves realise that something is wrong in their work. Also, rather than straight away accepting that they were wrong, they may be happy to collaborate and come out with something that corrects the original issue but can be sold as an extension or at least as inspired by their earlier work rather than just admitting openly that "we were wrong". Chances to have an effect are probably better before work is finally published in a journal, as long as it is still only on preprint servers such as arxiv, privately circulated, or presented in preliminary stages at conferences.

I have pointed out issues at conferences but mostly this didn't have much effect. Authors may defend their point, or even agree with me, but may not change it anyway. Maybe the odd author learnt something and changed their approach at least in their next work, but this is rather the exception than the rule. (Obviously we also need to take into account that when criticising others we may be wrong ourselves, or the issue may have potential for genuine controversy, so we shouldn't expect anyway that everyone criticised by us says "you are right and I was wrong".) Obviously one can always criticise work by talking to others, and quite a bit of this is going on, but it is hard to know whether and how much of an effect this has.

On top of that there are more or less well visited blogs and open forums where stuff is discussed, probably also things like facebook groups (which I don't do). Chances are for criticism in such places to have any effect, the blog needs to be popular and the writer a "big name". However there is a number of such places where it is at least possible to write criticism without going through the peer review process, and some of this is read and even responded to by the authors.

In my field, statistics, Andrew Gelman has a very popular blog, which he uses to criticise work, and also to reflect about how difficult it is to criticise work effectively in science. Unfortunately it is very normal that criticised work continues to be cited and used and taken as "true". Searching for "criticism" and related terms on his blog brings up lots of stuff.

Gelman's blog on Statistical Modeling, Causal Inference, and Social Science, example entry on "A ladder of responses to criticism, from the most responsible to the most destructive"

Answer 4

Intro

One way to look at a theory is to treat it like a model. Models are simplified versions of reality allowing analysis and making predictions about reality. However, no model or theory is perfect, thus the limitations of a model are as important as the model itself.

Some time ago, the theory of a flat earth was good enough. People did not travel very far, there was no need for precise time measurements, movement of celestial objects was explained with divine intervention. Indeed, if all you want is to measure your own small crop field, it does not matter if on a grand scale surface of the earth is not flat.

A globe is a model of the earth and shows that the shortest path from Europe to the US is actually over the north pole. Something that is not evident from a flat map. However, a globe is a really poor model to say anything about earth's composition, core, and atmosphere.

Back to critiquing a theory.

• You can present a better theory in some way outperforming the earlier one. However, there is nothing to say that the earlier theory was bad. Merely, your theory is slightly more complete. Round earth orbiting around the sun explains the day/night cycle, seasons, lunar and solar eclipses, and enables much more precise distance measurements over long distances. Thus it outperformed the flat earth theory.

• You can demonstrate the constraints of an existing theory, or identify a contradiction suggested by the theory. Here is a good example: https://www.youtube.com/watch?v=d_XuFkVdAYU Doing so invites more research into the area.

A good start to evaluate any theory, especially yours if you aim to propose one, is to use the 7 criteria:

• scope - what does the theory can predict and what it cannot?

• logical consistency - does it makes sense, is not self-contradicting?

• parsimony - is it simple and elegant? This is connected to the Occam's Razor, "The simplest explanation is almost always the correct one"

• utility - is it useful?

• testability - can we demonstrate it, and more importantly, can we disprove it? This is an important one in the light of fake news. There is no way to prove (or disprove) that the world is governed by a large pipe-smoking rabbit or lizard people. Luckily, this theory fails at logical consistency.

• heurism - does it provide a good basis for extension and further research?

• test of time - Do the above criteria hold true over time. Einstein's theory of relativity was criticized at the time. However, it turned out to be one of the most elegant and useful theories ever.

In academia, the accepted way to disseminate results is to publish them in journals and conferences proceedings.

Outside academia, you can make use of a new theory by creating a new product and bringing it to the market. It is somewhat common that companies do their own in-house research to advance their products and business.