Is it reasonable to not publish data from research for fear of undermining previous work?

Question

I am a postdoc working in a lab where we are finishing a large project with multiple measures (psychological, behavioral, etc). Some of the measures didn't turn out to have significant results (primarily for methodological reasons I believe). I suggested to the Lead Investigator that we write up some of these as "we tried these measures and didn't get the results we expected for various reasons: here are some lessons we have learned". The reply was:

My immediate response is not all that positive – especially if it undermined our published papers ... if you’ve published a result then further publications shouldn’t raise more limitations that should have been mentioned in the original publication.

I don't believe the results will undermine anything but, that aside, is his reasoning for not publishing results valid? Also, to be clear, we haven't yet published anything from this work.

I understand him to mean that he doesn't want the results published because they might go against previous work and completely object to such a thought. Have I possibly misinterpreted his words?

Answer 1

The statement you quote sounds terrible. It points in a direction where things approach the unethical. Publication of negative results are less common that positive although, I suspect, negative results ARE more common than positive. Since I am in an experimental field, I have never obtained the perfect results and always end up describing what went wrong; and I am not alone. Anyway, I think your idea of writing up your experience is good. Naturally, I cannot judge the merits of the experiences in terms of publicability.

So I think your intention is very sound and the reaction of the lead investigator suspect (or at best narrow-minded).

To me " further publications shouldn’t raise more limitations that should have been mentioned in the original publication" (if verbatim from the lead investigator and with my emphasis) sounds as if limitations were not included on purpose in the original study. So, if that is true then I can understand the nervousness. It sounds almost fraudulent.

Answer 2

If I understand you correctly, what you want to publish doesn't "go against" the previous work. It's fully consistent with it, but what is being added is what are often called negative results: namely, the discovery that a certain plausible method or established theory cannot be used in a certain manner to solve a certain problem.

In every branch of academia I know of, it is -- unfortunately -- very difficult to publish negative results. In the abstract, most people agree that this is a shame: convincingly arguing that researchers should not go down a certain path is obviously a service to the community. However, in a competitive publication system negative results are somehow not very, um, competitive. So trying to publish these results may not be the best use of your time unless you package them carefully -- very crudely, you will probably need to bundle them with "positive results".

The above is a relatively benign interpretation of your Lead Investigator's comments. It is alas also possible that he is saying that you should not try to publish your negative results because that information will reduce the prestige of your previously published paper. I agree that this sounds ethically dubious, yes, but depending on the field it is not clear that you are ethically required to publish further work explaining the limitations of your previous results. Here I would make a distinct between limitations and errors: errors are things that you should have known better than to include in the original version, and in many fields it is appropriate to print a retraction, erratum or corrigendum when you learn about them. On the other hand, that a publication contains ideas that are not completely definitive and whose future value is the subject of informed speculation....well, that's how research works.

I would say that the ethical issues become more acute depending upon the reaction that your published paper has received. If your paper has been very influential to researchers and is causing them to spend a lot of time exploring what you know to be a blind alley, then formal publication or no, it seems ethical to relate this information to them. If your publication has present nonacademic ramifications -- e.g. if you are in some kind of bio-medical field and your publication has resulted in patient protocols that you know to be suboptimal -- then you should be thinking about how to get the word out ASAP.

Anyway, it sounds like you need to have a conversation with your Lead Investigator about this. Find out what he meant and why.