How to manage thesis professor who is wrong

Question

I am a med school student who is writing her master's thesis. I finished compiling my thesis with all the statistical analyses and wrote them as well. My problem is that today, when talking to the professor who is my advisor, he thought that the statistical part was wrong and asked me to "correct" it in a way that I am sure it is absolutely wrong. Now he wants me to write things his way, even though the deadline is tomorrow. I tried to calmly explain the theory behind it, but I quickly realized that being a doctor, he understands virtually no math or statistics.

My problem is that aside from the tight deadline, I don't want my name to appear on an analysis that is that wrong. I don't even know what I would do during the dissertation since I know for sure that those things are wrong, and I wouldn't be able to defend my thesis. What would you do in my shoes? Should I just write the bulls**t and be over with it? I'm exhausted from all the work I have done, and can't find the force or motivation to work on something I know has 0 value.

About the analysis, if you think I might be the one wrong:

• It's a retrospective study where I pooled around 80 patients. Each of them had done an examination 2 times within a certain time period, but with two different machines (once with machine A, once with machine B). We wanted to see if machine B was better when considering around 20 parameters that describe radiation dose and image quality in various ways. So the goal was to see if the same patient had more favourable parameter values when using machine B, when compared to machine A. The professor was very clear in saying that he didn't want to see the overall difference between the two machines: he wants to see how much the average patient benefits when switching from machine A to machine B.
• what I did: since we wanted to see the difference within the same patient, I first computed the difference of each parameter when using machine A and machine B, for each patient (so I had all the differences for the 20 parameters). Since patient number was limited, I first assessed if each difference was normally distributed across the patients (using a test called saphiro-wilk). When it was normal, I used the a paired t-test. For parameters that were not normally distributed, I used the Wilcoxon's ranked sign test. Additionally, for all significant results I computed the Cohen's d to quantify the effect size.
• what he wants me to do: compute the mean and standard deviation of each parameter as it is (parameter 1 in machine A, and parameter 1 in machine B). Then run a normal t-test. According to him, the tests that I used are too complicated, no one ever heard of them and no one would understand them, and anyway they make no sense. According to him, a normal t-test preserves the notion that it was the same patient who repeated the test twice. Also, he changed his mind and out of all the 20 parameters he made me look during the study, he suggested that we delete some, and for some others, I should add them together or take the mean (which makes absolutely no sense?!?!?!)
• I'm no statistician, I'm just a regular med student that is passionate about science and math. I spent the last month reading books I wasn't familiar with, and writing code on python to implement my analysis. So of course I might be wrong. But the professor's version doesn't sound right to me. Moreover, I feel desperate because I put so much work in it. I just don't feel like going back to coding to implement his wrong ideas (I know it's a simple iteration and I know I can use scipy.stats and that it wouldn't be so long. But I just can't take it anymore)

Sorry for the rant. If you can share your mind I will be grateful. For now, I'm just crying my eyes out on my bed, out of frustration, but maybe I'm not being objective, since I'm running on almost no sleep.

Edit: I finished everything in time and the result is somewhat decent! There still are some parts that refer to the older analysis, but I included it in the appendix (as some of you suggested) and mentioned it, so it should be fine. I can't thank you enough for your kind and constructive support! Having the chance to talk it out and hear some feedback really helped in getting started again.

The tight timeline is not so unusual at my university - at least for my department. But this one was definitely extreme. I addressed the issue this morning when talking to the advisor, hopefully he will take my feedback into account when dealing with future students.

Answer 1

I’m not in a position to comment on the specifics of the problem, so will answer the abstract question: what to do if my advisor tells me to do something that I know is incorrect, as a condition for my graduation?

This is tough. I’d say that if you have time, get the opinion of additional experts on the topic (statistics) who may explain why your method makes sense. You should graduate on time, so I do suggest changing things in accordance to what your advisor said, and then follow it up with advice from a domain expert who will hopefully clarify.

As an aside, the more serious issue is that your advisor only saw your results a day before they were due. Research and mentoring are all about good and regular communication, which is where things broke down in my opinion.

Good luck!

Answer 2

I have to agree his arguments you've listed don't convince me either. Also, it appears you chose the tools mindfully and, if asked, can explain the decision. In a pinch, that should suffice.

I understand it has already been 24 hours since the question was asked and the deadline has come. You have made your honest best effort and followed scientific integrity. Whatever follows, you will learn from this experience and become a better academic, medic, scientist for it. This one paper, one day of revisions doesn't define you as an academic. As many joke, your thesis hopefully will be your worst paper (because there are many more better and better papers to be produced by you).

Answer 3

Academia is a system which rewards popularity (citations) over truth. It's a system which hasn't fundamentally changed even though it's known that most published findings are false (Ioannidis, 2005). Instead, most of science is still occupied with providing intricate theories from within ivory towers which do not need to work in the real-world at all. The only requirement is that the peers in the field like what you are saying, so you can just echo some important-sounding words around and get many citations. For example, "psychology’s near-total focus on explaining the causes of behavior has led much of the field to be populated by research programs that provide intricate theories of psychological mechanism but that have little (or unknown) ability to predict future behaviors with any appreciable accuracy." (Yarkoni & Westfall, 2017). Similarly, software effort estimation research is mostly based on a few tiny datasets from the 1980-1990s, which are then analyses and re-analysed with trendy data science techniques even though the data itself doesn't reflect the changes in software development since then (Jones, 2021).

As someone who is also close to finishing my PhD, you can be sure that I and many other academics will respect your decision in both cases. Yes, I would prefer that you choose the "right" way. However, I fully respect the fact that you have almost no say in the matter. I wouldn't blame you if you decide to follow your supervisor's orders. I wouldn't even blame your supervisor since he probably has many responsibilities and he probably tries to do the best he can within the constraints that he works with.

Answer 4

You seem to be omitting quite a lot of what is/was going on, but let's ignore that for now and focus on the question. Tests have the main purpose of showing that something is correct/incorrect (and by how much). You are writing your thesis for doctors - this is the department that will award your degree. Many/most will understand a simple t-test, but fewer have even heard of some of the fancier stuff you describe - let alone understand it.

Putting all of those arguments in support of machine B, while lacking basic ones, is therefore NOT convincing. Such people will read the thesis, see fancy analysis and conclude that if you need all of this fancy analysis to show some benefits, there is actually no benefit to be had in practice and all of this is just smoke and mirrors.

Is it still valuable to have these better tests? Of course, so I believe that you should start the "Analysis" section or whatever it is named with the basic test. IF it shows improvements then great but if not then too bad.

In any case you should continue with results of fancier statistical analysis in the same section. Results of those methods should be in the main part of the thesis, while code could be in appendix; perhaps detailed description/derivation too.

Now, about grouping of those indicators, you can always have a single "goodness" indicator - which would likely be "health" in your case. It is an aggregate of all other indicators you can think of and there can be millions. Obtaining that single number is prone to fudging even if unintentionally. On the other hand, measuring all the separate indicators often leads to stuff like "we get 1% reduction in number of floaters at the cost of 1% higher neck muscle pain" (caricature of the idealized case. Too often only improvements are given). Besides, it is easy to fall in the trap of thinking you got some improvement, only for it to happen by chance (think https://xkcd.com/882/).

I believe you should trust advisor here as he likely knows which indicators matter. Obviously you should NOT simply sweep discarded parameters under the rug, you should mention all the parameters you measured during study, together with what you combined or discarded and most importantly - why. Even if others do not agree with your decisions, at least they will know the potential issues and limitations of your study.