One of my student researchers is likely falsifying some results/data. What are the consequences?

Question

Relevant info and background:

I'm an engineering Post Doc at an American university. One of my roles is to basically function as a 'project manager' for a couple projects that have a number of Graduate-level RAs working on them. I have a good relationship with all the RAs and all are hard-working.

I'm convinced that one of the graduate RAs is falsifying computational results/data (also called "rigging data" by many) in some cases. Note that the individual appears to be doing this for only some cases, not all. I have many reasons to believe this, but here is a few: (1) inability to replicate various results, (2) finishing the work at a pace I think is not feasible, (3) finishing his work at home where he surely does not have the software environment to actually complete the work. There are also other reasons I believe this to be the case, but you get the point. I'm also convinced this has occurred for over 1 semester, so I probably need to report this since I am responsible for overseeing all the work. However, the student in general is a good person and hard worker. He has passed the preliminary exams and is finished with all classes - I'd hate to see him expelled from the university since he's this far into the program.

I have some questions:

1. What do you think could be the maximum punishment for this grad student/researcher? I'd feel terrible if it resulted in expulsion. I would think that you would have to receive at least one warning from the university before an expulsion, except in very extreme cases. I'd be fine if this resulted in suspension, and even losing funding, but for anything more I'd feel bad. What is the standard maximum punishment for these cases? Also, what is the most likely punishment?

2. What is the punishment for me if I don't report this problem? For instance, say I just pretended ignorance. It is extremely unlikely I would do this, but it's worth asking.

3. How common is this? I would think this happens once in a while - a grad student decides to be lazy and fabricate a small portion of the overall results to avoid working the weekend or something. An experienced professional would know this is seriously wrong, but not necessarily a mid-level PhD student.

Any advice from people with experience in this, professors, grad students, principle investigators, etc would be great

Answer 1

You have suspicions, but the evidence, as you sketch it here, is circumstantial. You need hard proof. Then you can (and must) act.

Falsifying data is a capital crime in academia. It wastes time, possibly years of other people's work. Don't let it get through. This person, if they indeed falsified data and would come through with this, will taint anybody and anything they had to do with - you, your group, your department, your university. Their results will be worthless, and so will be the degree you bestow on them.

You would feel sorry for that person if expulsed; but how sorry would you feel for a person who for 2 years will try to reproduce this grad student's results and fail for no fault of their own? How about their life and career? An honest mistake is one thing, but faking data? You are feeling sorry for the wrong person here; you'll spare the guilty and will let the innocent being impaled? A grad student is sufficiently mature to know better than to produce "synthetic" data.

How about the person abetting such a fabrication? Frankly, if caught, depending on the power structure that person may get away with a milder penalty "for not knowing what was going on", but in principle they should get the same, if not a harsher penalty, because they certainly cannot claim they didn't know that this is wrong; and they know the repercussions.

How common is it? Hard to say, but there were a number of large scandals (Jan Hendrik Schoen comes to mind), there is probably a halo of minor such attempts. From my own anecdotal stock: I once heard the conspiracy theory that spectroscopists would intentionally introduce "innocent" wrong factors into published formulas that could be interpreted as honest mistakes to prevent competitors from progressing. I didn't believe it, however, once I had to use such a formula from a paper, and to be satisfied I rederived it and some of its "brothers" myself in a tortuous process taking several weeks; lo and behold: I found that one of them had an integer factor wrong. It goes without saying that I have no real reason to assume it was intentional, but the conspiracy theory still lodges in the back of the mind.

Bottom line: if he really fakes data, letting this happen is not an option; but the evidence must be carefully and (important for fairness to the accused) confidentially vetted to establish whether this is indeed the case.

Answer 2

This misconduct is considered the ultimate misconduct in the research community. The offender is often stripped of his credentials and because of the tight knit nature of the scientific community, even if the credentials are not stripped the researcher may never find work as a researcher again. It will impact the ability to secure funding in the future.

If you are aware of it, as you claim to be, you can also be affected, ESPECIALLY if your name is on or associated with the paper. Additionally, if you are the one who secured the grant, this could backfire for you trying to secure grants in the future.

This is not common or uncommon, some people purposefully falsify data to support their hypothesis, but it is not always inaccurate. Sometimes researchers choose to only highlight some information and not other so that their hypothesis is supported and this is a more grey area.

BOTTOM LINE: If you know your student is falsifying data, then don't allow them to do so, for their career and for yours.

Answer 3

Communicate with the student. Let the student know your concerns.

The question seems rigged to determine what penalty may be appropriate, and how to kindly dish out the pain.

However, if we show good faith, then maybe we don't need to be quite as secretive. Say, "This resembles trouble. Here are the concerns." Then, if the student is innocent, the student may be able to explain things, and learn importance of proactively make things more clear so that suspicions don't grow into bigger problems than warranted.

If the student did do something wrong, maybe the student can correct things before they get further out of hand. The situation may be more correctable before more resources (including time) get spent on a road that may be wrong.

In education, the goal is often to help people do better. A common assumption is that people are typically inexperienced, and mistakes may be made. The goal isn't to try to maximize penalty for people who may be struggling with new skills. The goal is to try to get people in a good situation, including experience doing things desirably (including doing things properly, and successfully).

So, to re-cap this quite simply:

• if you're absolutely convinced that something is completely wrong, then go through the formal steps of handling such problems (reporting the issue, and whatever consequences follow through).
• If you're not, then communicate the observations that bring concern.
  • (If this communication results in more trouble being discovered, be ready to shift over to the first bullet point, as needed.)

Answer 4

What do you think could be the maximum punishment for this grad student/researcher?

Whatever the maximum punishment is, that punishment has been decided by the people running the university. If you consider your university to be a reasonably well-functioning institution (and I would hope you feel this way about the place where you have decided to spend several years of your career), you need to remember that the people making such decisions have much, much more experience than you in handling all different kinds of academic misconduct. Thus, the punishment is likely to have been well-calibrated over many years and based on a large amount of cumulative experience. What makes you think that your personal judgment on this question is more wise or likely to be correct than such a body of accumulated knowledge and experience?

By not reporting your suspicions, you would essentially be saying "I know better than everyone else what needs to happen to this student, so I will usurp the institution's right to properly bring the student to account for his actions and just act based on my own gut feeling to save myself from the feeling of guilt over the punishment that the student would receive (even though any such punishment would be 100% the student's fault)." This line of thinking is simply wrong. The punishment is not, and shouldn't be, your decision. You have a duty to report the misconduct, and by not doing so you would be making yourself complicit in all its many potentially harmful consequences, which were described quite well in the other answers.

Answer 5

You don't need to kick up a big fuss about it.

While it is definitely the case that any case of data fabrication is worthy of the levels of punishment it incurs in academia, it is not very clear that this is actually happening here. And in any case, the repercussions of scientific falsification should be very clear at any level, even for undergraduate students.

Inability to replicate results is extremely common in all scientific fields, and the overarching likelihood is that the analysis or experiments were carried out incorrectly for some reason. In the vast majority of cases, that is all there is to the story.

Simply deal with this problem as you would with any other inexplicable scientific result. Walk through the entire protocol, troubleshooting all potential issue spots, and exclude variables as required. In the extremely unlikely case that you find that the student was actually falsifying data, you must report it, but it seems unlikely to me that it is going to be the case.

Answer 6

If this person is falsifying data now, this person will continue to do so later as a PI. While you're sure to feel bad about it, science as a whole requires you to address the situation. When the public loses faith in science, we all suffer.

There are many ways to address this in a discreet manner (to ensure your intuition is accurate). Why not have this person walk you through the data/analysis step by step from ground zero?

Answer 7

In terms of immediate authority, I assume you and the student in question both ultimately report to a professor. I expect that professor is one or more of the PI on the supporting grant, the student's thesis advisor, and your supervisor. I really hope you have a strong, trusting relationship with this professor, for a few reasons:

• they will be the first line of investigation and response in dealing with this situation, and likely carry more personal/reputational and institutional responsibility for it than you do
• the student has likely worked with them longer than you have (postdoc there maybe 1-2 years, vs ABD student)

Basically, you don't want to end up in a position where your actions lead the professor to hold this against you. That could lead to withdrawn/non-renewed funding for you, withheld or weakened recommendations for future positions, and so forth. You really need the professor on board with the suspicions before any wheels of process start moving.

If there's some administrator responsible for this sort of issue that you know and trust not to jump the gun, you could potentially speak to them first to get your concerns on record before bringing them to the professor, to avoid the risk of the professor trying to sweep them under the rug and/or throw you under the bus.

Edit to add 1:

Ultimately, though, resolving this situation now, while the student is still pre-PhD, is in their best interest. If they aren't doing anything wrong, then they'll learn how to conduct their work in a more traceable, transparent, supportable, and reproducible manner. If they are, there's at least a chance that they can get straight without a permanent black mark on their career. Once they've gotten that degree, any such allegation could lead to it being revoked, grants they've received being suspended or cancelled, etc. This is the last point in their career where they can learn appropriate boundaries and reasonably hope to rehabilitate themselves.

Answer 8

I think three statements that you make are just your impressions and as you know that these are your impressions, you are not completly sure that student in question falsifies the data. Otherwise, I think you would not have asked the question here.

The best thing to do in order to be sure 100 % is to replicate all results with this student in your office on your computer. Otherwise, I think your statements are just your own impressions, without any solid evidence.

If you see that data is falsified, then you should report it.

Answer 9

I agree with Captain Emacs's answer, but there is something missing that I feel is important, namely:

Ask the RA directly whether he is fabricating any data, and while asking tell him why it is wrong to do so, and also that if he really does it and anyone finds out he can be expelled. At the same time tell him that at this juncture the best thing to do now is to redo all tests properly, meaning that he records all the random seeds used so that his data is completely reproducible.

After that it is likely that the problem will be resolved more or less satisfactorily, because it is generally difficult to write a program that looks normal and yet find a special random seed that causes it to have special behaviour. (It is possible but increasingly improbable for larger-scale tests.)

Answer 10

Falsifying data is a big no-no. It's on par with (and possibly worse than) plagiarism. It can ruin careers, and can lead to a whole host of huge problems (we don't need any more Andrew Wakefields). So if the student is doing this, you absolutely must report it and cannot feel bad.

That said, from the information you provided, there really isn't strong evidence. If I were on that jury, I would acquit without a second thought.

1) Working from home: Can he connect to a network to access the needed software? Can he run the program in the lab, get the raw data (say in a text file, or spreadsheet) take it home and do post processing/analysis there?

2) Not replicating data: I've written programs and ran simulations that performed beautifully and satisfied all the tests. But when I get to the group meeting, it fails. Why? Because I changed something that "wouldn't affect the results or the existing tests" (Ha!) between the time I originally got it working and the meeting. Or maybe an initial guess was changed. It might only take a few minutes to fix on my own, but in a meeting/high pressure environment I can't fix it right there. To me, that seems like a plausible explanation. (And I'm assuming there's no randomization in the code, I've had Monte Carlo approaches give significantly different results depending on the seed used).

3) Working faster than you expect: I see two possible explanations for this: a) The student is better than you think. b) The student is worse than you think. For (a), perhaps the student is able to crank out code fast, when he hits his stride and has a good mental map of where to go and how things should fit together (this "gunslinger" approach can be effective, but also can let bugs show up that make data replication difficult). Or he has written scripts to run several computations simultaneously or overnight. For (b), perhaps the student "hacks" everything in the code. Hardcodes things that should not be hardcoded, for example. Messes with things that shouldn't be messed with. This can give the illusion of working fast, but results in unmaintainable or inconsistent code, essentially borrowing time from the future.

Obviously, you have access to more information than we do, so perhaps these explanations don't apply. I would suggest talking to the student about the results, though not in an accusing way. Ask him to explain the results, explain what he did, and how he did it. Look through the code that he uses with him, make sure you both understand it. Perhaps there's honest mistakes to be corrected. Maybe there isn't a problem. If he seems to have no idea what he did or can't explain the procedure, then you probably want to bring up your concerns with the PI. But, under no circumstances can you let data falsification continue.

Answer 11

Tell the student that this doubt exists but in a one-on-one situation

To clear this doubt, ask him to make his results fully reproducible. It is in his own interest to show that he did not falsify anything. Show him that there is "immediate danger" that this gets investigated.

If he did not falsify anything (and your doubts were wrong), then this is a viable route. It requires some effort, but of course it can be done (and should be done, anyway). Then no damage is done, you only force him to work more transparently.

Allow him to redact work, if no harm has been done yet

This is the only "easy way out" that is in my opinion acceptable. In particular if nothing has been published outside of the university, you can allow him to redact falsified material, in order to replace it with real work. This may be punishment enough at this stage: It may set him back half a year towards graduation! But it may also require additional measures, depending on the severity.

He may then learn a key lesson here: while you may get away in highschool and maybe even undergrad, once the work gets more closely reviewed, misbehavior, copypasting and data fabrication is likely to be discovered, and this is not a good way of working. A backlash could come any time, and may ruin his reputation.

In the case that he admits cheating on this project, I would consider also reviewing earlier work, too.

If harm has been done, you want to redact anyway

If anything of this has been published yet, your name or your professors name is likely to appear on it, or at least be associated with it. In this case, you really will want to have this resolved...

At this point, it may be necessary for your own reputation to trigger a formal investigation; partially to clear yourself from any responsibility.

Answer 12

In a nutshell,

1) you must get a decent proof of misconduct, and

2) if/when you get it, you should report it immediately, even if this means the definite end of his career

Falsifying data is a cardinal sin for a scientist, and as many others remarked, it may waste years of work for other scientists try to replicate (and possibly improve) the faulty results. I was once caught in a situation where my student was trying to replicate somebody else's results, and it seriously impacted her PhD work, since the original authors' selective reporting (they later found that their solution works only in very limited cases, but did not share that finding until much later).

Now, before you rush to your superiors, I would advise you to talk to student about his methodology. Explain him that as (de-facto) project leader you have responsibility to guarantee that all results conform to scientific standards and that after you went through his work, you suspect there might be a problem with his results, as a result of unintentional mistakes or inexperience on his part. Start with this - if he made unintentional mistake (or even multiple mistakes), he will probably more than glad to learn from them and work hard to correct them. For a PhD student, this is sufficiently vague and yet serious that, if he is honest, he will work really hard to correct the problems (and redo the experiments). In that case it is your decision whether you trust him enough to have him on the team, and if you don't want, you should simply explain to your superiors that he does not fulfill your criteria, since he makes too many mistakes, and you do not need such people on the project, period.

I understand that this is a very difficult task for you, since you will have to waste your own time to go through his work and make sense of it, and probably you have your hands full with other work.

On the other hand, it may be pretty easy to spot if he is really dishonest or trying to hide something, because if he took a shortcut the first time by falsifying the results, I very much doubt he will "waste" his time correcting them - more likely he will try to weasel out or start making excuses, which then really means a red flag, and gives you a really good grounds to either confront him directly (usually it won't be necessary, as he will probably start digging his hole deeper and deeper) or just go and report him to the superiors. Because, if his mistakes were unintentional or result of carelessness but he does not feel he needs to correct them, he still deserves to be reported and sanctioned - refusing to learn from mistakes that others point out and refusing to correct them is almost as bad as falsifying data.

I do advise against going to superiors based only on a hunch, because an accusation of falsifying data may ruin his career even if he is not guilty, and further graduate students may become reluctant to work with you, fearing the same treatment (and some may even interpret your actions in a way that you got rid of competition down the road, which is the last thing you need).

And, since you are his superior, you are guilty if you do nothing, and the problems get discovered in his further career. You must act.