Timeline for Isn't it problematic to look at the data to decide to use a parametric vs. non-parametric test?
Current License: CC BY-SA 4.0
11 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jun 28 at 8:29 | comment | added | Stephan Kolassa | @fgp: I absolutely agree. But then these exploratory studies should be clearly labeled as such, and should use the NHST framework only with huge disclaimers. This is not what I am seeing: most studies are sold as confirmatory, which gives a distorted picture of how certain we can be of their results. | |
| Jun 28 at 8:09 | comment | added | fgp | @StephanKolassa To be frank, I find this approach to statistics not very constructive. A lot of research is exploratory, and many important discoveries would never have been made had we always insisted on a strictly executed design study, collect data, test hypothesis approach. I believe that a more nuanced approach is called for. Sometimes (say, during drug trails) strict adherence to procedures is necessary. At other times, a more exploratory approach is perfect reasonable. | |
| Jun 26 at 18:53 | comment | added | anjama | @Coris Many ecologists have that problem. Just imagine the labor, time, equipment, travel, and permitting costs of getting pilot data from a species in Antarctica, or any other remote location. Or trying to get data for a species where the population size is so small that you can't really have separate pilot and study data sets. Sometimes there are options, like collaborating with someone who happens to be doing other research in the study area, relying on data from prior studies, or trying to find possible data analogs from other systems, but not always. | |
| Jun 25 at 21:01 | comment | added | Stephan Kolassa | Yes, that makes sense. Of course, one could turn the argument around: especially when data is scare or hard to obtain (or you are running tests on living subjects), it makes sense to think about your analysis before you collect data and may find out afterwards than it was all for nothing. This may take other forms than collecting pilot data... but the "collect data first, worry about analysis later" paradigm surely is not the right way. | |
| Jun 25 at 20:49 | comment | added | Coris | Thanks for the answer! Regarding your paragraph about planning analyses, I wonder if there are disciplines where it's not possible or very difficult to gather pilot data, due to very expensive or very scarce study material, which can only lead to very small sample sizes (I have archeology in mind, but I'm not very knowledgeable about it, and perhaps there are better examples than that). I guess that in this case, we could use data from other similar studies, instead of pilot data from the specific study we plan to conduct. | |
| Jun 25 at 20:37 | vote | accept | Coris | ||
| Jun 25 at 16:34 | comment | added | Peter Flom | I recall one woman who ran a big group at (I think) Penn State. She insisted that all her researchers consult statisticians early and often. She also said she wanted to see all grant proposals 3 months before they were due. Her groups had a fantastic record of "acceptance" (not even "revise and resubmit")> | |
| Jun 25 at 16:30 | comment | added | Frans Rodenburg | @PeterFlom This sounds painfully familiar. I don't have a perfect solution, but at my institute, during courses where students perform their own mini-research, we grade them on their proposals, which are required to include an intended analysis. They can come by the statisticians of our department for help. | |
| Jun 25 at 16:25 | comment | added | Peter Flom | When I worked assisting researchers, either at Rusk Institute, NDRI or Downstate Medical Center, I constantly told them to come see me when they had a glimmer of an idea of something they were interested in. They almost never did. At Rusk, one person had years of questionnaires from her patients and gave me the files and asked me to do something with them. She had changed the questions multiple times, had no research question ... Oy! | |
| Jun 25 at 16:09 | comment | added | Dave |
Rather, people collect data and only afterwards come here to ask how to analyze it. To consult the statistician after...
|
|
| Jun 25 at 16:02 | history | answered | Stephan Kolassa | CC BY-SA 4.0 |