1
$\begingroup$

I am analyzing some IHC data on the density of cells in two brain regions(factor 1) in two closely related species(factor 2). My data is composed of an n of 6 for each species and is not normally distributed(Shapiro-Wilk test). so I went for the non-parametric aligned ranks transformation ANOVA (based on the recommendations in: https://rcompanion.org/handbook/F_16.html). Till here everything should be fine but the problem starts from here. After I run the aligned ranks transformation ANOVA I get significant main effects(factor1: P= 0.040, F= 4.377; factor2: P= 0.042, F= 4.287) and a significant interaction between factor1 and factor2 (factor1 X factor2: P= 0.032, F=4.748). Based on these results I expect that the density of my cells will be different in an area and species-specific way. Therefore I conducted some post hoc tests, however, I didn't get any significant value in my post hoc tests. I know this is possible and this might indicate that the associations of species and area are not strictly addictive or that with our sample, it is not possible to specify which differences are responsible for our interaction (or main effects).

Considering all of this and that I know this is a small sample size and I cannot increase it, my question is: how can I handle this situation? From the perspective of a publication, how can I explain these results? could a transformation of my data help us understand where the effects of the ANOVA come from (is this a good approach even?)? Is there any test I can conduct to solve this problem? Should I just report it and that is it or am I missing something that could help me understand the discrepancy between the ANOVA and the post-hocs? If this is the case how should I proceed(please in this case explain it to me as clearly as possible)?

Thank you very much for all the help that could be provided. I saw there are similar questions posted but non of them was my specific case or in non of them i found a real answer that I could understand (maybe due to my lack of knowledge on this tests). Thanks again for any help this will be very helpful for me.

Improve this question
$\endgroup$
5
  • 6
    $\begingroup$ So you have two regions, two species and six data points per cell, so 24 data points altogether, right? I would say the very first order of business should be to plot your data. After that, there probably will be little more to say that your sample size is too small. Quite apart from your current predicament, I already find it heroic to fit a four-parameter model to just 24 data points. In your power and sample size calculation, you must have worked with huge effect sizes to be detected. It apparently turns out the true effect sizes are smaller. $\endgroup$
    – Stephan Kolassa
    Commented Oct 27, 2023 at 12:06
  • 2
    $\begingroup$ Along the lines of what @StephanKolassa wrote, the only way you can make progress with n=24 is for the response variable to be ultra high resolution with incredibly good technical replication (e.g., test/retest reliability). $\endgroup$
    – Frank Harrell
    Commented Oct 27, 2023 at 12:27
  • $\begingroup$ You ask about transforming data. I'd try a log transform which is based on the assumption that the data are lognormal. Here is one paper saying that cell densities are lognormal: academic.oup.com/cercor/article/33/16/9439/7219677 $\endgroup$
    – Harvey Motulsky
    Commented Oct 27, 2023 at 14:11
  • 1
    $\begingroup$ I have 24 points altogether but each of the 24 values is a mean of 5 values along the rostrocaudal extent of my brain areas( the 5 Sections with the higher immunoreactive cells in both brain areas, visually selected before my quantifications). $\endgroup$
    – Daniel Corrales
    Commented Oct 27, 2023 at 15:10
  • 2
    $\begingroup$ Ah, but your last comment is interesting! Why are you working off 24 averages rather than your 120 original values? Use a mixed model, AKA repeated measures, and things may look quite different. $\endgroup$
    – Stephan Kolassa
    Commented Oct 27, 2023 at 22:41

2 Answers 2

Reset to default
5
$\begingroup$

From my recollection of studying aligned rank tests in graduate school, I think they are somewhat ad hoc and are too parametric and possibly not robust, because you have to decide on a measure of central tendency to use in the alignment subtraction process. The mean in particular may not be robust.

To my mind many nonparametric procedures have been superseded by semiparametric ordinal models. For example the proportional odds (PO) model contains the Wilcoxon (paired and unpaired) and Kruskal-Wallis tests as special cases. You can include both factors, and their interaction (good luck estimating an interaction effect with n=24) easily in a PO model, and get all the likelihood ratio $\chi^2$ tests and estimates you need, from that one model. Learning resources for ordinal models are here.

The link you provided for aligned rank tests seems to be using rank-transformed ANOVA and calling it aligned rank test. This is not correct terminology. Rank-transformed ANOVA is a purely ad hoc procedure that does not give the right interpretation of interaction.

Improve this answer
$\endgroup$
4
$\begingroup$

First, I agree with everything Stephen said in his comment.

Second, to get an understanding of the situation, you can also make a table of the predicted values for each of the four combinations of brain region and species. The fact that there is an interaction means that the relationship between species and cell density is different for the two regions and, similarly, the relation between region and density is different for the two species. This is why you cannot say how much is due to each main effect. In some cases, the interaction is strong enough (and in the direction opposite to the main effects) that there is no difference at all in the DV for either IV taken alone.

Third, you talk about significant and not significant. This is not what you should talk about. Look at the various effect sizes. Andrew Gelman has an article titled "The Difference between 'significant' and 'not significant' is not, itself, statisticially significant" that you could look at.

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.