Making 3 comparisons with a ternary variable

Question

I have a study in which I want to study the effect of a ternary variable $x$ in interaction with a binary variable $y$. The ternary variable is categorical, and for some reasons, I want to do the $3$ comparisons between the $3$ groups.

I understood that to do one comparison, I have to do 2 contrasts. For example if I’m interested in group1 vs group3, I define $C_1= (-1,0,1)$ and $C_2=(-1,2,-1)$. If I want to test all my comparisons, should I do $6$ contrasts then ? Won’t the regressors be colinear in this case ? Is there an easier way to do this ?

Thank you for your help !

---

Thanks for your answer! Your code works perfectly fine, but I'm not sure how I get the answer I need. In fact, when I use a reference for my ternary variable (1 as reference,2 and 3 as the 2 other values) and run a simple glm model, I get an output like that :

## y        9.782e-03  1.418e-04  69.008  < 2e-16 ***   
## x2       8.677e-05  9.265e-04   0.094  0.92539    
## x3      -3.617e-03  9.229e-04  -3.919 8.96e-05 ***
## y:x2    -5.481e-04  2.935e-04  -1.868  0.06184 .  
## y:x3     9.430e-04  2.941e-04   3.206  0.00135 **

Here you see that you have access to the p-values regarding the interaction of x with the variable y for the two first comparisons. I would like to get the third interaction term.

When I run the multiple comparisons with emmeans, I get something like that (it has more lines than that):

##  (y0 x1) - (y1 x2)  -0.060 0.0189 159##   -3.183  0.0186
##  (y0 x1) - (y0 x2)  -0.006 0.0277 159##   -0.216  0.9999
##  (y0 x1) - (y1 x2)  -0.046 0.0277 159##   -1.658  0.5601
##  (y0 x1) - (y0 x3)   0.207 0.0277 159##    7.472  <.0001

As I have access to all comparisons, I cannot simply analyze the interaction between variables x and y. I would like to have the same kind of output as the first chunk, but with the three comparisons, is there a way to get that?

Answer 1

Number of Comparisons

If I understand correctly, you have two categorical variables, one with $2$ and and one with $3$ categories. You want to compare all combinations of categories ($2\times3 =6$).

If my understanding is correct, you have not $6$, but $15$ possible comparisons:

combn(6, 2)
#      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12] [,13] [,14] [,15]
# [1,]    1    1    1    1    1    2    2    2    2     3     3     3     4     4     5
# [2,]    2    3    4    5    6    3    4    5    6     4     5     6     5     6     6

Obtaining all Contrasts

If you are using R, you can easily obtain all such comparisons with the emmeans package, which also has the added benefit of correcting for multiple testing by default:

library("emmeans")

# Simulate some data
set.seed(1234)
n    <- 100
x1   <- factor(sample(c("A", "B"), n, replace = TRUE))
x2   <- factor(sample(c("X", "Y", "Z"), n, replace = TRUE))
X    <- model.matrix(~ x1 * x2)
beta <- runif(6, -1, 1)
y    <- X %*% beta + rnorm(n) 
DF   <- data.frame(x1, x2, y)
rm(n, x1, x2, X, beta, y)

# Fit a model
LM <- lm(y ~ x1 * x2, DF)

# Perform all pairwise comparisons w/ multiple testing correction
EMM <- emmeans(LM, specs = c("x1", "x2"))
pairs(EMM)
#  contrast  estimate    SE df t.ratio p.value
#  A X - B X   0.0703 0.351 94   0.200  1.0000
#  A X - A Y   0.7067 0.385 94   1.835  0.4484
#  A X - B Y   1.0937 0.347 94   3.151  0.0258
#  A X - A Z  -0.6081 0.378 94  -1.611  0.5937
#  A X - B Z  -0.6802 0.351 94  -1.938  0.3859
#  B X - A Y   0.6364 0.366 94   1.740  0.5091
#  B X - B Y   1.0234 0.326 94   3.144  0.0263
#  B X - A Z  -0.6784 0.358 94  -1.896  0.4113
#  B X - B Z  -0.7504 0.330 94  -2.276  0.2141
#  A Y - B Y   0.3870 0.362 94   1.069  0.8923
#  A Y - A Z  -1.3148 0.391 94  -3.360  0.0140
#  A Y - B Z  -1.3869 0.366 94  -3.792  0.0035
#  B Y - A Z  -1.7018 0.354 94  -4.807  0.0001
#  B Y - B Z  -1.7739 0.326 94  -5.450  <.0001
#  A Z - B Z  -0.0721 0.358 94  -0.201  1.0000

---

Edit: What you are asking for in the updated question is an overall effect of the interaction term. You can obtain that through an ANOVA table:

summary.aov(LM)

#             Df Sum Sq Mean Sq F value   Pr(>F)    
# x1           1   0.63   0.634   0.614    0.435    
# x2           2  41.76  20.882  20.227 4.94e-08 ***
# x1:x2        2   0.88   0.440   0.426    0.655    
# Residuals   94  97.04   1.032                     
# ---
# Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
```

Answer 2

It is cumbersome to do all 15 comparisons between cells, as suggested by another answer. I think maybe you want to do 3 comparisons at each level of $x$. This can be done as follows:

mod <- glm(...)   # the model you fitted

library(emmeans)
EMM <- emmeans(mod, ~ y | x)
EMM   # Display the means (I hope you're interested in these)
pairs(EMM)

Interaction contrasts

Perhaps what you are looking for is interaction contrasts, which are contrasts of contrasts. With the above EMM, you could use

contrast(EMM, interaction = "pairwise", by = NULL)

With x at 3 levels and y at 2 levels, this would indeed yield three contrasts. They would be the differences of each x comparison between the two levels of y; i.e.,

[x1 - x2 | y = 1] - [x1 - x2 | y = 2], 
[x1 - x3 | y = 1] - [x1 - x3 | y = 2], and
[x2 - x3 | y = 1] - [x2 - x3 | y = 2]