You are calculating the cross section of the process $u$ with $p_1$ and $\bar{d}$ with $p_2$ scatter and end up as $u$ with $p_3$ and $\bar{d}$ with $p_4$. Keep that in mind. You are not calculating the cross section for ending up as $u$ with $p_4$ and $\bar{d}$ with $p_3$ (for example).
For u-channel, the outgoing particles need to be identical, because the particle incoming as $p_1$ goes out with $p_4$ instead of $p_3$. Here, $p_4$ is defined as the outgoing $\bar{d}$ momentum, so having a $u$ leave with momentum $p_4$ is not the same outgoing state, and should not be incuded in the sum for calculating this process.
For s-channel mediated by a gluon or a photon, the incoming and outgoing particles need to be one another's antiparticles. The tree-level diagram can be phrased as "an up quark and an anti-up quark annihilate and become a gluon, which then becomes a new down and anti-down quark" (for example). Doesn't make sense to do this with an up and an anti-down - they can't annihilate and become a gluon. However, technically, there's a tree-level diagram for s-channel mediated by a W-boson. But that diagram is highly, highly suppressed by the W boson propagator.