There are embeddings $i_1,i_2$ of $SU(2)\cong S^3$ into $SU(4)$ as the top-left and respectively bottom-right $2\times2$ blocks. As you point out, both $i_1$ and $i_2$ induce isomorphisms on $\pi_3$. Since $i_1,i_2$ are conjugate embeddings, they induce the same isomorphisms.
Now let $j:SU(2)\rightarrow SU(4)$ be the diagonal subgroup inclusion. Notice that this inclusion can be factored
$$j:SU(2)\xrightarrow{\Delta}SU(2)\times SU(2)\xrightarrow{i_1\times i_2}SU(4)\times SU(4)\xrightarrow{\mu}SU(4)$$
where $\mu$ is the multiplication on $SU(4)$. To understand the action of $j$ on $\pi_3$ it is enough by the Hurewicz Theorem to understand its action on $H_3$, and hence on $H^3$ by the Universal Coefficient Theorem.
To this end let $x$ generate $H^3(SU(4))$. Then $i_1^*x=i_2^*x$ are generators of $H^3(SU(2))$. Call this image $y$. We have
$$\mu^*x=x\otimes 1+1\otimes x$$
and hence
$$j^*x=\Delta^*(i_1^*x\otimes 1+1\otimes i_2^*x)=i_1^*x+i_2^*x=2\cdot y.$$
That is, $j^*$ induces multiplication by $2$ on $H^3$.
We conclude that $j:\pi_3(SU(2))\rightarrow \pi_3(SU(4))$ is multiplication by $2$ (I'm being sloppy with signs because I want to use $i_1$ to identify each group with $\mathbb{Z}$). Thus if $X$ is the quotient $SU(4)/j(SU(2))$, then
$$\pi_3(X)\cong \mathbb{Z}/2.$$
Clearly this construction can be generalised by considering the diagonal embeddings of $SU(2)$ into $SU(2\cdot n)$ for any $n\geq2$. Another generalisation would consider the diagonal embeddings of $Sp(1)$ into $Sp(n)$.
To tie the room together, notice that under the exceptional isomorphism
$$Spin(4)\cong Spin(3)\times Spin(3),$$
the natural inclusion $Spin(3)\hookrightarrow Spin(4)$ corresponds to the diagonal subgroup of $Spin(3)\times Spin(3)$. The group $Spin(4)$ is not simple, but the method above explains why
$$\pi_3(Spin(n)/Spin(3))\cong\mathbb{Z}/2$$
for any $n\geq5$. Of course,
$$Spin(n)/Spin(3)\cong V_{n-3}(\mathbb{R}^n)$$
for $n\geq5$ is just the Stiefel manifold of $(n-3)$-frames in $\mathbb{R}^n$.
