Consider the differentiable functions $L^1(x,\theta^1),L^2(x^2,\theta^2),L^3(x^3,\theta^3)$, where every $x_k,\theta^k$ are real vectors, for $k=1,2,3$. Also define $\theta=(\theta^1,\theta^2,\theta^3)$ (and note that $x$ is not $ (x^1,x^2,x^3)$).
What is the jacobian, with respect to $\theta$, of $f(x,\theta)=L^3(L^2(L^1(x,\theta^1),\theta^2),\theta^3)?$
This question arose when computing the gradients in the backpropagation phase of a neural network, and I got a result that I don't think is correct (or at least it is not the one which backpropagation algorithms use).
Here is my try. Using the chain rule:
$Jf=JL^3 \cdot J(L^2(L^1(x,\theta^1),\theta^2),\theta^3)=JL^3 \begin{pmatrix} J_{x,\theta^1,\theta^2}L^2(L^1(x,\theta^1),\theta^2) & 0\\ 0 & I \end{pmatrix}=\left ( J_{x^3}L^3\cdot J_{x,\theta^1,\theta^2}L^2(L^1(x,\theta^1),\theta^2)\middle |J_{\theta^3}L^3\right )$
Hence $Jf=\left ( J_{x^3}L^3\cdot JL^2(L^1(x,\theta^1),\theta^2)\middle |J_{\theta^3}L^3\right )$, and by the above reasoning: $$Jf=\left ( J_{x^3}L^3\cdot \left (J_{x^2}L^2\cdot JL^1\middle | J_{\theta^2}L^2 \right )\middle |J_{\theta^3}L^3\right )=\left ( J_{x^3}L^3\cdot J_{x^2}L^2\cdot JL^1 \middle | J_{x^3}L^3 \cdot J_{\theta^2}L^2 \middle | J_{\theta^3}L^3\right )$$
I must conclude that $J_\theta f=\left ( J_{x^3}L^3\cdot J_{x^2}L^2\cdot J_{\theta^1}L^1 \middle | J_{x^3}L^3 \cdot J_{\theta^2}L^2 \middle | J_{\theta^3}L^3\right )$: is this correct?