Questions tagged [jacobian]
For statistical questions involving the Jacobian matrix (or determinant) of first partial derivatives. For purely mathematical questions about the Jacobian it is better to ask at math SE https://math.stackexchange.com/.
86
questions
1
vote
1
answer
38
views
Transformation of a Random Variable
I am working on this problem for class, where the setup is the following:
Let X be a single observation from the $beta(\theta,1)$ pdf.
(a) Let $Y=-(logX)^{-1}$. Evaluate the confidence coefficient of ...
- 345
2
votes
1
answer
68
views
Reversible-jump MCMC and Poisson processes
Suppose we have a time interval $t \in [0, T]$ in which events occur as a Poisson process with some arbitrary time-dependent rate $\lambda(t)$. These events occur at times $Y=(Y_1, Y_2, \dotso, Y_M)$ ...
- 31
0
votes
0
answers
25
views
Computing the Jacobian in the Extended Kalman Filter with Non-additive noise
I have the following problem. I have the following Kalman filter:
$ \boldsymbol{x}_k=\boldsymbol{x}_{k-1} + \boldsymbol{w}_k$
$ \boldsymbol{y}_k=h(\boldsymbol{x}_{k}, \boldsymbol{v}_k)$
where $\...
1
vote
0
answers
41
views
A confusion about computing transformation of random variables
Let $(X,Y)$ be a pair of random variables with joint pdf $f_{XY}$. Let $(U,V)$ be two random variables obtained from $(X,Y)$ by $U = u(X,Y)$ and $V = v(X,Y)$ where $u$ and $v$ are, say, nice ...
- 11
1
vote
0
answers
32
views
Monotonicity of softmax (considering updates from all variables)
There's a relevant question here that doesn't quite answer my question, but I'm unable to comment.
Define softmax to be $$a_i = \text{softmax}(u_i)= \frac{e^{u_i}}{\sum_j{e^{u_j}}}$$
As the linked ...
- 11
-1
votes
1
answer
73
views
What is the relation between the coefficients of linear models and the Jacobian matrix?
What is the relation between the coefficients of linear models and the Jacobian matrix? Should the matrix of coefficients of a (generalized) linear model be thought about as the Jacobian?
1
vote
1
answer
27
views
A question on computational complexity of a numerical differentiation (equation (5.77)) in Bishop's Pattern Recognition and Machine Learning
In page 249 of Christopher M. Bishop's book "Pattern Recognition and Machine Learning", it is said
Again, the implementation of such algorithms can be checked by using
numerical ...
- 333
2
votes
1
answer
314
views
Mismatch between the dimensions of Jacobian matrixes when calculating derivatives during backprop?
I am trying to understand how back propagation works for a linear layer using minibatches by following this post: https://web.eecs.umich.edu/~justincj/teaching/eecs442/notes/linear-backprop.html.
...
- 359
1
vote
0
answers
231
views
How does the full derivative of softmax + cross entropy have the correct dimensions?
The blog post the softmax function and its derivative explains the following:
Imagine that each input has $N$ features / pixels / etc.
Imagine each input can be classified into $C$ classes
Let the ...
- 359
3
votes
1
answer
76
views
How does the fixed point interation in invertible resnets work?
I feel like I am missing some easy point about this invertible resnet paper which is making it hard for me to grasp how the fixed point iteration works.
stated simply, the residual connection in a ...
- 942
2
votes
1
answer
318
views
Is the Jacobian term needed if the prior is on the transformation parameter?
Suppose I have a strictly positive parameter $\sigma$ and I need to estimate it using the random walk Metropolis-Hasting algorithm.
I know that I can do a parameter transform, i.e., $\beta=log(\sigma)$...
- 453
0
votes
1
answer
953
views
Jacobian Matrix of an Element wise operation on a Matrix
Is it right in saying that the Jacobian Matrix of a Matrix output of an elementwise operation to the same input is a diagonal matrix ?
Context below.
From ref 1 it is clear that when you have an ...
- 231
3
votes
2
answers
220
views
Derivation of ELBO in ADVI Paper, Jacobian of Elliptical Transformation
I've been following the ELBO derivations in the paper Automatic Differentiation Variational Inference and have a few questions. With the model $p(x,\theta)$, they first transform $\theta$ so that it ...
- 33
0
votes
1
answer
1k
views
Change of variables by doing a transformation with a Jacobian versus finding an inverse
I have been solving one problem and there is something unclear to me in the solutions.
Namely, let's consider a probability density $p_x(x)$ defined over a continuous variable $x$, and suppose that we ...
- 117
1
vote
1
answer
4k
views
What is the Hessian of the Gaussian likelihood
I am trying to learn the fine differences between different methods of Kronecker factoring for approximate curvature (like [1], and [2]) which require taking the Hessian of the pre-activations of the ...
- 942