0
$\begingroup$

How to take the variance of a second order expansion? $\text{Var}\left[aX+bY+cXY+mX^2+nY^2\right]$

Let say we have 5 real-valued constant parameters $\{a,\ b,\ c,\ m,\ n\}$, and two random variables $X$ and $Y$ such their means values are $\mu_x$ and $\mu_y$, their variances are $\sigma_x^2$ and $\sigma_y^2$, and they are correlated with coefficient $\rho_{xy}$.

I want to know how to calculate the following variance and which value it has based on the previous parameters: $$\text{Var}\left[aX+bY+cXY+mX^2+nY^2\right]$$

  1. Could this be done if "nothing" is known about the probability distribution of each random variable (maybe different distributions for each one)? (but considering that it is true that each one has finite parameters $\mu_i$, $\sigma_i^2$, and $\rho_{ij}$).
  2. If point (1) is not possible: Could this be answer if $X$ and $Y$ are normally distributed? (I found the Isserlis' theorem and Stein's lemma, but I got stuck anyway).

I aiming to find a formula I could directly use, so please explain if your results are based in any assumptions (unfortunately, independence is not applicable for the result I am looking for).

$\endgroup$
6
  • 1
    $\begingroup$ You could expand your variance expression but I'm not sure you have everything you need to recover a nice formula, if there is one. $\endgroup$
    – user170231
    Commented Jan 4 at 19:13
  • 1
    $\begingroup$ Point (1) is impossible, because the given variance can assume any value (or may fail to exist) if there is no further constraints on the joint distribution of $X$ and $Y$. As for Point (2), it is in principle possible to expand the variance assuming $X$ and $Y$ are jointly normal (i.e. multivariate normal). However, the answer is very lengthy, being a polynomial in the variables $a, b, c, m, n, \mu_X, \mu_Y, \sigma_X, \sigma_Y, \rho_{XY}$ consisting of $48$ monomials. $\endgroup$ Commented Jan 4 at 19:28
  • $\begingroup$ @SangchulLee if I state that $\mu_i$, $\sigma_i$, and $\rho_{ij}$ exists (so are finite), Does it means that $$\text{Var}[aX+bY+cXY+nX^2+mY^2]$$ also exist and is finite? or could be that is not granted?.... also, do you have any link/reference to see the 48 monomial expansion? $\endgroup$
    – Joako
    Commented Jan 4 at 19:38
  • 1
    $\begingroup$ Unfortunately, still the answer is negative. For example, let $(X,Y)$ have a joint density $$f(x,y)=\frac{2}{\pi^2 (x^4+1)((x+y)^4+1)},\qquad x,y\in\mathbb{R}.$$ Then we can verify that $$\mu_X=0,\qquad\mu_Y=0,\qquad\sigma_X=1,\qquad\sigma_Y=\sqrt{2},\qquad\rho_{XY}=-\frac{1}{\sqrt{2}},$$ but $\mathbf{E}[X^3]$ fails to exist, and the same is true for OP's variance. $\endgroup$ Commented Jan 4 at 20:38
  • $\begingroup$ @Aruralreader I am not ChatGPT... I am an electrician with engineering degree... you could fact check due my translation mistakes along my questions, since I am not native in english $\endgroup$
    – Joako
    Commented Jul 6 at 5:55

2 Answers 2

1
$\begingroup$

Point (1) is impossible, because the given variance can assume any value (or may fail to exist) if there is no further constraints on the joint distribution of $X$ and $Y$.

As for (2), the answer is affirmative if $X$ and $Y$ are jointly normal (i.e., multivariate normal). In this case, it is easy to check that

$$ (X, Y) \quad \stackrel{d}= \quad \bigl(\mu_X + \sigma_X Z, \mu_Y + \sigma_Y(\rho_{XY} Z + \sqrt{1-\smash[b]{\rho_{XY}^2}} W) \bigr)$$

for i.i.d. standard normal $Z$ and $W$. Using this, we can in principle compute the variance of OP's variable

$$S = aX + bY + cXY + mX^2 + nY^2.$$

However, the answer turns out to be nasty. For simplicity, let

\begin{align*} \mu_X^{(2)} &= \mathbf{E}[X^2] = \mu_X^2 + \sigma_X^2, \\ \mu_Y^{(2)} &= \mathbf{E}[Y^2] = \mu_Y^2 + \sigma_Y^2, \\ \sigma_{XY}^{(2)} &= \mathbf{Cov}(X, Y) = \rho_{XY}\sigma_X\sigma_Y. \end{align*}

Then $\mathbf{Var}(S)$ can be expanded as:

Expansion of Var(S)

(Forgive my laziness for choosing to upload a formula instead of typing it out. This is too long, though...)

$\endgroup$
2
  • $\begingroup$ Thanks for the answer. Sincerely I wasn't expecting at all that it would be such a mess. In this question I am trying to explore the variable $z = \ln(e^x+e^y)=x+\ln(1+e^{y-x})$: there I found that the function $\ln(1+e^n)$ behaves like a smooth approximation of $\frac{n+|n|}{2}$, so for big "$n$" I have that $\ln(1+e^n)\approx n $, telling me that when $y\gg x$ then $z\approx y$ so I should have $\sigma_z^2 \approx \sigma_y^2$. But also by a Taylor's expansion of $\ln(1+e^n)\approx \ln(2)+\frac{n}{2}+\frac{n^2}{8}+O(x^4)$, (...) $\endgroup$
    – Joako
    Commented Jan 4 at 22:47
  • $\begingroup$ (...) I found that $z \approx \ln(2)+\frac{x+y}{2}+\frac{(y-x)^2}{8}$. If I keep only the 1st order term (so, $y-x\approx 0$), then $\sigma_z^2 \approx \frac14 (\sigma_x^2+\sigma_y^2+2\sigma_x\sigma_y\rho_{xy})$, the first 3 terms of your answer, and I am wanting to know what will happen when the difference moves between the linear to the saturated region. Since every term is positive, I see the variance will fastly increase by all the additional terms, as one variable gain size compared with the other, to somehow become again just one term comparable to the variance of the bigger one. $\endgroup$
    – Joako
    Commented Jan 4 at 22:56
1
$\begingroup$

For the first point, not, as it was explained in the comments. For the 2nd point, if we can assume normality, it is tedious and perhaps not so easy: $$ \text{Var}\left[aX+bY+cXY+mX^2+nY^2\right] = \mathbb{E}\left( aX+bY+cXY+mX^2+nY^2 - \mu \right)^2, $$ where $\mu = \mathbb{E}\left( aX+bY+cXY+mX^2+nY^2\right) = a\mathbb{E}(X) + b\mathbb{E}(Y) + c\mathbb{E}(XY) + m\mathbb{E}(X^2) + n\mathbb{E}(Y^2)$.

So now expand the parentheses to find the formula: $$ \text{Var}\left[aX+bY+cXY+mX^2+nY^2\right] = a^2\mathbb{E}(X^2) + b^2\mathbb{E}(Y^2) + c^2\mathbb{E}(X^2Y^2) + \dots - \mu^2. $$ I'm not going to bother with this simple expansion. I'm sure you can derive that yourself. You will then need to know the expectations of products of normal distributions. The product of two normal r.v.s is known and can be expressed as a chi-squared distributed r.v.. Or find it explicitly here. For the product of three or four normal r.v., it's much harder to get something explicit. You can try to read this paper.

Addendum: I'm not assuming joint normality like the other answer.

$\endgroup$
1
  • $\begingroup$ Thanks for the answer and the interesting links. I see it could be a real mess, I wasn't expecting it to be so complicated. Please check the comment section of the other answer, there I explain was I was trying to understand, and you answer was useful for the intuition I made with the mentioned answer. $\endgroup$
    – Joako
    Commented Jan 4 at 22:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .