5
$\begingroup$

I am dealing with two scenarios: 1) Non-Gaussian data distribution and 2) non-stationary data).

First, I am planning to use a variational autoencoder for modeling the probability distribution of the non-Gaussian data distribution in the latent space. (Note, the input of the encoder part will be the non-Gaussian data). Then, I will it to perform some classification tasks.

However, is it possible to use the variational autoencoder to deal with non-Gaussian Distribution data? (because the non-Gaussian data is not generated from a Gaussian distribution).

Second, I also want to use the variational autoencoder to deal with non-stationary data. Would it be possible to use it with non-stationary data or should I consider other ML techniques.

Improve this question
$\endgroup$
7
  • 2
    $\begingroup$ why non-gaussian in parenthesis? you don't mean to say it's a synonym to non-stationary, i hope. stationarity has little to do with Gaussian. $\endgroup$
    – Aksakal
    Commented Apr 2, 2021 at 14:29
  • $\begingroup$ Yes, I didn't mean it is a synonym. What I mean with Non-Gaussian or non-stationary data. But, I see the confusion because of the parenthesis. Sorry for that, I will fix it. $\endgroup$
    – Amhs_11
    Commented Apr 2, 2021 at 17:42
  • 1
    $\begingroup$ Are you saying that the input data to the auto-encoder is non-gaussian, or that you wish to use a non-gaussian latent data representation in the VAE? $\endgroup$
    – Sycorax ♦
    Commented Apr 2, 2021 at 17:47
  • $\begingroup$ variational autoencoders are supposed to be used on non gaussian data. they try to transform them in a gaussian distributed latent variable, but the input data doesn't have to be gaussian. $\endgroup$
    – carlo
    Commented Apr 2, 2021 at 17:52
  • 1
    $\begingroup$ VAE will work with non-Gaussian data. they, and most ML techniques, won't work very well with non-stationary data out of the box $\endgroup$
    – Aksakal
    Commented Apr 2, 2021 at 17:55

1 Answer 1

Reset to default
4
$\begingroup$

Yes. The input\output of VAE doesn't need to be gaussian. The decoder can take the multivariate gaussian and transform it to other (possibly) non-gaussian distribution. The latent space of VAE is gaussian because of the following properties:

  • It is continuous, which is suitable for generating objects which are continuous by nature, and enables continuous interpolation.
  • It can easily be represented using only 2 parameters and is thus convenient to work with and can be represented easily.
  • The KL divergence between two gaussian (the second loss term in VAE) has analytical solution, and thus can be easily optimized using gradient descent
  • Gaussian is a natural pick for a prior of weights for neural network (it can be proved that it is equivalent to L2 regularization)
  • You can easily sample this distribution and use the reparameterization trick for the optimization.

However, gaussian is not the only choice for latent space of VAE (but for sure the most popular).

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you for your explanation!, so is it possible to consider the VAE as a non-parametric approach since it can deal with Non-Gaussian data?. I just wanted to understand the concept for that reason I asked this question. $\endgroup$
    – Amhs_11
    Commented Apr 3, 2021 at 18:19
  • 1
    $\begingroup$ It depends on the exact definition of parametric model see discussion here: stats.stackexchange.com/questions/322049/…. But definitely they can deal with non gaussian data $\endgroup$
    – ofer-a
    Commented Apr 3, 2021 at 19:10
  • $\begingroup$ Thanks @ofer-a for your explanation! $\endgroup$
    – Amhs_11
    Commented Apr 4, 2021 at 5:57

Not the answer you're looking for? Browse other questions tagged or ask your own question.