Questions tagged [latent-variable]
26 questions
40
votes
5 answers
What is the difference between latent and embedding spaces?
In general, the word "latent" means "hidden" and "to embed" means "to incorporate". In machine learning, the expressions "hidden (or latent) space" and "embedding space" occur in several contexts. More specifically, an embedding can refer to a…
nbro
- 42,615
- 12
- 119
- 217
6
votes
1 answer
Why is the evidence equal to the KL divergence plus the loss?
Why is the equation $$\log p_{\theta}(x^1,...,x^N)=D_{KL}(q_{\theta}(z|x^i)||p_{\phi}(z|x^i))+\mathbb{L}(\phi,\theta;x^i)$$ true, where $x^i$ are data points and $z$ are latent variables?
I was reading the original variation autoencoder paper and I…
user8714896
- 825
- 1
- 9
- 24
5
votes
2 answers
How to generate new data given a trained VAE - sample from the learned latent space or from multivariate Gaussian?
To generate synthetic dataset using a trained VAE, there is confusion between two approaches:
Use learned latent space: z = mu + (eps * log_var) to generate (theoretically, infinite amounts of) data. Here, we are learning mu and log_var vectors…
Arun
- 255
- 2
- 8
4
votes
1 answer
What are some new deep learning models for learning latent representation of data?
I know that autoencoders are one type of deep neural networks that can learn the latent representation of data. I guess there should be several other models like autoencoders.
What are some new deep learning models for learning latent…
Kadaj13
- 143
- 4
3
votes
1 answer
Clarification on the training objective of denoising diffusion models
I'm reading the Denoising Diffusion Probabilistic Models paper (Ho et al. 2020). And I am puzzled about the training objective. I understood (I think) the trick regarding the reparametrization of the variance in terms of the noise:
$$\mu_\theta(x_t,…
user3903647
- 31
- 1
3
votes
2 answers
Do we also need to model a probability distribution for the decoder of a VAE?
I'm working on understanding VAEs, mostly through video lectures of Stanford cs231n, in particular lecture 13 tackles on this topic and I think I have a good theoretical grasp.
However, when looking at actual code of implementations, such as this…
ytolochko
- 385
- 2
- 6
2
votes
0 answers
VAE suffers from posterior collapse under all hyper parameters
I am trying to find a low-dimensional latent space representation for a bunch of simulated data. No matter what VAE architecture I try and no matter how I tweak it, the output of the VAE is always the same, where all values hover around 0; it does…
Patrickens
- 131
- 2
2
votes
2 answers
What is the meaning of log p(x) in VAE math and why is it constant
I was reading the article on medium, where the author cites this equation for Variational Inference:
\begin{align*}
\text{KL}(q(z|x^{(i)})||p(z|x^{(i)})) &= \int_z q(z|x^{(i)})\text{log}\frac{q(z|x^{(i)})}{p(z|x^{(i)})} dz \\
&=…
Kiran Manicka
- 113
- 6
2
votes
0 answers
Pointers to (deep) latent variable models that admit analytical approximations
I am aware that there is a plethora of deep generative models out there (e.g. variational autoencoders (VAE), GANs) that can model high-dimensional data as the images of latent variables under a non-linear mapping (typically neural network).
In more…
ngiann
- 121
- 4
2
votes
1 answer
how the GAN architecture maintain similar images close in the latent space?
I am learning about generative models, and I don't quite understand how the GAN architecture can maintain similar generated images close in the latent space. For example, an autoencoder and a variational autoencoder can map similarly generated…
Cesar Ruiz
- 680
- 2
- 8
2
votes
1 answer
Could it make any sense to choose a larger dimension for the latent space of the VAE with respect to the original input?
Could it make any sense to choose a larger dimension for the latent space of the VAE with respect to the original input?
For example, we may want to learn how to reconstruct a relatively low-dimensional input (let's say $20$ dimensions), then could…
James Arten
- 307
- 1
- 12
2
votes
0 answers
Why do hypercube latent spaces perform poorer than Gaussian latent spaces in generative neural networks?
I have a quick question regarding the use of different latent spaces to represent a distribution. Why is it that a Gaussian is usually used to represent the latent space of the generative model rather than say a hypercube? Is it because a Gaussian…
AlphaBetaGamma96
- 121
- 2
2
votes
0 answers
Does bottleneck size matter in Disentangled Variational Autoencoders?
I suppose that picking an appropriate size for the bottleneck in Autoencoders is neither a trivial nor an intuitive task. After watching this video about VAEs, I've been wondering: Do disentangled VAEs solve this problem?
After all, if the network…
fabs
- 21
- 1
2
votes
1 answer
In this VAE formula, why do $p$ and $q$ have the same parameters?
In $$\log p_{\theta}(x^1,...,x^N)=D_{KL}(q_{\theta}(z|x^i)||p_{\phi}(z|x^i))+\mathbb{L}(\phi,\theta;x^i),$$ why does $p(x^1,...,x^N)$ and $q(z|x^i)$ have the same parameter $\theta?$
Given that $p$ is just the probability of the observed data and…
user8714896
- 825
- 1
- 9
- 24
2
votes
0 answers
How can VAE have near perfect reconstruction but still output junk when using random noise input
I am creating a VAE for time series data using CNNs. The data has 4800 timesteps and 4 features. It is standardized and normalized. The network I am using is implemented in Keras as follows. I have used a MSE reconstruction error:
# network…
Samyak Shah
- 21
- 1