16

I just learned about GAN and I'm a little bit confused about the naming of Latent Vector.

  • First, In my understanding, a definition of a latent variable is a random variable that can't be measured directly (we needs some calculation from other variables to get its value). For example, knowledge is a latent variable. Is it correct?

  • And then, in GAN, a latent vector $z$ is a random variable which is an input of the generator network. I read in some tutorials, it's generated using only a simple random function:

    z = np.random.uniform(-1, 1, size=(batch_size, z_size))

then how are the two things related? why don't we use the term "a vector with random values between -1 and 1" when referring $z$ (generator's input) in GAN?

malioboro
  • 2,859
  • 3
  • 23
  • 47

2 Answers2

11

It is called a Latent variable because you cannot access it during train time (which means manipulate it), In a normal Feed Forward NN you cannot manipulate the values output by hidden layers. Similarly the case here.

The term originally came from RBM's (they used term hidden variables). The interpretation of hidden variables in the context of RBM was that these hidden nodes helped to model the interaction between 2 input features (if both activate together, then the hidden unit will also activate). This principle can be traced to Hebb's rule which states "Neurons that fire together, wire together." Thus RBM's were used to find representation of models, in a space (generally lower dimensional than than the original). This is the principal used in Auto Encoder's also. Thus as you can see we are explicitly, not modelling the interaction between 2 features, how the process is occurring is "hidden" from us.

So, the term latent basically can be attributed to the following ideas:

  • We map higher dimensional data to a lower dimensional data with no prior convictions of how the mapping will be done. The NN trains itself for the best configuration.
  • We cannot manipulate this lower dimensional data. Thus it is "hidden from us.
  • As we do not know what each dimension means, it is "hidden" from us.
5

Latent is a synonym for hidden.

Why is it called a hidden (or latent) variable? For example, suppose that you observe the behaviour of a person or animal. You can only observe the behaviour. You cannot observe the internal state (e.g. the mood) of this person or animal. The mood is a hidden variable because it cannot be observed directly (but only indirectly through its consequences).

A good example of statistical model that is highly based on the notion of latent variables is the hidden Markov model (HMM). If you understand the HMM, you will understand the concept of a hidden variable.

nbro
  • 42,615
  • 12
  • 119
  • 217