How are the weights connected in a neural network?

Question

Recently I was giving an interview and while I was explaining how an auto encoder works, the interviewer asked me "how did you connect the weights in the network?" I wasn't sure of the answer and thought maybe he was asking about the activation function used. But that didn't answer his question. Does anyone know, how to answer this question correctly.

Answer 1

Weights connection here most likely refers to the structure of the network, that is, how the weights of encoder and decoder are shared and connected between their respective layers. (V)AEs typically have a symmetric structure meaning the decoder mirrors the encoder, thus in some cases tied weights could be used where the decoder's weights are simply the transpose of the encoder's weights layerwise to reduce the number of trainable parameters and enforces a symmetric connection between encoding and decoding. For image data, convolutional autoencoders could be used and thus the weights are shared across spatial dimensions preserving the spatial structure of the input.