1

I understand that the batch size is the number of examples you pass into the neural network (NN). If the batch size is 10, it means you feed the NN 10 examples at once.

Assuming I have an NN with a single Dense layer. This Dense layer of 20 units has an input shape (10, 3). This means that I am feeding the NN 10 examples at once, with every example being represented by 3 values. This Dense layer will have an output shape of (10, 20).

I understand that the 20 in the 2nd dimension comes from the number of units in the Dense layer. However, what does the 10 (Batch Size) in the first dimension mean? Does this mean that the NN learns 10 separate sets of weights (with each set of weights corresponding to one example, and one set of weights being a matrix of 60 values:3 features x 20 units)?

nbro
  • 42,615
  • 12
  • 119
  • 217
jaksnak
  • 13
  • 3

1 Answers1

1

The Dense layers outputs 20 values per example. And since you have 10 examples in the batch the output is (10, 20) (one set of 20 values per example in the batch). The nn doesn't learn 10 separate sets of weights. Each set of 20 values is computed with the same weight (and bias if you have any). So if say example 2 and 5 had the same input values, they'll always have the same output values.

dbalchev
  • 56
  • 1