6

I'm trying to create a simple blogpost on RNNs, that should give a better insight into how they work in Keras. Let's say:

model = keras.models.Sequential()
model.add(keras.layers.SimpleRNN(5, return_sequences=True, input_shape=[None, 1]))
model.add(keras.layers.SimpleRNN(5, return_sequences=True))
model.add(keras.layers.Dense(1))

I came up with the following visualization (this is only a sketch), which I'm quite unsure about:

enter image description here

The RNN architecture is comprised of 3 layers represented in the picture.

Question: is this correct? Is the input "flowing" thought each layer neuron to neuron or only though the layers, like in the picture below. Is there anything else that is not correct - any other visualizations to look into?

enter image description here

Update: my assumptions are based on my understanding from what I saw in Geron's book. The recurrent neurons are connected, see: https://pasteboard.co/JDXTFVw.png ... he then proceeds to talk about connections between different layers, see: https://pasteboard.co/JDXTXcz.png - did I misunderstand him or is it just a peculiarity in keras framework?

Mindaugas Bernatavičius
  • 161
  • 3

1 Answers1

0

The first image is correct. The information will flow from left to right in each layer and from top to bottom in between layers.

razvanc92
  • 1,158
  • 1
  • 9
  • 18