8

I'm training both DQN and double DQN in the same environment, but DQN performs significantly better than double DQN. As I've seen in the double DQN paper, double DQN should perform better than DQN. Am I doing something wrong or is it possible?

nbro
  • 42,615
  • 12
  • 119
  • 217
Angelo
  • 211
  • 2
  • 17

2 Answers2

2

There is no thorough proof, theoretical or experimental that Double DQN is better then vanilla DQN. There are a lot of different tasks, paper and later experiments only explore some of them. What practitioner can take out of it is that on some tasks DDQN is better. That's the essence of Deep Mind's "Rainbow" approach - drop a lot of different methods into bucket and take best results.

mirror2image
  • 735
  • 7
  • 15
1

That may happen when the value of the state is bad. You can find the example and explain about that in the link below.

See this:https://medium.freecodecamp.org/improvements-in-deep-q-learning-dueling-double-dqn-prioritized-experience-replay-and-fixed-58b130cc5682

I_Al-thamary
  • 52
  • 1
  • 16