Can AlphaZero considered as Multi-Agent Deep Reinforcement Learning?

Question

Can AlphaZero considered as Multi-Agent Deep Reinforcement Learning?

I could not find a clear answer on this. I would say yes it is Multi Agent Learning, as there are two Agents playing against each other.

Answer 1

Depends on perspective.

On one hand, you have an agent playing in an environment with another agent also evolving. This falls under the definition of Multi-Agent Learning, as can be seen with works such as

• Michael Bowling and Manuela Veloso. Multiagent learning using a variable learning rate. Artificial Intelligence, 136(2):215 – 250, 2002.

• Michael Bowling. Convergence and no-regret in multiagent learning. In Proceedings of the 17th International Conference on Neural Information Processing Systems, NIPS’04, pages 209–216, Cambridge, MA, USA, 2004. MIT Press.

• M. D. Awheda and H. M. Schwartz. Exponential moving average q-learning algorithm. In 2013 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), pages 31–38, April 2013.

• Sherief Abdallah and Victor Lesser. A multiagent reinforcement learning algorithm with non-linear dynamics. Journal of Artificial Intelligence Research, 33:521–549, 2008.

However, you can also claim that you simply have a single agent learning on a non-stationary environment (the environment contains both the game rules and the opponent), and you simply learn on that basis. From this perspective, there is no multi-agent learning at all.