5

I am currently writing an engine to play a card game and I would like for an ANN to learn how to play the game. The game is currently playable, and I believe for this game a deep-recurrent-Q-network with a reinforcement learning approach is the way to go.

However, I don't know what type of layers I should use, I found some examples of Atari games solved through ANN, but their layers are CNN (convolutional), which are better for image processing. I don't have an image to feed the NN, only a state composed of a tensor with cards in the player's own hand and cards on the table. And the output of the NN should be a card or the action 'End Turn'.

I'm currently trying to use TensorFlow but I'm open to any library that can work with NN. Any type of help or suggestion would be greatly appreciated!

Paulo Neves
  • 59
  • 4

2 Answers2

2

The game state consists of the location of all the hidden cards, so you probably need a softmax layer, 52*n, where n is the number of locations.

I'm not very sure that a NN is a good match.

chrishmorris
  • 171
  • 2
1

With images, you can use CNN because of the translational invariance. A filter which is good in one area will probably be good in another area, too.

With images, you must use CNN because otherwise, there would be too many weights to train.

With your game, it depends on the representation and the exact rules. Note that Alpha Zero uses a set of 19 x 19 inputs with CNN for playing Go.

In a game like Bridge, where each card has its color and rank, there's a kind of translational invariance. Having Ace and Queen is a bit similar to having King and Jack - in both cases you have a 50% chance of catching the card in between. At the same time, the strengths of AQ and KJ are very different, so a pure CNN would improbably work well.

The more important symmetry is the one among colors. After the auction, there's one or none trump color and all other colors are equivalent. This probably means that the corresponding weights should be the same.

In some card games, many cards are special and there's no symmetry at all. You didn't tell us anything about your game, so it's hard to give a more concrete advice.

maaartinus
  • 563
  • 3
  • 9