I'm attempting to make a bot for the Connect 4 competition on http://riddles.io
My bot isn't horrible, like it's getting up the ladder, but it cannot compete with the winning bots.
I'm using a Neural Network which is fully connected with one hidden layer. Internally it uses the sigmoid function as the activator in each Neuron. I've trained it over 500,000 games with TD-Lambda back propagation. The alpha and beta values (i.e. learning rates) are set to 0.1 each, and the lambda for the eligibility trace is set to 0.7. There are 2 outputs nodes, one to give the value of this position for Player 1, the other to give the value of the position for Player 2. Upon a win, these are back-propagated with -1 for a loss and 1 for a win, on a draw they are both back-propagated with a 0.
There is a bias input and weight for every neuron as well.
All of the weights are initialised to a random value between +-4*sqrt(6/totalNumberWeights).
The board state is represented to the network as:
- For each space in the board, 2 values:
- For the first value if player 1 occupies this space it's a
1, otherwise it's a0 - For the second value if player 2 occupies this space it's a
1, otherwise it's a0 - If both are
0it would mean it's a free spot
- For the first value if player 1 occupies this space it's a
- For each space on the board another 2 values:
- For the first value, if placing a token here would result in a Connect 4 for player 1, it's a
1, otherwise a0 - For the second value, if placing a token would get player 2 a Connect 4, then it's a
1otherwise a0 - So if no one would win by placing a token here, it's two
0values for these two inputs
- For the first value, if placing a token here would result in a Connect 4 for player 1, it's a
- Two final values, the first indicating whether it's player 1's turn or not, then second indicating if it's player 2's turn or not
What I see when my bot is playing is that it makes what I assume are somewhat clever moves, like it's preparing for both horizontal and diagonal moves.
But when the other bot will get a Connect 4 on the next move, my bot fails to place a token there. The best bot in the comp seems capable of setting itself up to get three in a row with a free space on both sides, so that it will definitely get a Connect 4. Again, my bot does not seem to be able to see this coming.
What I think the issue is, other than potentially my learning rates, is that I've represented the board in a bad way. Is there a better way to represent it to the Network so that it can more accurately estimate the value of a board state, and so that it doesn't fail to identify an immediate Connect 4 threat?
