Developing Games That Learn

I’ve been reading a book published back in 1996 (eons ago in the timeline of computer games) called Developing Developing Games That Learn (by Len Dorfman, Narendra K. Ghosh) in the hopes that it will have something I can use in my PS2 Dominoes Game.

There are few books around that actually discuss the topic of game AI with specific examples, and I do recommend this book to add to your collection. But, be aware that like many AI books, although it solves a specific problem, it may not be a solution for your particular situation.

One of the things that I really liked about the book was the authors’ attitude toward making both the games and the experience of developing the game enjoyable. For example…

“Even though we would never claim that utilizing the OAI paradigm in your program development cycle might imbue your program with sentient consciousness, will power, or self awareness, we strongly encourage you to “play pretend” and imagine that it does …”

“… So go wild! … and give a name to the portion of the code that makes decisions, responds, and demonstrates learning. Anthropomorphize to the max.

“In our case, we named Drop Four’s decision making and learning code Dr. Plopper. When Dr. Plopper followed a loss by altering her play so as to eventually facilitate a wily win, we’d shout, “Dr. Plopper’s Revenge!"… ("Fun is good!” - Dr. Seuss.)”

From the Introduction to Developing Developing Games That Learn Developing Games That Learn.

I’m about half way through it (it is an easy read) but unfortunately, I do not think there is much that I can use in this particular game. The AI learning technique demonstrated in the book is called Objective Artificial Intelligence (OAI), and it is nice idea that uses memory to “teach” the computer not to make the same mistake twice.

The basic idea is to think of the coded AI as “instinct". When the PC looses, it records the series of moves made that led to the loss (and generalizes the pattern for things like symmetry) and stores this to the disk. It would also record the resulting pattern of “wins". During the next match, before making a move, the program first does a quick check of memory to see if this pattern has occurred previously (either as a win or loss), and makes a move to prevent the loss pattern, or to repeat a winning pattern. If the pattern is not in memory, the program returns to the instinct to decide which move to make.

The book gives a set of questions to analyze whether or not the OAI technique is useful for your code (game or not). The examples in the book are for the games Tic-Tac-Toe and Connect-Four.

As I have said, I do not think that my Domino game will be well served by applying this technique. The problem is that the game of dominoes is different than the example games in a couple significant ways.

• Players in Tic-Tac-Toe and Connect-Four only have one type of move to make. The player either plays an “X or O” or a “Black or Red” chip. In Dominoes there are 28 different types tiles that the player might play.
• In the example games, the player is not limited by the number of pieces in their hand. They can play as many chips as they want until the game is over. This means that the code can be written to perform the same logic each time, only analyzing the patterns on the board. However, In Dominoes the choice and number of tiles is different every round, so the code has to look at both the game board AND the tiles in hand to determine if there is a repeated pattern.

I can think of a few limited example where a computer might learn not to make the same mistake twice, but it seems much more important (and much more simple) to have the AI just make sure that they are not “setting the player up” on the subsequent move.

Further, in Dominoes, the number of types of patterns is significantly higher. I think this would require playing the games hundreds of times before the AI was faced with a similar situation which it would have learned from.

It may be useful if Dominoes is thought of as a series of mini-games that are as important as the total game. I could envision an AI play logic that goes something like this based on the player’s chosen level of difficulty.

Easy: The AI places the first scoring tile in finds. If none are found, it places the tile with the highest value total value.

Medium: The AI places the tile that scores the highest points for that round. If no tile will score (or if multiple placements result in the same score), the AI places the tile with the highest point value.

Hard: Same as Medium, however the AI also checks the possible combinations of tiles that remain available for the other player to use … and if it would leave it open for the player to follow up with a significantly high score, the AI would not make the play unless necessary. (A weighted value is given to each possible move, and the AI chooses which move to make accordingly)

Additionally on the Hard setting, the AI should also “remember” if the player had the first move because of a double tile. If so, the AI knows that the player has the double tile and would use that when weighing the decision.

The point is… that the OAI technique just does not seem to be relevant to this game. Dominoes is much more like a card game, where each hand is unique and memorizing patterns of play is not nearly as useful as card counting and knowing the odds.