The Distributed Chess Project

Web Hosting by Netfirms | Free Domain Names by Netfirms

THE DISTRIBUTED CHESS PROJECT

Creating Chess-Playing Artificial Neural Networks with Distributed Evolutionary Algorithms

Results and User Statistics as of 4/26/2003 (Archive)

Currently the project has 658 registered participants.

Since 06/23/2002 a cumulated computing time of 16 years 57 days 17 hours 32 min 28 sec has been contributed.

The following links lead to the result tables for two different problem sets, the first of which focuses on middlegame tactics while the second is endgame oriented.

Results on Tactical Problems

Results on Endgame Problems

Results on RrPp-Endgame Problems

Results on QqRrNnBb-Positions

... and what does it all mean? Interpretation of the Results

What I call the 'test set' consists of a number of chess problems with known best continuations. This set must be viewed in contrast to the 'training set', also consisting of a number of different problems. The test problems are presented to the net after the training is finished in order to see how well a neural net generalizes from the training problems. This is most important, since a neural net that does well on the training set but not on an independent test set isn't worth its money.

And this is the good news: There definitely is generalization to some degree in the results so far. During training, the neural nets do in fact capture some positional patterns, that enable them to find the correct continuations most of the time when confronted with the test problems.

Numerically, the results for the tactical problems are encouraging and rather impressive, because a neural net with e.g. 75% test performance picks the only correct move from a variety of 30 to 50 possible legal moves for 3 out of 4 problems it sees for the very first time. From the pessimist point of view, however, it still fails on 1 out of 4 problems. Moreover, since the typical training performance is higher, we don't have perfect generalization. Therefore it must be the goal to find neural nets with test performance as high as the training performance.

For the endgame problems the picture is not so clear. Since the overall performance is way lower than for the tactical problems, it seems that the endgame problems contain features that are not so easily captured by neural nets. But keep in mind, that in order to tell whether a neural net has learned some chess, one has to compare the observed neural net performance to the expected performance one would obtain if one was randomly guessing the best moves. This performance is inversely proportional to the average number of legal moves. For typical endgame positions with about 20 possible continuations this number is therefore just 5%. Thus, a neural net that performs consistently better than 5% on positions with 20 continuations, has in fact learned some chess. So, let's give it some more time, since the performance for the endgame problems is better than the expected random performance.

What you as active project participants can and should do at this point is fiddle around with the free parameters of both neural networks and genetic algorithm in order to find a configuration that maximizes the test performance. Any questions? Please check the documentation first, but you may contact me directly as well via Ralf.Seliger@t-online.de