Skip to main content

Hex board game AI with self-play learning based on the AlphaZero algorithm

Project description

Azalea

playing to learn to play

Azalea is a reinterpretation of the AlphaZero game AI learning algorithm for the Hex board game.

Features

  • Straightforward reimplementation of the AlphaZero algorithm except for MCTS parallelization (see below)
  • Pre-trained model for Hex board game
  • Fast MCTS implementation through Numba JIT acceleration.
  • Fast Hex game move generation implementation through Numba.
  • Parallelized self play to saturate Nvidia V100 GPU during training
  • AI policy evaluation through round robin tournament, also parallelized
  • Tested on Ubuntu 16.04
  • Requires Python 3.6 and PyTorch 0.4

Differences to published AlphaZero

  • Single GPU implementation only - tested on Nvidia V100, with 8 CPU's for move generation and MCTS, and 1 GPU for the policy network.
  • Only Hex game is implemented, though the code supports adding more games. Two components are needed for a new game: move generator and policy network, with board input and moves output adjusted to the new game.
  • MCTS simulations are not run in parallel threads, but instead, self-play games are played in parallel processes. This is to avoid the need for a multi-threaded MCTS implementation while still maintaining fast training speed and saturating the GPU.
  • MCTS simulation and board evaluations are batched according to search_batch_size config parameter. "Virtual loss" is used as in AlphaZero, to increase search diversity.

Installation

Clone the repository and install dependencies with Conda:

git clone https://github.com/jseppanen/azalea.git
conda env create -n azalea
source activate azalea

The default environment.yml installs GPU packages but you can choose environment-cpu.yml for testing on a laptop.

Playing against pretrained model

python play.py models/hex11-20180712-3362.policy.pth

This will load the model and start playing, asking for your move. The columns are labeled a–k and rows 1–11. The first player, playing X's, is trying to draw a vertical connected path through the board, while the second player, with O's, is drawing a horizontal path.

O O O O X . . . . . . 
 . . . . . . . . . . . 
  . . . . . . . . . . . 
   . . . . X . . . . . . 
    . . . . . X . . . . . 
     . . . . . . . . . . . 
      . . . . X . . . . . . 
       . . . . . . . . . . . 
        . . . X . . . . . . . 
  x      . . . . . . . . . . . 
 o\\      . . . . . . . . . . . 
last move: e1
Your move? 

Model training

python train.py --config config/hex11_train_config.yml --rundir runs/train

Model comparison

python compare.py --config config/hex11_eval_config.yml --rundir runs/compare <mode1> <model2> [model3] ...

Model selection

python tune.py

References

Project details


Release history Release notifications

This version

0.1.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for azalea, version 0.1.0
Filename, size File type Python version Upload date Hashes
Filename, size azalea-0.1.0-py3-none-any.whl (29.1 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size azalea-0.1.0.tar.gz (22.5 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page