pokerl 0.0.4

RL environments for poker games

PokeRL

pokerl is a poker Reinforcement Learning (RL) environment.

Currently the only game supported is No Limit Texas Hold'em.

Usage

agent@pokerl:~$ pip install pokerl

A game is created by creating a new instance of the Game class:

from pokerl import Game
game = Game()

The constructor accepts various parameters, such as the number of players and the initial credits:

game = Game(num_players=4, start_credits=100)
game.reset() # Init game state

At each step, there is an active player and an active state, which captures the state of the game as seen by the active player. The active player performs actions by passing a valid integer value to game.step(action):

from pokerl.enums import PokerMoves
game.step(PokerMoves.CALL)
game.step(PokerMoves.FOLD)
game.step(PokerMoves.ALL_IN)

The type of action must be int or np.integer

The game.active_state property is an object of type Game.StateView that has various informations that can be used by the agent to make informed decisions. In particular, valid_actions is a one-hot encoded np.ndarray of valid actions:

# Choose random action
valids = game.active_state.valid_actions
action = np.random.choice(len(valids), p=valids / np.sum(valids))
game.step(action)

If an invalid action is passed to game.step (e.g. if you try to call a bet but you don't have enough credit) an AssertionError is thrown.

game.step also returns a 3-element tuple indicating whether the game is over, the hand is over and/or the round of bets is over:

done, _, _ = game.step(action)
if done: print('game is over')

After each hand, the net profit of each player is saved in game.payoffs.

For more info see the Wiki page.

Environments

If you are familiar with OpenAI Gym, the racerl.envs environments expose an API similar to that of gym.Env. The PokerGameEnv simulates an entire game and returns a non-zero reward after each hand, depending on the player's payoffs and bets. To use the PokerGameEnv create a new instance by passing a list of opponents:

from pokerl import Game
from pokerl.envs import PokerGameEnv

def random_agent(state: Game.StateView) -> int:

	actions = state.valid_action_indices
	return np.random.choice(list(actions))

env = PokerGameEnv([random_agent, random_agent, random_agent], num_players=4)

An opponent must be a callable object which receives the game state as seen by the agent and returns the action to perform. The usual methods env.reset() and env.step(action) can be used to reset the state and perform actions on the environment:

# Evaluate agent on one run

R = []
state = env.reset()
done = False
while not done:
	action = predict(state)
	state, rwd, done = env.step(action)
	R.append(rwd)

print('score', np.sum(R))

The reward is the sum of the payoffs.

Network

The network branch is an experimental branch where it is possible to play online games. Many functionalities, however, are also available in the main branch.

To create a server, create a new instance of the pokerl.network.PokerGameServer class:

from pokerl.network import PokerGameServer
server = PokerGameServer(num_players=3)

The constructor accepts the same parameters of Game. To start the server and play a game, use server.setup(host, port) followed by server.run():

server.setup('localhost', 25560)
#server.setup() Listen on default host and port
server.run()

A client is an instance of pokerl.network.PokerGameClient. The contructor accepts a strategy policy agent, which ideally is a subclass of pokerl.agents.PokerAgent:

from pokerl.agents import RandomAgent
client = PokerGameClient(agent=RandomAgent)

To connect to a server, use client.connect(host, port):

client.connect('localhost', 25560)
client.connect() # Use default host and port

Once all the players have connected, the game begins.

At the moment, it is not possible to follow the actions of other clients.

Contributors

• Andrea Mecchia @ sneppy
• Guglielmo Manneschi @ nondecidibile