pokerdf

Converts poker hand history files into structured Pandas DataFrames, making it easier to analyze your games.

Fast and reliable, pokerdf is able to convert 20,000 hand history files, or 450,000 hands, into .parquet per minute, in a MacBook Air M4 with 10-core CPU. The modeling command then builds the star schema at 5 million player events per minute.

Currently supports PokerStars. Make sure hand histories are saved in English.

Introduction

Converting raw hand histories into structured data is the first step toward building a solid poker strategy and maximizing ROI. What are the optimal VPIP, PFR, and C-BET frequencies for No Limit Hold'em 6-Max? In which specific situations is a 3-Bet most profitable? When is bluffing a clear mistake? Once your data is organized in a Pandas DataFrame, the analytical explorations become unlimited, opening new possibilities to fine-tune your decision-making.

In the processed DataFrame, each row corresponds to a specific player in a specific hand, containing all relevant information about that instance of the game. Below, you’ll find an example of hand history before and after processing.

Before

PokerStars Hand #219372022626: Tournament #3026510091, $1.84+$0.16 USD Hold'em No Limit - Level I (10/20) - 2020/10/14 10:33:59 BRT [2020/10/14 9:33:59 ET]
Table '3026510091 1' 3-max Seat #1 is the button
Seat 1: VillainA (500 in chips) 
Seat 2: garciamurilo (500 in chips) 
Seat 3: VillainB (500 in chips) 
garciamurilo: posts small blind 10
VillainB: posts big blind 20
*** HOLE CARDS ***
Dealt to garciamurilo [6h Ks]
VillainB is disconnected 
VillainA: folds 
garciamurilo: calls 10
VillainB: checks 
*** FLOP *** [4d Qs Qd]
garciamurilo: checks 
VillainB: checks 
*** TURN *** [4d Qs Qd] [3s]
garciamurilo: checks 
VillainB: bets 20
garciamurilo: folds 
Uncalled bet (20) returned to VillainB
VillainB collected 40 from pot
VillainB: doesn't show hand 
*** SUMMARY ***
Total pot 40 | Rake 0 
Board [4d Qs Qd 3s]
Seat 1: VillainA (button) folded before Flop (didn't bet)
Seat 2: garciamurilo (small blind) folded on the Turn
Seat 3: VillainB (big blind) collected (40)

After

Installation

pip install pokerdf

Usage

First, navigate to the directory where you want to save the output:

cd output_directory

Then, run the package to convert all your hand history files:

pokerdf convert /path/to/handhistory/folder

After the process completes, you’ll see an output similar to the following:

output_directory/
└── output/
   └── 20250510-105423/
      ├── 20200607-T2928873630.parquet
      ├── 20200607-T2928880893.parquet
      ├── 20200607-T2928925240.parquet
      ├── 20200607-T2928950825.parquet
      ├── 20200607-T2928996127.parquet
      ├── 20200607-T2929005994.parquet
      ├── ...
      ├── fail.txt
      └── success.txt

Details

1. Inside output you’ll find a subfolder named with the session ID, in this case, 20250510-105423, containing all .parquet files.
2. Each hand history file is converted into a .parquet file with the exact same structure, allowing you to concatenate them seamlessly.
3. Each .parquet file follows the naming convention {DATE_OF_TOURNAMENT}-T{TOURNAMENT_ID}.parquet.
4. The file fail.txt provides detailed information about any files that failed to process. This file is only generated if there are failures.
5. The file success.txt lists all successfully converted files.

Incremental pipeline

You may want to build a pipeline to incrementally feed your table with new hand history data. In that case, you can import the convert_txt_to_tabular_data function and use it in your workflows. Refer to the docstrings and explore its usage within the package to better understand how it works.

Metadata

Every moment in the output is expressed in CET, as a fixed UTC+1. PokerStars writes each hand with the time of the platform in Eastern Time — next to your own local time, or alone when your client wrote no local time — so Eastern Time is the one anchor present in every hand, and it is the one the converter reads. It is then brought to CET honouring the daylight saving of the United States, so the result is a real instant; the offset of the output itself never shifts, which means the difference between two timestamps is always the time that actually elapsed.

The clock of the player is kept too, in HandStartTimeLocal and HandTimezone: analysing fatigue or playing habits needs the hour the player was actually looking at, and that is lost the moment it is discarded. It is null when the client wrote no local time, since nothing else in the file reveals where the player was. Read the abbreviation as provenance only, never as an offset: the same one means different things in different countries (CST is UTC-6 in the United States and UTC+8 in China). The offset of the player is the distance between the local moment and the one in CET, which is unambiguous.

Data Modeling

For advanced analytics, you will need to transform the data generated with the package and explore different data models. The final structure of your data may vary depending on the specific goals of your project. You will find below a suggestion of dimensional model (star schema) split into four tables that may be useful for most cases: fact_player_action works as the fact table, holding one row per event of a player in a hand, while dim_tournament, dim_hand and dim_final_rank work as dimension tables.

The reasoning behind this design:

• The fact holds the events; dim_hand holds their constant frame. Every row of the fact describes one event — who acted, from which seat and position, with which stack, facing which board, holding which cards and with which outcome ahead. The context that never changes inside a hand (the moment it was dealt, in CET and on the clock of the player, plus table size, players dealt in, level, blinds and ante) lives in dim_hand, keyed by Owner + TournID + HandID, keeping the fact lean.
• Posts are events, not metadata. The ante and blind posts are rows like any action, carrying the real (possibly partial, when all-in) amounts. This makes the pot a pure running sum, gives a row to players that never acted voluntarily (a big blind winning a walk, an all-in on the post), and lets the dynamic Stack be reconstructed uniformly.
• Each dimension answers one question at one grain. dim_tournament describes the tournament as observed by an owner (Owner + TournID, since each owner enters it at a moment of their own); dim_hand holds the constant context of each hand (Owner + TournID + HandID, since two archives can log the same hand, each with its own local timestamp); dim_final_rank holds the outcome of each player in the tournament. There is no player-hand dimension on purpose: the outcome of a player in a hand and the cards revealed at showdown are future events of the hand, and they belong on the rows of the fact.
• The outcome of the hand is registered on every row of its player. Result (folded, lost, mucked, non-sd win, won), Balance (the amount collected from the pot) and RevealedShowDownPokerHand (the combination the platform itself named at showdown) are future events recorded early, exactly like the revealed cards: knowing how the hand ended is what makes the behavior that led there analyzable.
• The reconstructed amounts follow the platform's own arithmetic (bet levels, short all-in blinds, calls above a short post) and were validated against the raw logs: the final TotalPot matches the reported "Total pot" in 100% of 135k+ real hands.
• The known holdings are inferred on every row, at the moment of the row. OwnerC1..C2 and OwnerCombination are hand context like the board: they fill every row of the hand, so any behavior can be analyzed against the owner's holding without joins. RevealedShowDownC1..C2 and RevealedShowDownCombination register, on every row of a player that revealed cards at showdown — the owner included — what the show at the end proved the player was holding: a future event recorded early, which is what makes the behavior that led to it analyzable. The combinations name the best hand each holding makes with the board visible at that moment, each with an integer score from 1 (High Card) to 10 (Royal Flush), ready for aggregation. The evaluator was validated against the platform's own showdown labels: 100% agreement over 70k+ real showdowns.

You can generate these four tables automatically with the modeling command, pointing to a folder of .parquet files produced by the convert command:

pokerdf modeling /path/to/parquet/files

The command concatenates all files and saves the four tables as .parquet inside ./modeling/{SESSION_ID}/.

Sharing the data: --gdpr

A hand history is not only about you. Under the European General Data Protection Regulation, the nicknames of the other players are personal data — online identifiers of natural persons, in the sense of Article 4(1) and Recital 30 — even though the files came from your own client. The tournament and hand identifiers link every row back to the platform records, and the timestamps allow a hand to be matched against publicly available tournament results. Before sharing a dataset, or moving it outside your own machine, run:

pokerdf modeling /path/to/parquet/files --gdpr full

Two GDPR principles guide what the command does:

• Data minimisation (Article 5(1)(c)) — what identifies without helping the analysis is not produced. The final-rank dimension is left out entirely: the nickname, final rank and prize of every player mirror publicly available tournament results, the easiest re-identification path there is. The fact, dim_tournament and dim_hand are generated, with two things taken out of them: every time column (HandStartTimeCET, HandStartTimeLocal, TournFirstHandTimeCET, TournFirstHandTimeLocal), because a timestamp matched against publicly available tournament schedules identifies the tournament; and the time zones (HandTimezone, TournTimezone), because a time zone is a rough statement of where the owner lives. Every other attribute leaves whole.
• Pseudonymisation (Article 4(5)) — TournID, HandID, Player and Owner are replaced by salted BLAKE2b digests in every generated table, with the same salt, so the tables keep joining (and the owner receives the same pseudonym in Owner and in Player). The same nickname always maps to the same pseudonym, so grouping keeps working, but nothing points back to a person or to a hand that can be looked up on the platform.

Everything that makes the data worth analyzing is preserved: the order of the actions, the amounts, the pot, the stacks, the board, the positions — and the cards. Your own hole cards (OwnerC1, OwnerC2), the cards revealed at showdown (RevealedShowDownC1, RevealedShowDownC2) and the combinations derived from them are kept in every mode, since the decisions in the dataset can only be studied against the holdings they were made with, and cards shown at the table describe the game, not a person.

Two modes are available:

By default the salt is random and never stored. Recital 26 draws the line between anonymous and personal data at whether re-identification is reasonably likely — and without the salt, a nickname cannot be confirmed by hashing a guess, so the pseudonyms are irreversible. To append new sessions to an existing dataset, the pseudonyms must be stable across runs, so inform your own salt:

pokerdf modeling /path/to/parquet/files --gdpr full --salt your-secret

With a kept salt the result remains pseudonymized personal data in the sense of the GDPR, not anonymous data: protect the salt as carefully as the original files, since whoever holds it can confirm a nickname by hashing it.

Each session writes an anonymization.txt report next to the data, describing what was applied and which risks remain — the most relevant being that the owner plays in every hand of their own archive, so even in full mode the pseudonym present in all rows is the owner, linked to the holdings it played; and that a hand remains described by its board and its exact bet sequence, which is close to unique for anyone holding another copy of it. None of this replaces assessing, for your own case, whether sharing the data is lawful.

fact_player_action

One row per event of a player: the ante and blind posts open each hand as rows (with the real — possibly partial — amounts that left each stack), followed by every action, sorted exactly as the hand unfolded: rounds in chronological order, starting from the first seat to act (the seat after the big blind on preflop, the seat after the button postflop). The amounts are reconstructed by replaying each round with the betting rules of the game, so they reflect the chips that actually moved.

dim_tournament

One row per tournament per owner — the key is Owner plus TournID: the dimension describes the tournament as observed by an owner's archive.

The moments are those of the first hand the owner's client logged, which is not necessarily when the tournament started: a player who registers late enters an event already running. The hand histories carry no other signal of the real start — the platform writes it in the tournament summary files, which this package does not read — so the columns are named after what they actually hold.

dim_hand

One row per hand of each owner's archive, holding the context that is constant across the events of the hand — the key is Owner + TournID + HandID, since two archives can log the same hand, each with its own local timestamp.

dim_final_rank

One row per player in each tournament of each owner's archive — the key is Owner + TournID + Player: the ranks are as observed by the owner, and an owner eliminated early does not see everyone's final rank.

License

MIT Licence