smm2sim

This package simulates the GSA Mario Maker 2 Endless Expert League regular season and playoffs using a simple, customizable Monte Carlo method.

Installation

The package is on [PyPI] and can be installed with pip:

pip install smm2sim

How it works

During each simulation, smm2sim uses the methods described below to assign a winner to all remaining matches in the season. It then calculates seasonal point totals and breaks ties to determine playoff seeding, and the playoffs are simulated match-by-match. The playoff structure is assumed to be single-elimination best-of-3 matches with no reseeding.

Before beginning the simulations, each player is assigned a power rating (PWR), such that a player with a PWR of 8 would be expected to score an average of 8 points in a 15 minute match. By default, the base power rankings for each player are a simple average of their past results (excluding points scored during untimed tiebreakers). Custom ranking systems are also supported, which can be combined with the default ratings or replace them entirely. The individual rating systems and the combined rankings can be regressed to the mean (or to custom player-specific values) as desired.

The player PWR rankings are adjusted at the beginning of each season simulation by a random amount, determined using a normal distribution with mean 0 and a user-provided standard deviation (1 point by default):

adjusted_pwr = [PWR] - numpy.random.normal(0, [rank_adj])

This adjustment represents the uncertainty in each player's base PWR projection, which includes both model error and potential player skill changes. Higher values equate to more variance in outcomes.

Each match consists of 3 simulated games. When simulating a game, player A's PWR is compared to player B's PWR. The resulting point differential is used to generate a normal cumulative distribution function, which estimates player A's probability of winning the game. This win probability is compared to a random number to determine the simulated winner of the game:

pwr_difference = [PWR A] - [PWR B]
win_probability = 1 - scipy.stats.norm(pwr_difference, [stdev]).cdf(0)
is_winner = numpy.random.random() < win_probability

The standard deviation used to generate the normal distribution ([2.5 points by default]) is configurable.

Usage

Basics

Each simulation is controlled by a Simulate object. You create an object by specifying the number of simulations:

import smm2sim as smm2
simulation = smm2.Simulate(n_sims=10000)

If desired, you can customize the values of the PWR rank adjustment used at the beginning of each simulation and the standard deviation used when simulating individual games:

simulation = smm2.Simulate(n_sims=10000, rank_adj=1, st_dev=2.5)

PWRsystems

You can customize how the power rankings are generated by creating a PWRsystems object. You create an object by indicating which systems to include; the built-in system is called "srs":

systems = smm2.PWRsystems(srs=True)
simulation = smm2.Simulate(n_sims=10000, pwr_systems=systems)

You can also use your own rating system by creating a generic PWR object and passing it a pandas DataFrame containing the custom rankings. The DataFrame must include one column called 'Player' containing the name of each player (case sensitive) and another column containing the rankings. The name of the ranking column should be unique from those of the other systems being used (so don't use "SRS"):

my_sys_df = pd.DataFrame([{'Player':'A','Power':7},{'Player':'B','Power':5}])
my_sys = smm2.PWR(values=my_sys_df)
systems = smm2.PWRsystems(others=my_sys)

You can also combine multiple systems. The weights for each system (default = 1) can be specified using the built-in objects for each system:

my_sys_df = pandas.DataFrame([{'Player':'A','Power':7},{'Player':'B','Power':5}])
my_sys = smm2.PWR(weight=1, values=my_sys_df)
systems = smm2.PWRsystems(srs=smm2.SRS(weight=2), others=my_sys)

To use multiple custom systems, pass a list of PWR objects instead of a single PWR object:

df1 = pd.DataFrame([{'Player':'A','Power1':7},{'Player':'B','Power1':5}])
df2 = pd.DataFrame([{'Player':'A','Power2':2},{'Player':'B','Power2':6}])
my_sys_1 = smm2.PWR(weight=2, values=df1)
my_sys_2 = smm2.PWR(weight=1.5, values=df2)
systems = smm2.PWRsystems(srs=True, others=[my_sys_1, my_sys_2])

Regression

Optionally, you can choose to regress the ratings generated by each system by creating a Regression object (if regress_to is omitted, no regression will be used). By default, PWR values will be regressed to the sample mean:

my_sys = smm2.SRS(weight=2, regress_to=smm2.Regression())

You can use fixed weighting by specifying a decimal between 0 and 1, or variable weighting based on the percentage of a specified number of games played (the default option):

#(PWR * 0.75) + (sample_mean * 0.25)
regression_fixed = smm2.Regression(weight=0.25)
#((PWR * games_played) + (sample_mean * max(0, 12 - games_played))) / max(12, games_played)
regression_variable = smm2.Regression(n_games=12)

You can regress PWR to a fixed value rather than using the sample mean:

regression = smm2.Regression(to=0, weight=0.5)

You can also specify a custom regression value for each player using a pandas DataFrame. The DataFrame must contain one column called 'Player' containing the player names (case sensitive) and another called 'Baseline' for the regression values:

df = pd.DataFrame([{'Player':'A','Baseline':5},{'Player':'B','Baseline':8}])
regression = smm2.Regression(to=df, n_games=33)

In addition to (or instead of) regressing the values for individual PWR systems, you can choose to regress the final results after combining the various systems:

regression = smm2.Regression(n_games=12)
systems = smm2.PWRsystems(regress_to=regression, srs=True, others=my_sys)

Execution and Analysis

Once you've set up your Simulate object, use run() to execute the simulation.

simulation = smm2.Simulate(n_sims=10000)
simulation.run()

The run() method will return a reference to the Simulate object, so this syntax is also acceptable:

simulation = smm2.Simulate(n_sims=10000).run()

By default, run() will use the joblib package to run the simulations in parallel; this can be overridden by setting parallel=False:

simulation = smm2.Simulate(n_sims=100).run(parallel=False)

Once the simulation has executed, the results are aggregated and stored in several related dataframes. These can either be directly accessed using the simulations property:

standings = sim.simulations.standings
regularseason = sim.simulations.regularseason
seeding = sim.simulations.seeding
playoffs = sim.simulations.playoffs

Or returned as copies using class methods:

standings = sim.standings()
regularseason = sim.regularseason()
seeding = sim.seeding()
playoffs = sim.playoffs()

By default, all of the aggregated dataframes use MultiIndexes incorporating the simulation number and the within-simulation row number. The class methods include an option to extract the MultiIndex into separate columns in the dataframe:

standings_reindexed = sim.standings(reindex=True)

You can also entirely disable the generation of aggregated statistics, in which case the results are stored as a list of Simulation objects:

sim = smm2.Simulate(n_sims=100000).run(combine=False)
for simulation in sim.simulations.values:
    rankings = simulation.rankings
    standings = simulation.standings
    regularseason = simulation.regularseason
    seeding = simulation.seeding
    playoffs = simulation.playoffs

[PyPI]: https://pypi.org/project/smm2sim/