powerlawrs 0.0.14

A fast package for analyzing Power-Law distributions in empirical data written in Rust.

powerlawrs

powerlawrs is a Python package for analyzing power-law distributions in empirical data. It is built on a high-performance Rust crate powerlaw, providing both speed and ease of use for Python users. The methodology is heavily based on the techniques and statistical framework described in the paper 'Power-Law Distributions in Empirical Data' by Aaron Clauset, Cosma Rohilla Shalizi, and M. E. J. Newman.

Features

• Parameter Estimation: Estimates the parameters (x_min, alpha) of a power-law distribution from data.
• Goodness-of-Fit: Uses the Kolmogorov-Smirnov (KS) statistic to find the best-fitting parameters.
• Data Visualization: Includes a plot() method to visually inspect the data and the fitted model on a log-log scale.
• Vuongs Closeness Test: Model selection by comparing vectors of Log-Likelihoods from two distributions.
• Additional Distributions: Provides functionality for other distributions, such as the exponential distribution.
• High Performance: Computationally intensive tasks are parallelized in the Rust core for significant speedups.
• Flexible API: Offers both a simple functional API for quick analyses and a class-based API for more detailed work.

Installation

Prerequisites

• Python 3.8+
• Rust (the package is built from Rust source)
• uv (this project uses uv for environment and package management)

Setup and Installation via pip

1. Create and activate a virtual environment:

   # Create the environment
   uv venv powerlaw
   
   # Activate the environment
   source powerlaw/bin/activate
   

2. Install the package.

   uv pip install powerlawrs
   

Setup and Installation from Source

1. Clone the repository:

   git clone https://github.com/aulichny3/powerlawrs.git
   cd powerlawrs
   

2. Create and activate a virtual environment: This project is configured to use the powerlaw virtual environment with uv.

   # Create the environment
   uv venv -p powerlaw
   
   # Activate the environment
   source powerlaw/bin/activate
   

3. Install the package: To install the package in editable mode and include all development dependencies, run:

   # Install maturin
   uv tool install maturin
   
   # Install the package using maturin
   uv tool run maturin develop --uv
   
   # Install development dependencies
   uv pip install -r requirements.txt
   

   This installs powerlawrs in editable mode, so any changes you make to the source code will be immediately available.

Dependencies

• numpy
• matplotlib

Development dependencies (for running the example notebooks) are listed in [project.optional-dependencies].dev in pyproject.toml.

Usage

The powerlawrs package offers two primary ways to analyze your data: a simple functional API and a more detailed class-based API.

Functional API (Recommended)

The powerlawrs.fit() function is the most straightforward way to fit a power-law distribution to your data. See the Quickstart notebook for an example.

import powerlawrs
import polars as pl

# 1. Load your data into a list, Polars Series, or numpy array 
# The data should be a 1-dimensional array of numbers.
data = pl.read_csv("reference_data/blackouts.txt", has_header=True).to_series()

# 2. Fit the data:
p = powerlawrs.fit(data)

# 3. Print the ParetoFit object results:
print(p.ParetoFit)

Visualizing the Fit

After fitting the data, you can use the plot() method to visually inspect the fit.

# Assuming 'data' is loaded and fitted as above
p.plot()

This will generate two plots: one showing the CCDF of the full data with the scaled model, and another showing just the tail of the distribution.

Working with Other Distributions

powerlawrs also provides tools for other common distributions.

import powerlawrs

# Analyze a shifted exponential distribution
data = [1.2, 1.5, 1.9, 2.3, 2.8, 3.1, 3.5]
x_min = 1.0

# Estimate the lambda parameter
lambda_hat = powerlawrs.exponential.estimation.lambda_hat(data, x_min)
print(f"Estimated Lambda: {lambda_hat}")

# Create an exponential distribution object
exp_dist = powerlawrs.exponential.Exponential(lambda_hat, x_min)
print(f"PDF at x=2.0: {exp_dist.pdf(2.0)}")

Class-based API

Module Hierarchy

The powerlawrs package is structured hierarchically to organize its functionality logically and provide a clean API. The main submodules, accessible directly from import powerlawrs, are:

• powerlawrs.dist: Contains implementations of various probability distributions (e.g., Pareto, Lognormal, Exponential) for fitting and analysis.
• powerlawrs.stats: Provides statistical functions, including descriptive statistics, random number generation tools, and goodness-of-fit tests like the Kolmogorov-Smirnov (KS) test.
• powerlawrs.util: Offers utility functions, such as data loading, parameter calculation for simulations, and synthetic data generation.

This structure allows for clear separation of concerns and easier navigation of the library's features. For example, to access the KS test functions, you would typically import from powerlawrs.stats.ks.

For more fine-grained control, you can see the API examples in Notebooks/02 - API.ipynb.

Jupyter Notebook Examples

The `Notebooks folders provides a detailed walkthrough of the package's functionalities. After installing the development dependencies, you can run it with:

# Make sure your virtual environment is active
source powerlaw/bin/activate

# Start Jupyter Lab
uv run --active jupyter lab

Limitations

1. Only the continuous case of the Pareto Type I Distribution is considered for parameter estimation, goodness of fit, and hypothesis testing at this time. The example data in the documentation is discrete, thus the results are only an approximation.
2. Domain knowledge of the data generating process is critical given the methodology used by this package is based on that proposed by the referenced material. Specifically the 1-sample Kolmogorov-Smirnov test is used for goodness of fit testing which assumes i.i.d data. Many natural processes data are serially correlated, thus KS testing is not appropriate.
3. This is highly alpha code; backwards compatibility is not guaranteed and should not be expected.
4. Many more known and unknown.

License

This project is licensed under either of:

• Apache License, Version 2.0, (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.