PyTools

py_tools is a research-oriented Python toolkit for numerical economics, time series, estimation, and dataset loading utilities.

Installation

Install from PyPI:

pip install dgreenwald-py-tools

Or clone the repository and install in editable mode:

pip install -e .

Optional extras:

pip install "dgreenwald-py-tools[ml]"        # scikit-learn, patsy
pip install "dgreenwald-py-tools[datasets]"  # pandas-datareader, python-dotenv
pip install "dgreenwald-py-tools[scraping]"  # requests, beautifulsoup4, lxml
pip install "dgreenwald-py-tools[nlp]"       # nltk
pip install "dgreenwald-py-tools[mpi]"       # mpi4py
pip install "dgreenwald-py-tools[dev]"       # pytest, ruff

If you use dataset loaders, set:

export PY_TOOLS_DATA_DIR=/path/to/data

Or create a .env file in the repository root (or a parent directory):

PY_TOOLS_DATA_DIR=/path/to/data

Slurm jobs

py_tools.cluster renders typed single-job and array definitions and can submit the resulting scripts with sbatch --parsable:

from py_tools.cluster import (
    SLURM_ARRAY_TASK_ID,
    SlurmArray,
    SlurmJob,
    SlurmResources,
    submit_slurm,
    write_slurm_script,
)

job = SlurmJob(
    name="analysis",
    command=("python", "worker.py", "--index", SLURM_ARRAY_TASK_ID),
    workdir="/cluster/project",
    log_dir="/cluster/project/output/slurm",
    resources=SlurmResources(time="04:00:00", memory="16G", account="research"),
    activate="/cluster/venv/bin/activate",
    array=SlurmArray(task_count=20, max_concurrent=4),
)
script = write_slurm_script(job, "/cluster/project/output/slurm/jobs.slurm")
submission = submit_slurm(script)
print(submission.job_id)

py_tools.datasets will load .env automatically when python-dotenv is installed (included in the datasets extra).

Quick Start

from py_tools import data, time_series, state_space

# Example: call a utility function from a core module
# (see module docstrings/source for full APIs)

You can also import dataset loaders through:

from py_tools import datasets

available = datasets.list_datasets()
df = datasets.load_dataset("fred", codes=["UNRATE"])

Module Overview

Development

pip install -e ".[dev]"
python -m pytest          # run tests
ruff check .              # lint

Contributing

• Keep changes focused and commit one logical change at a time.
• Follow repository coding and workflow guidelines in AGENTS.md.
• Prefer adding deterministic tests in tests/ for new behavior.

License

MIT — see LICENSE.