pipzap 0.5.0

Dependency optimization and pruning made simple

⚡️ PipZap ⚡

Zap the mess out of the dependency jungles

Overview

PipZap is a command-line tool designed to optimize Python dependency management by pruning redundant dependencies, and easing the transition to the modern formats.

It processes a dependency file, such as requirements.txt, uv's or Poetry's pyproject.toml, and outputs a minimal list of direct dependencies, removing those satisfied transitively.

Leveraging uv for dependency resolution, PipZap ensures your dependency list is lean and functional, making it easier to maintain and integrate into modern workflows.

Why PipZap?

Python dependency management can be a mess, especially with scientific libraries where outdated practices persist. Projects (even recent ones) often ship with a requirements.txt generated by pip freeze, leaving you with a sprawling list of pinned dependencies, many of which are redundant.

Take the easy_ViTPose project as an example: its requirements.txt lists 50 packages, including transitive dependencies like certifi and numpy.

See below for a real example with easy_VitPose

PipZap can transform this into a concise, modern format such as poetry, uv, or even a good old requirements.txt with just the essential direct dependencies. This makes it much easier to adopt any package into a modern codebase.

Moreover, even in well-maintained codebases, accidental over-specification creeps in, and PipZap cuts through that noise, keeping your dependency list clean and intentional.

Features

• Dependency Pruning: Eliminates redundant dependencies satisfied transitively.
• Format Auto-Detection: Automatically recognizes requirements.txt, uv, Poetry, and conda environment files.
• Flexible Output: Outputs in requirements, poetry, uv, or conda formats.
• Python Version Handling: Extracts from pyproject.toml or accepts via CLI for requirements.txt.
• Isolated Resolution: Resolves dependencies in a temporary, isolated environment to avoid impacting your project.
• Dependency Discovery: Scan source files with --discover to find actually used packages.
• Preserve All Mode: Use --preserve-all to ensure no transitive dependencies are lost after pruning.
• Verbose Logging: Optional rich logging for detailed insights.

Installation

Install PipZap with pip:

pip install pipzap

Usage

Eliminate redundant dependencies from a file:

pipzap requirements.txt -p 3.11
pipzap pyproject.toml
pipzap pyproject.toml -o pruned.toml -f poetry -v

• Use -p/--python-version to specify (recommended; defaults to the current environment for requirements.txt).
• Python version is auto-detected from pyproject.toml when present.
• Output format defaults to the input format unless specified with -f/--format (e.g., -f requirements).
• Use -d/--discover to scan Python source files and keep only packages that are actually imported.
• Use --preserve-all to re-verify the pruned output and add back any dependencies that would be missing.

Supported Formats

• requirements.txt: Pip-style with --extra-index-url support.
• UV pyproject.toml: Parses [project.dependencies] and [project.requires-python].
• Poetry pyproject.toml: Handles [project.dependencies] (modern) and [tool.poetry.dependencies] (legacy).
• Conda environment.yml: Extracts and processes pip dependencies, preserving conda-specific sections.

How It Works

1. Parsing: Identifies the file format and extracts direct dependencies.
2. Resolution: Uses uv to build the dependency graph in a temporary, isolated pyproject.toml.
3. Pruning: Identifies and removes transitive redundancies.
4. Formatting: Outputs the pruned dependencies in the specified format (requirements, poetry, or uv).

Examples

Pruning requirements.txt from easy_ViTPose

Input (requirements.txt from easy_ViTPose):

certifi==2023.7.22
charset-normalizer==3.2.0
coloredlogs==15.0.1
contourpy==1.1.1
cycler==0.11.0
ffmpeg==1.4
filelock==3.12.4
filterpy==1.4.5
flatbuffers==23.5.26
fonttools==4.43.0
humanfriendly==10.0
idna==3.4
imageio==2.31.3
importlib-resources==6.1.0
jinja2>=3.1.3
kiwisolver==1.4.5
lazy_loader==0.3
MarkupSafe==2.1.3
matplotlib==3.8.0
mpmath==1.3.0
networkx==3.1
numpy==1.26.0
onnx==1.14.1
onnxruntime==1.16.0
opencv-python==4.8.0.76
packaging==23.1
pandas==2.1.1
Pillow>=10.2.0
protobuf==4.24.3
psutil==5.9.5
py-cpuinfo==9.0.0
pycocotools==2.0.8
pyparsing==3.1.1
python-dateutil==2.8.2
pytz==2023.3.post1
PyWavelets==1.4.1
PyYAML==6.0.1
requests==2.31.0
scikit-image==0.21.0
scipy==1.11.2
seaborn==0.12.2
six==1.16.0
sympy==1.12
tifffile==2023.9.18
tqdm==4.66.1
typing_extensions==4.8.0
tzdata==2023.3
ultralytics==8.2.48
urllib3>=2.0.7
zipp==3.17.0

Command:

pipzap requirements.txt -p 3.11 -f poetry

Output (pruned to direct dependencies):

[tool.poetry.dependencies]
ffmpeg = "1.4"
filterpy = "1.4.5"
importlib-resources = "6.1.0"
lazy_loader = "0.3"
onnx = "1.14.1"
onnxruntime = "1.16.0"
pycocotools = "2.0.8"
scikit-image = "0.21.0"
typing_extensions = "4.8.0"
ultralytics = "8.2.48"
zipp = "3.17.0"

Contributing

Contributions are welcome! To get started:

1. Fork the repository.
2. Install dev dependencies: pip install -e .[dev].
3. Run tests: pytest.
4. Submit a pull request. Follow the Ruff linting rules and ensure type safety with mypy.

License

PipZap is licensed under the MIT License. See LICENSE for details.

Acknowledgments

• Powered by uv for dependency resolution.
• Born from the chaos of overgrown Python dependency lists.