pipreq-distill 0.1.0

Resolve pip dependency conflicts by intelligently removing problematic packages

pipreq-distill

Resolve pip dependency conflicts by intelligently removing problematic packages.

When pip install fails with dependency conflicts, pipreq-distill queries PyPI to find the largest possible collection of compatible packages by removing the oldest and most problematic dependencies.

Installation

pip install pipreq-distill

Or with uv:

uv pip install pipreq-distill

Usage

Basic Usage

# From requirements.txt to resolved.txt
pipreq-distill --input requirements.txt --output resolved.txt

# From stdin to stdout
cat requirements.txt | pipreq-distill > resolved.txt

# Dry run (show what would be removed)
pipreq-distill --input requirements.txt --dry-run

pyproject.toml Support

# Process pyproject.toml dependencies
pipreq-distill --input pyproject.toml --output pyproject.toml --format pyproject

# Process optional dependencies
pipreq-distill --input pyproject.toml --group optional.dev --dry-run

# Update in-place
pipreq-distill --input pyproject.toml --output pyproject.toml

Protect Specific Packages

# Never remove requests or flask
pipreq-distill --input requirements.txt --keep "requests,flask"

JSON Output

# Get structured output for scripting
pipreq-distill --input requirements.txt --json-output

Output:

{
  "kept": ["flask==2.3.0", "requests==2.31.0"],
  "removed": [
    {"name": "old-pkg", "version": "1.0.0", "score": 150.5}
  ],
  "kept_count": 2,
  "removed_count": 1
}

How It Works

1. Fetches package metadata from PyPI (with caching)
2. Extracts dependencies from wheels or source distributions
3. Detects version conflicts between packages
4. Scores packages for removal based on:
   • Conflict involvement (packages in more conflicts score higher)
   • Reverse dependencies (packages with fewer dependents score higher)
   • Package age (older packages score higher)
   • Dependency age (packages depending on old packages score higher)
5. Iteratively removes the highest-scoring package until no conflicts remain
6. Outputs the largest possible set of compatible packages

CLI Options

Option	Description	Default
--input	Input file (requirements.txt or pyproject.toml)	stdin
--output	Output file	stdout
--format	Format: auto, requirements, pyproject	auto
--group	Dependency group for pyproject.toml	dependencies
--keep	Comma-separated packages to protect	None
--dry-run	Show what would be removed	False
--json-output	Output as JSON	False
--verbose	Show detailed progress	False
--timeout	PyPI request timeout (seconds)	30
--max-concurrent	Max concurrent PyPI requests	20

Example

Given conflicting requirements:

aliyun-python-sdk-core>=2.16.0
jmespath>=1.0.1

Running:

$ pipreq-distill --input requirements.txt --verbose

Parsed 2 requirements
Fetching 2 packages in parallel...
Fetched 2 packages successfully
Iteration 1: 1 conflicts in 1 cluster(s)
Removing aliyun-python-sdk-core (score: 641.4)
Resolved to 1 compatible packages

jmespath==1.0.1

The tool identified that aliyun-python-sdk-core requires jmespath<1.0.0, which conflicts with jmespath>=1.0.1. Since aliyun-python-sdk-core is older and has the conflicting requirement, it was removed.

Dependency Groups

For pyproject.toml, you can specify different dependency groups:

• dependencies - PEP 621 project.dependencies (default)
• optional.dev - project.optional-dependencies.dev
• optional.test - project.optional-dependencies.test
• dev - tool.uv.dev-dependencies

Caching

PyPI responses are cached in $TMPDIR/pipreq_distill_cache for 24 hours to speed up repeated runs.

Limitations

• Assumes you want the latest compatible versions
• Conditional dependencies (platform-specific markers) are simplified
• Uses a greedy algorithm (not guaranteed globally optimal)
• May download wheels/sdists for packages missing JSON metadata

License

Apache-2.0