Lightweight pip dependency resolver with deptree preview functionality based on the PubGrub algorithm
pipgrip is a lightweight pip dependency resolver with deptree preview functionality based on the PubGrub algorithm, which is also used by poetry. For one or more PEP 508 dependency specifications, pipgrip recursively fetches/builds the Python wheels necessary for version solving, and optionally renders the full resulting dependency tree.
pipgrip vs. poetry
poetry offers package management with dependency resolution, essentially replacing pip/setuptools. This means that poetry packages don't contain
setup.py, and hence are not compatible with
pip install -e: poetry projects would have to be converted to setuptools-based projects with e.g. dephell. To avoid such hassle, pipgrip only requires the selected package(s) + dependencies to be available to pip in the ususal way.
pipgrip vs. pipdeptree
For offline usage, pipdeptree can inspect the current environment and show how the currently installed packages relate to each other. This however requires the packages to be pip-installed, and (despite warnings about e.g. cyclic dependencies) offers no form of dependency resolution since it's only based on the (single) package versions installed in the environment. Such shortcomings are avoided when using pipgrip, since packages don't need to be installed and all versions available to pip are considered.
This pure-Python, OS independent package is available on PyPI:
pip install pipgrip
This package can be used to:
- Render an exhaustive dependency tree for any given pip-compatible package(s) with
- Alleviate Python dependency hell by resolving the latest viable combination of required packages
- Avoid bugs by running pipgrip as a stage in CI pipelines
- Detect version conflicts for given constraints and give human readable feedback about it
- Warn for cyclic dependencies in local projects [and install them anyway]:
pipgrip -v --tree . [--install -e]
- Install complex packages without worries:
pipgrip --install aiobotocore[awscli]
- Generate a lockfile with a complete working set of dependencies for worriless installs:
pipgrip --lock aiobotocore[awscli] && pip install aiobotocore[awscli] --constraint ./pipgrip.lock
- Combine dependency trees of multiple packages into one unified set of pinned packages:
pipgrip --lock --install --tree -v .[boto3] s3transfer==0.2.1
See also known caveats.
$ pipgrip --help Usage: pipgrip [OPTIONS] [DEPENDENCIES]... pipgrip is a lightweight pip dependency resolver with deptree preview functionality based on the PubGrub algorithm, which is also used by poetry. For one or more PEP 508 dependency specifications, pipgrip recursively fetches/builds the Python wheels necessary for version solving, and optionally renders the full resulting dependency tree. Options: --install Install full dependency tree after resolving. -e, --editable Install a project in editable mode. --user Install to the Python user install directory for your platform -- typically ~/.local/, or %APPDATA%\Python on Windows. -r, --requirements-file FILE Install from the given requirements file. This option can be used multiple times. --lock Write out pins to './pipgrip.lock'. --pipe Output space-separated pins instead of newline- separated pins. --json Output pins as JSON dict instead of newline- separated pins. Combine with --tree for a detailed nested JSON dependency tree. --sort Sort pins alphabetically before writing out. Can be used bare, or in combination with --lock, --pipe, --json, --tree-json, or --tree-json-exact. --tree Output human readable dependency tree (top-down). Combine with --json for a detailed nested JSON dependency tree. Use --tree-json instead for a simplified JSON dependency tree (requirement strings as keys, dependencies as values), or --json-tree-exact for exact pins as keys. --tree-ascii Output human readable dependency tree with ASCII tree markers. --reversed-tree Output human readable dependency tree (bottom-up). --max-depth INTEGER Maximum (JSON) tree rendering depth (default -1). --cache-dir DIRECTORY Use a custom cache dir. --no-cache-dir Disable pip cache for the wheels downloaded by pipper. Overrides --cache-dir. --index-url TEXT Base URL of the Python Package Index (default https://pypi.org/simple). --extra-index-url TEXT Extra URLs of package indexes to use in addition to --index-url. --pre Include pre-release and development versions. By default, pip implicitly excludes pre-releases (unless specified otherwise by PEP 440). -v, --verbose Control verbosity: -v will print cyclic dependencies (WARNING), -vv will show solving decisions (INFO), -vvv for development (DEBUG). -h, --help Show this message and exit.
Exhaustive dependency trees without the need to install any packages (at most build some wheels).
$ pipgrip --tree pipgrip pipgrip (0.6.7) ├── anytree (2.8.0) │ └── six>=1.9.0 (1.15.0) ├── click>=7 (7.1.2) ├── packaging>=17 (20.9) │ └── pyparsing>=2.0.2 (2.4.7) ├── pip>=7.1.0 (21.0.1) ├── pkginfo>=1.4.2 (1.7.0) ├── setuptools>=38.3 (53.0.0) └── wheel (0.36.2)
For more details/further processing, combine
--json for a detailed nested JSON dependency tree. See also
--tree-ascii (no unicode tree markers), and
--tree-json-exact (simplified JSON dependency trees).
--lock option, resolved (pinned) dependencies are additionally written to
$ pipgrip --tree --lock botocore==1.13.48 'boto3>=1.10,<1.10.50' botocore==1.13.48 (1.13.48) ├── docutils<0.16,>=0.10 (0.15.2) ├── jmespath<1.0.0,>=0.7.1 (0.9.5) ├── python-dateutil<3.0.0,>=2.1 (2.8.1) │ └── six>=1.5 (1.14.0) └── urllib3<1.26,>=1.20 (1.25.8) boto3<1.10.50,>=1.10 (1.10.48) ├── botocore<1.14.0,>=1.13.48 (1.13.48) │ ├── docutils<0.16,>=0.10 (0.15.2) │ ├── jmespath<1.0.0,>=0.7.1 (0.9.5) │ ├── python-dateutil<3.0.0,>=2.1 (2.8.1) │ │ └── six>=1.5 (1.14.0) │ └── urllib3<1.26,>=1.20 (1.25.8) ├── jmespath<1.0.0,>=0.7.1 (0.9.5) └── s3transfer<0.3.0,>=0.2.0 (0.2.1) └── botocore<2.0.0,>=1.12.36 (1.13.48) ├── docutils<0.16,>=0.10 (0.15.2) ├── jmespath<1.0.0,>=0.7.1 (0.9.5) ├── python-dateutil<3.0.0,>=2.1 (2.8.1) │ └── six>=1.5 (1.14.0) └── urllib3<1.26,>=1.20 (1.25.8) $ cat ./pipgrip.lock botocore==1.13.48 docutils==0.15.2 jmespath==0.9.5 python-dateutil==2.8.1 six==1.14.0 urllib3==1.25.8 boto3==1.10.48 s3transfer==0.2.1
NOTE: Since the selected botocore version is older than the one required by the recent versions of boto3, all boto3 versions will be checked for compatibility with botocore==1.13.48.
If version conflicts exist for the given (ranges of) package version(s), a verbose explanation is raised.
$ pipgrip auto-sklearn~=0.6 dragnet==2.0.4 Error: Because dragnet (2.0.4) depends on scikit-learn (>=0.15.2,<0.21.0) and auto-sklearn (0.6.0) depends on scikit-learn (<0.22,>=0.21.0), dragnet (2.0.4) is incompatible with auto-sklearn (0.6.0). And because no versions of auto-sklearn match >0.6.0,<1.0, dragnet (2.0.4) is incompatible with auto-sklearn (>=0.6.0,<1.0). So, because root depends on both auto-sklearn (~=0.6) and dragnet (==2.0.4), version solving failed.
NOTE: If older versions of auto-sklearn are allowed, PubGrub will try all acceptable versions of auto-sklearn. In this case, auto-sklearn==0.5.2 requires scikit-learn (<0.20,>=0.19), making it compatible with dragnet==2.0.4.
If cyclic dependencies are found, it is noted in the resulting tree.
$ pipgrip --tree -v keras==2.2.2 WARNING: Cyclic dependency found: keras depends on keras-applications and vice versa. WARNING: Cyclic dependency found: keras depends on keras-preprocessing and vice versa. keras==2.2.2 (2.2.2) ├── h5py (2.10.0) │ ├── numpy>=1.7 (1.18.1) │ └── six (1.14.0) ├── keras-applications==1.0.4 (1.0.4) │ ├── h5py (2.10.0) │ │ ├── numpy>=1.7 (1.18.1) │ │ └── six (1.14.0) │ ├── keras>=2.1.6 (2.2.2, cyclic) │ └── numpy>=1.9.1 (1.18.1) ├── keras-preprocessing==1.0.2 (1.0.2) │ ├── keras>=2.1.6 (2.2.2, cyclic) │ ├── numpy>=1.9.1 (1.18.1) │ ├── scipy>=0.14 (1.4.1) │ │ └── numpy>=1.13.3 (1.18.1) │ └── six>=1.9.0 (1.14.0) ├── numpy>=1.9.1 (1.18.1) ├── pyyaml (5.3) ├── scipy>=0.14 (1.4.1) │ └── numpy>=1.13.3 (1.18.1) └── six>=1.9.0 (1.14.0)
- PubGrub doesn't support version epochs, the main reason PyPA chose resolvelib over PubGrub for their new resolver.
- Package names are canonicalised in wheel metadata, resulting in e.g.
path.py -> path-pyand
keras_preprocessing -> keras-preprocessingin output.
- VCS Support isn't implemented yet.
--reversed-treeisn't implemented yet.
pip install -rdoes not accept
.as requirement, it is omitted from lockfiles, so
--pipeshould be used when installing local projects (or using
--lockand then passing it to pip using
- The equivalent of e.g.
pip install ../aiobotocore[boto3]is not yet implemented. However, e.g.
pipgrip --install .[boto3]is allowed.
make help for options like installing for development, linting and testing.
BSD 3-Clause License
Copyright (c) 2020, ddelange
All rights reserved.
Release history Release notifications | RSS feed
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size pipgrip-0.6.11-py2.py3-none-any.whl (52.0 kB)||File type Wheel||Python version py2.py3||Upload date||Hashes View|
|Filename, size pipgrip-0.6.11.tar.gz (47.3 kB)||File type Source||Python version None||Upload date||Hashes View|
Hashes for pipgrip-0.6.11-py2.py3-none-any.whl