Composable command line interface toolkit
Project description
pipgrip
pipgrip is a lightweight pip dependency resolver with deptree preview functionality based on the PubGrub algorithm, which is also used by poetry. For one or more PEP 508 dependency specifications, pipgrip recursively fetches/builds the Python wheels necessary for version solving, and optionally renders the full resulting dependency tree.
pipgrip vs. poetry
poetry offers package management with dependency resolution, essentially replacing pip/setuptools. This means that poetry packages don't contain setup.py
, and hence are not compatible with pip install -e
: poetry projects would have to be converted to setuptools-based projects with e.g. dephell. To avoid such hassle, pipgrip only requires the selected package(s) + dependencies to be available to pip in the ususal way.
pipgrip vs. pipdeptree
For offline usage, pipdeptree](https://github.com/naiquevin/pipdeptree) can inspect the current environment and show how the currently installed packages relate to each other. This however requires the packages to be pip-installed, and (despite warnings about e.g. cyclic dependencies) offers no form of dependency resolution since it's only based on the (single) package versions installed in the environment. Such shortcomings are avoided when using pipgrip, since packages don't need to be installed and all versions available to pip are considered.
Installation
This pure-Python, OS independent package is available on PyPI:
pip install pipgrip
Usage
This package can be used to:
- Alleviate Python dependency hell by resolving the latest viable combination of required packages
- Render an exhaustive dependency tree for any given pip-compatible package(s) with
--tree
- Detect version conflicts for given constraints and give human readable feedback about it
- Find dependency conflicts in local projects:
pipgrip --tree .
- Install complex packages without worries using:
pip install -U --no-deps `pipgrip --pipe aiobotocore[awscli]`
- Generate a lockfile with a complete working set of dependencies for worriless installs (see known caveats):
pipgrip --lock -tree aiobotocore[awscli] && pip install -U --no-deps -r ./pipgrip.lock
pipgrip aiobotocore[awscli] | pip install -U --no-deps -r /dev/stdin
$ pipgrip --help
Usage: pipgrip [OPTIONS] [DEPENDENCIES]...
Options:
--lock Write out pins to './pipgrip.lock'.
--pipe Output space-separated pins instead of newline-
separated pins.
--json Output pins as json dict instead of newline-
separated pins.
--tree Output human readable dependency tree (top-down).
--reversed-tree Output human readable dependency tree (bottom-up).
--max-depth INTEGER Maximum tree rendering depth (defaults to -1).
--cache-dir PATH Use a custom cache dir.
--no-cache-dir Disable pip cache for the wheels downloaded by
pipper. Overrides --cache-dir.
--index-url TEXT Base URL of the Python Package Index (default
https://pypi.org/simple).
--extra_index-url TEXT Extra URLs of package indexes to use in addition to
--index-url.
--pre Include pre-release and development versions. By
default, pip only finds stable versions.
-v, --verbose Control verbosity: -v will print cyclic dependencies
(WARNING), -vv will show solving decisions (INFO),
-vvv for development (DEBUG).
--help Show this message and exit.
Dependency trees
Exhaustive dependency trees without the need to install any packages (at most build some wheels).
$ pipgrip --tree pipgrip
pipgrip (0.0.3)
├── anytree (2.7.3)
│ └── six (1.13.0)
├── click (7.0)
├── packaging>=17 (20.0)
│ ├── pyparsing>=2.0.2 (2.4.6)
│ └── six (1.13.0)
├── pkginfo>=1.4.2 (1.5.0.1)
├── setuptools>=38.3 (44.0.0)
└── wheel (0.33.6)
Lockfile generation
Using the --lock
option, resolved (pinned) dependencies are additionally written to ./pipgrip.lock
.
$ pipgrip --lock boto3
boto3==1.10.46
botocore==1.13.46
docutils==0.15.2
jmespath==0.9.4
python-dateutil==2.8.1
six==1.13.0
urllib3==1.25.7
s3transfer==0.2.1
Version conflicts
If version conflicts exist for the given (ranges of) package version(s), a verbose explanation is raised.
$ pipgrip auto-sklearn~=0.6 dragnet==2.0.4
Error: Because dragnet (2.0.4) depends on scikit-learn (>=0.15.2,<0.21.0)
and auto-sklearn (0.6.0) depends on scikit-learn (<0.22,>=0.21.0), dragnet (2.0.4) is incompatible with auto-sklearn (0.6.0).
And because no versions of auto-sklearn match >0.6.0,<1.0, dragnet (2.0.4) is incompatible with auto-sklearn (>=0.6.0,<1.0).
So, because root depends on both auto-sklearn (~=0.6) and dragnet (==2.0.4), version solving failed.
NOTE: If older versions of auto-sklearn are allowed, PubGrub will try all acceptable versions of auto-sklearn. In this case, auto-sklearn==0.5.2 requires scikit-learn (<0.20,>=0.19), making it compatible with dragnet==2.0.4.
Cyclic dependencies
If cyclic dependencies are found, it is noted in the resulting tree.
$ pipgrip --tree keras==2.2.2
keras==2.2.2 (2.2.2)
├── h5py (2.10.0)
│ ├── numpy>=1.9.1 (1.18.0)
│ └── six>=1.9.0 (1.13.0)
├── keras-applications==1.0.4 (1.0.4)
│ ├── h5py (2.10.0)
│ │ ├── numpy>=1.9.1 (1.18.0)
│ │ └── six>=1.9.0 (1.13.0)
│ ├── keras==2.2.2 (2.2.2, cyclic)
│ └── numpy>=1.9.1 (1.18.0)
├── keras-preprocessing==1.0.2 (1.0.2)
│ ├── keras==2.2.2 (2.2.2, cyclic)
│ ├── numpy>=1.9.1 (1.18.0)
│ ├── scipy>=0.14 (1.4.1)
│ │ └── numpy>=1.9.1 (1.18.0)
│ └── six>=1.9.0 (1.13.0)
├── numpy>=1.9.1 (1.18.0)
├── pyyaml (5.2)
├── scipy>=0.14 (1.4.1)
│ └── numpy>=1.9.1 (1.18.0)
└── six>=1.9.0 (1.13.0)
Known caveats
pip install -U `pipgrip package`
without--no-deps
is unsafe while pip doesn't yet have a built-in dependency resolver, and leaves room for interpretation by pip- Package names are canonicalised in wheel metadata, resulting in e.g.
path.py -> path-py
andkeras_preprocessing -> keras-preprocessing
in output - VCS Support isn't implemented yet
--reversed-tree
isn't implemented yet- Since
pip install -r
does not accept.
as requirement, it is omitted from lockfiles, so--pipe
should be used when installing local projects - Installing packages using pipgrip is not very intuitive, so maybe pipgrip needs a stable
--install
flag
Development
Create a virtual environment and get ready to develop:
make install
This make-command is equivalent to the following steps:
Install pre-commit and other continous integration dependencies in order to make commits and run tests.
pip install -r requirements/ci.txt
pre-commit install
With requirements installed, make lint
and make test
can now be run. There is also make clean
, and make all
which runs all three.
To import the package in the python environment, install the package (-e
for editable installation, upon import, python will read directly from the repository).
pip install -e .
See also
BSD 3-Clause License
Copyright (c) 2020, ddelange
All rights reserved.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for pipgrip-0.0.3-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 53c7afea15639c1b4288d0cf7dcad3ed227fd062c579d0ef9f4964a15ec8818e |
|
MD5 | ec5ec5329c075d8f5b5bf913b77355ad |
|
BLAKE2b-256 | 4412a8e0dafde834690df36892f03e0b7b3280dd4dd3f36a9f8547d84c6df3e8 |