Python Environment and Package Manager
Abode: Friendly Python Packaging
Most experienced Python users know that Python packaging is rough. Abode is an attempt to make things nicer by extending Conda. It uses Conda under the hood, but makes things a bit easier and robust.
Conda is by far my preferred packaging solution. It works as an environment manager (better than virtualenv in my opinion) as well as a package manager. The default repository is focused on data science software. However, Conda also installs packages from PyPI with pip. Also, unlike pip, Conda installs non-Python libraries such as MKL and CUDA that often increase operation speeds by 10-100x.
An aside: I recommend installing Miniconda instead of the full Anaconda distribution. The vast majority of people won't need every single package in the Anaconda distribution. So save yourself some time, bandwidth, and storage. Install Miniconda, then create environments and install packages as needed.
The Main Attraction
The biggest issue with Conda is saving an environment's dependencies and sharing it across platforms. You can get the dependencies using
conda env export > environment.yml. Say I have an environment where I have installed Flask, Numpy, and PyTorch. Even though those are the only packages I intentionally installed, Conda installs all the dependencies as well. Conda exports the environment to a YAML file that looks like:
name: flask channels: - pytorch - defaults dependencies: - blas=1.0=mkl - ca-certificates=2019.5.15=1 - certifi=2019.6.16=py37_1 - cffi=1.12.3=py37hb5b8e2f_0 - intel-openmp=2019.4=233 - libcxx=4.0.1=hcfea43d_1 - libcxxabi=4.0.1=hcfea43d_1 - libedit=3.1.20181209=hb402a30_0 - libffi=3.2.1=h475c297_4 - libgfortran=3.0.1=h93005f0_2 - mkl=2019.4=233 - mkl-service=2.0.2=py37h1de35cc_0 - mkl_fft=1.0.14=py37h5e564d8_0 - mkl_random=1.0.2=py37h27c97d8_0 - ncurses=6.1=h0a44026_1 - ninja=1.9.0=py37h04f5b5a_0 - numpy=1.16.4=py37hacdab7b_0 - numpy-base=1.16.4=py37h6575580_0 - openssl=1.1.1c=h1de35cc_1 - pip=19.1.1=py37_0 - pycparser=2.19=py37_0 - python=3.7.4=h359304d_1 - pytorch=1.2.0=py3.7_0 - readline=7.0=h1de35cc_5 - setuptools=41.0.1=py37_0 - six=1.12.0=py37_0 - sqlite=3.29.0=ha441bb4_0 - tk=8.6.8=ha441bb4_0 - wheel=0.33.4=py37_0 - xz=5.2.4=h1de35cc_4 - zlib=1.2.11=h1de35cc_3 - pip: - click==7.0 - flask==1.1.1 - itsdangerous==1.1.0 - jinja2==2.10.1 - markupsafe==1.1.1 - werkzeug==0.15.5 prefix: /Users/mat/miniconda3/envs/flask
The issue here is that defining the versions potentially breaks the environment on platforms other than the original one. I created this environment on my MacBook. It is not guaranteed that all these dependencies are available for Linux or Windows, likely breaking the environment on these platforms.
Also, a lot of the time we're not concerned with the exact versions of our packages. Instead we are okay with the newest versions, or any version greater than some release with a specific feature. In these cases, locking to specific versions is overly strict.
You can create essentially the same environment with this file:
name: flask channels: - pytorch - defaults dependencies: - numpy - pip - python=3 - pytorch - pip: - flask prefix: /Users/mat/miniconda3/envs/flask
Conda will take this file and solve all the necessary dependencies on whatever platform you're using. Of course if there are specific versions you want (like
python=3 here) you can define those.
Abode manages Conda environments by creating and editing minimal environment files like these. Hopefully this will allow users to take advantage of the great things Conda is doing, while making the environments portable.
- Python 3.6+ because I like f-strings
- Conda, as noted above, I suggest installing Miniconda instead of the full Anaconda distribution
Abode is available from PyPI:
pip install abode
Warning: This is very early. Use at your own risk.
So far this is what I have implemented.
Create an environment
To create a new environment with Python 3 installed:
abode create -n env_name python=3
Behind the scenes this creates an environment file and uses Conda to create an environment from the file.
Create from environment file
To create an environment from an environment file (a YAML file created by Abode or Conda):
abode create -f FILE
Enter an environment
I haven't figure out how to do this without using conda in the shell, so for now:
conda activate env_name
This is pretty high priority for me. It's awkward to switch between abode and conda commands.
Installing packages is the same as Conda:
abode install numpy matplotlib
Behind the scenes, Abode is adding these dependencies to the environment file, then updating the environment from the file.
Install with pip
--pip flag to install a package with pip.
abode install flask --pip
Install from a non-default channel
The channel option
-c adds the channel to the environment file.
abode install pytorch -c pytorch
This updates all the packages in the environment without locked versions.
Export the environment
abode export > environment.yml
This just copies the active environment's dependency file to the new file.
List packages in the current environment
To print out the list of all installed packages, equivalent to
If you want to see which packages have been installed with Abode:
abode list -f
abode list --file
List environments managed with Abode
abode env list
Happy to work with you on this. Create an issue or a pull request. Say hi.
Release history Release notifications | RSS feed
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size abode-0.1.4-py3-none-any.whl (9.7 kB)||File type Wheel||Python version py3||Upload date||Hashes View|
|Filename, size abode-0.1.4.tar.gz (6.4 kB)||File type Source||Python version None||Upload date||Hashes View|