Skip to main content

Create a local pypi repository to use pip off-line.

Minirepo is a command-line program that downloads Python packages from pypi.python.org, so you can use pip without internet. I needed to maintain a python repository in an isolated cluster environment, and after trying several tools to mirror pypi index, I dedided to make my own tool.

Some mirroring tools such us bandersnatch didn’t meet my requirements, because I wanted to do a selective mirror, only downlowing all sources for python 2.7, for example. Bandersnatch gets the full content, about 140GB at the time of my first version of minirepo.

Then I was inspired by basket, which is almost what I wanted, but you need to specify the list of packages to download. I ended up using the json API to get the packages that I needed, and then calling basket to download or update the packages. In the end, that approach was slow and buggy, so I wrote this small program to do what I just needed. Now, my minirepo folder has about 12GB with only the latest packages, and it takes about 20 minutes to mirror.

Installation

Use pip

The easiest way to install it is to use pip:

$ pip install minirepo

Or download and install

Download the package file from https://pypi.python.org/pypi/minirepo, or the latest development version from https://github.com/sganis/minirepo, then:

$ tar xvzf minirepo-1.0.3.tar.gz
$ cd minirepo-1.0.3
$ python setup.py install

You can also use git:

$ git clone https://github.com/sganis/minirepo.git
$ cd minirepo
$ python setup.py install

Usage

# run it from the command line:
$ minirepo

# or run the python script if you didn't install it:
$ ./minirepo.py

The firt time it’s executed, the program will ask you for the local repository path, which defaults to ~/minirepo in Linux. A json configuration file is created and saved as ~/.minirepo, that you can edit to your preferences. This configuration file looks like this:

{
        "processes"       : 20,
        "repository"      : "/home/user/minirepo"
        "package_types"   : ["bdist_egg","bdist_wheel","sdist"],
        "extensions"      : ["bz2","egg","gz","tgz","whl","zip"],
        "python_versions" : ["2.7","any","cp27","py2","py27","source"],
}

Minirepo uses packages_types, extensions, and python_versions as filters. I was analysing the full list of packages available in pypi.python.org, and it looks that all the options are something like the list below, you can try any other option. For me, I was only interested in python 2.7 packages, sources, wheels and eegs distributions, and some extensions.

PYTHON_VERSIONS = [
        '2', '2.2', '2.3', '2.4', '2.5', '2.6', '2.7', '2.7.6', '3.0', '3.1',
        '3.2', '3.3', '3.4', '3.5', 'any', 'cp25', 'cp26', 'cp27', 'cp31',
        'cp32', 'cp33', 'cp34', 'cp35', 'py2', 'py2.py3', 'py26', 'py27',
        'py3', 'py32, py33, py34', 'py33', 'py34', 'python', 'source'
]

PACKAGE_TYPES = [
        'bdist_dmg', 'bdist_dumb', 'bdist_egg', 'bdist_msi', 'bdist_rpm',
        'bdist_wheel', 'bdist_wininst', 'sdist'
]

EXTENSIONS = [
        'bz2', 'deb', 'dmg', 'egg', 'exe', 'gz', 'msi', 'rpm', 'tgz', 'whl', 'zip'
]

Use pip without internet

$ pip install --no-index --find-links=/home/user/minirepo <package-name>

I prefer to setup environment variables in my profile so I don’t have to provide extra command line arguments.

# save these 2 variable in your profile
$ export PIP_NO_INDEX=true
$ export PIP_FIND_LINKS=/home/user/minirepo

# then run pip as usual
$ pip install <package-name>

Metadata

Release files for minirepo 1.0.3

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for minirepo 1.0.3
File Size Uploaded
minirepo-1.0.3.tar.gz 7.0 kB Details

Release files / minirepo-1.0.3.tar.gz

Download URL minirepo-1.0.3.tar.gz
Size 7.0 kB
Tags Source
SHA-256 checksum
How to use checksums
e695dae5c37ca92847d629e187ed8351bcb4360f4429ca0b0690dd6fb18d1664
BLAKE2b-256 checksum
How to use checksums
123b99710fa8aa8cff42cd39cb7160fcdec4b94b9894c08e90871be871d310bc
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No

Release history Release notifications | RSS feed

This release

1.0.3 This release

1 release file

1.0.2

1 release file

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page