This is a pre-production deployment of Warehouse, however changes made here WILL affect the production instance of PyPI.
Latest Version Dependencies status unknown Test status unknown Test coverage unknown
Project Description

Package-cache is a simple caching proxy for package downloads. Just configure a list of upstream sources (with the --source option) and point clients at the package-cache server. The first time a package is requested, we download that package from one of the sources and cache it locally, while also streaming it to the client. Future requests for that package are streamed directly from the local cache. This helps reduce the load on the network and source servers, if you have a number of local clients that will repeatedly request the same files (e.g. Gentoo’s distfiles).

We don’t do anything fancy with Cache-Control headers, since package source files should include the version stamp in the filename itself (e.g. my-package-0.1.2.tar.gz). Files are cached after the first request, and stored forever. This means that every package you’ve ever requested will still be there if you need it later. That’s nice, but it will end up consuming a fair amount of disk space. You might want to periodically cull the cache, using access times to see which files you are unlikely to want in the future.

Package-cache is written in Python, and has no dependencies outside the standard library.

Running package-cache

By default, we’ll use Python’s reference WSGI implementation to run our application:

$ package-cache --source http://distfiles.gentoo.org/

For other command-line options, see:

$ package-cache --help

If you need a more perfomant backend, you might try Gunicorn.

Gentoo

There’s an OpenRC init script in contrib/openrc/init.d, and a package-cache ebuild in my wtk overlay. To use package-cache as a caching proxy for your distfiles downloads, add the wtk overlay to layman and run:

# emerge --ask --verbose net-proxy/package-cache
# rc-update add default net-proxy/package-cache
# rc-service package-cache start

You can tweak the parameters by setting variables in /etc/conf.d/package-cache (PC_USER, PC_GROUP, CACHE_DIR, HOST, PORT, SOURCES, and PC_OPTS). See the init script for default values.

Once you’ve setup your package-cache service, just point your local package manager to the cache server instead of the usual mirror. For Portage, that’s going to be something like:

GENTOO_MIRRORS=”http://my-package-cache-server.net:4000/

in your /etc/portage/make.conf.

If you don’t want to tweak your clients (perhaps there are many of them, or they are out of your direct control), you can add some firewall rules to your router to transparently proxy specific Gentoo mirrors. With an internal eth1 and an internal proxy on 192.168.0.11, that looks something like:

# CACHE_IP=192.168.0.11
# for SOURCE_IP in $(dig +short distfiles.gentoo.org);
> do
>   iptables --table nat --append PREROUTING --protocol tcp \
>     --in-interface eth1 ! --source "${CACHE_IP}" \
>     --destination "${SOURCE_IP}" \
>     --match tcp --destination-port 80 \
>     --jump DNAT --to-destination "${CACHE_IP}:4000" ;
> done

To remove those entries later, repeat the command with --delete instead of --append. You may need to list the SOURCE_IP values explicitly if the DNS entries have changed. Run:

# iptables --table nat --list PREROUTING --numeric

to list the entries. See iptables(8) and iptables-extensions(8) for more details.

Release History

Release History

0.2

This version

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

Download Files

Download Files

TODO: Brief introduction on what you do with files - including link to relevant help section.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
package-cache-0.2.tar.gz (18.5 kB) Copy SHA256 Checksum SHA256 Source Feb 22, 2014

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting