z3c.pypimirror 1.0.16

A module for building a complete or a partial PyPI mirror

Setting up a PyPI “simple” index

This package provides a mirror for the PyPI simple interface, http://cheeseshop.python.org/simple/.

To set up a mirror:

• install this package using setuptools (easy_install, buildout, etc.) so that the script, pypimirror script is installed.

• create your mirror configuration file maybe based on config.cfg.sample

• Run the pypimirror script passing the name of the mirror configuration file

  This will initialize the mirror.

• Set up a cron job to run update-mirror periodically.

LICENSE

Zope Public License 2.1 (ZPL 2.1)

Authors

z3c.pypimirror took place at the Blackforest Sprint 2008 and involved the the following persons:

• Daniel Kraft (Lead)

• Gottfried Ganssauge

• Josip Delic

• Andreas Jung

Changes

Change history

1.0.16 (unreleased)

• added more or less sane filename detection heuristics (filename from url)

• fetch-since-days is configurable now.

1.0.15 (20010-02-08)

• LP #385387: fixed issues with incompatible BeautifulSoup 3.1 version

• fixed incompatiblity with new PyPI download URLs (../../packages/..)

1.0.14 (2009-05-29)

• Ignoring invalid external html pages (thanks Sridhar Ratnakumar)

1.0.13 (2009-01-26)

• taking ‘home_url’ into account for mirroring external resources

1.0.12 (2009-01-19)

• fix for Sourceforge download URLs

1.0.11 (2009-01-14)

• fixed issue with broken links generated by a base_url with a trailing slash

1.0.10 (2008-11-25)

• Removing dupes from package list

1.0.9 (2008-11-07)

• Display time in UTC

1.0.8 (2008-11-04)

• Fixed typo in PyPI

• Added a nice link for google to our project page ;)

1.0.7 (2008-11-03)

• Added a footer line to index.html indicating the version and no of pkgs synced.

1.0.6 (2008-10-30)

• renamed index.html to full.html

• renamed index2.html to index.html

1.0.5 (2008-10-27)

• disabled ‘cleanup’ option since it does play nicely with the –update-fetch option

1.0.4 (2008-10-24)

• add version number to HTTP user-agent header

1.0.3 (2008-10-24)

• using fetch_url() for making HTTP requests in order to specify a dedicated user-agent header

1.0.2 (2008-10-24)

• fixed packaging issues

1.0.1 (2008-10-24)

• new options: –initial-fetch and –update-mode

1.0 (2008-10-06)

• Release and code move to launchpad

0.2.10 (2008-09-09)

• demo release for DZUG tutorial day

0.2.9 (2008-09-04)

• fixed issue with logger changes in 0.2.8

0.2.8 (2008-09-03)

• added highly experimental follow_external_index_pages option in order to deal with external in a better way

• fetching external download files only if they match the current package name

• some more logging

• using TimedRotatingFileHandler for logging (rotating after one day)

0.2.7 (2008-09-02)

• bahhh…forgotten pdb call :->

0.2.6 (2008-09-02)

• made fetch_index() more robust

0.2.5 (2008-09-02)

• fixed NameError in exception handling

0.2.4 (2008-08-31)

• added logging support

• minor internal cleanup

• commandline options handling updated

0.2.3 (2008-08-29)

• added –follow-external-links option

0.2.2 (2008-08-28)

• renamed index.html to index2.html

• renamed full.html to index.html

• some internal cleanup

0.2.1 (2008-08-28)

• ls() returns filenames in sorted order

0.2.0 (2008-08-28)

• setup.py cleanup and fixes

• fixed license file

• updated README.txt

0.1.1 (2008-08-27)

• corrected the index pages for packages (have full urls now)

• added a full.html for packages (you better gz this ;) )

• removed html page download for external sites

0.1.0 (2008-08-27)

• initial release