devpi-server 1.1

devpi-server: reliable fast pypi.python.org caching server

• issues: https://bitbucket.org/hpk42/devpi/issues

• IRC: #devpi on irc.freenode.net.

• repository: https://bitbucket.org/hpk42/devpi

• mailing list: https://groups.google.com/d/forum/devpi-dev

• compatibility: unix-py26/py27 and windows/py27

consistent robust pypi-cache

You can point pip or easy_install to the root/pypi/+simple/ index, serving as a self-updating transparent cache for pypi-hosted and external packages. Cache-invalidation uses the latest and greatest PyPI protocols. The cache index continues to serve when offline and will resume cache-updates once network is available.

github-style indexes

Each user can have multiple indexes and upload packages and docs via standard setup.py invocations. Users, indexes (and soon projects and releases) are manipulaed through a RESTful HTTP API.

index inheritance

Each index can be configured to merge in other indexes so that it serves both its uploads and all releases from other index(es). For example, an index using root/pypi as a parent is a good place to test out a release candidate before you push it to PyPI.

good defaults and easy deployment

Get started easily and create a permanent devpi-server deployment including pre-configured templates for nginx and cron.

separate tool for Packaging/Testing activities

The complimentary devpi-client tool helps to manage users, indexes, logins and typical setup.py-based upload and installation workflows.

See http://doc.devpi.net for getting started and documentation.

1.1

• systematically test pypi/mirror code against all 34K pypi projects so that we know that all http/https installable archive links that pypi offers are correctly recognized by devpi-server’s root/pypi index.

• if no pypi mirror state is known, devpi-server now calls pypi to obtain names/serials. It will fail to start if no such initial connection is possible. Once a first mirror state is known, subsequent devpi-server starts will not perform this initial query.

• speed up and make more reliable all operations on private packages which have no pypi.python.org release: we can now determine if a project exists on pypi and under which name exactly without remote queries or redirects to pypi.python.org.

• fix issue45: register/upload package names are now properly validated and redirects take place if e.g. a project was registered as “name-sub” and “+simple/name_sub” is queried.

• new –upgrade-state command to allow for easy and safe in-place upgrading of server state. This is not guranteed to be possible for all future releases which might require using –export with an older version and –import with a newer version.

• new –export/–import options to dump and import server contents: users, indexes, docs, release files and (test) attachments. Note that root/pypi (PyPI-caching information) will not be exported/imported. (maybe in the future if there is demand).

• fix issue49: both push and import/export now support docfiles. Note, however, that docfiles relate to a project as a whole and are not tied to a particular version. This property is inherited from the PyPI standard upload_docs action and cannot be changed without interfering or replacing the upload_docs protocol of setuptools/sphinx.

• fix issue51: return 200 code if release file is successfully uploaded but jenkins could not be triggered (previously returned 500)

• reject simple/NAME if NAME contains non-ascii characters (PEP426 naming rules)

• devpi-server now returns a X-DEVPI-API-VERSION and X-DEVPI-SERVER-VERSION header. For future incompatible changes these versions allow clients to reject interactions.

• also add “.serverversion” file and write it if it does not exist, and make devpi-server use it to verify if operating on a compatible server data layout, otherwise bail out.

• address issue43: –gendeploy now uses pip without –pre and explicitely instructs pip to install the exact same version of devpi-server with which –gendeploy is issued.

• fix issue46 – for GET /root/pypi/ only show a link to the simple page instead of computing “latest in-stage packages” which is only useful for devpi’s user indices.

• fix issue37: upload with expired login causes proper 401

1.0

• rename “–datadir” to “serverdir” to better match the also picked up DEVPI_SERVERDIR environment variable.

• fix a strange effect in that sometimes tools ask to receive a package url with a “#md5=…” arriving at the server side. We now strip that part out before trying to serve the file.

• on startup don’t create any initial indexes other than the “root/pypi” pypi caching mirror.

• introduce --start, --stop and --log commands for controling a background devpi-server run. (these commands previously were implemented with the devpi-client and the “server” sub command)

• fix issue27: provide full list of pypi names in root/pypi’s simple view (and simple pages from inheriting indices)

• default to “eventlet” server when creating deployment with –gendeploy

• fix issue25: return 403 Forbidden when trying to delete the root user.

• fix name mangling issue for pypi-cache: “project_name*” is now matched correctly when a lookup for “project-name” happens.

• fix issue22: don’t bypass CDN by default, rather provide an “–bypass-cdn” option to do it (in case you have cache-invalidation troubles)

• fix issue20 and fix issue23: normalize index specs internally (“/root/dev” -> “root/dev”) and check if base indices exist.

• add Jenkins build job triggering for running the tests for a package through tox.

• inheritance cleanup: inherited versions for a project are now shadowed and not shown anymore with getreleaselinks() or in +simple pages if the “basename” is exactly shadowed.

• fix issue16: enrich projectconfig json with a “+shadow” file which lists shadowed “versions”

• initial wheel support: accept “whl” uploads and support caching of whl files from pypi.python.org

• implemented internal push operation between devpi indexes

• show “docs” link if documentation has been uploaded

• pushing releases to pypi.python.org will now correctly report the filetype/pyversion in the metadata.

• add setting of acl_upload for indexes. Only the owning user and acl_upload users may upload releases, files or documentation to an index.

• add –passwd USER option for setting a user’s password server-side

• don’t require email setting for creating users

0.9.4

• fix issue where lookups into subpages of the simple index (simple/NAME/VER) would not trigger a 404 as they should.

0.9.3

• fixed issue9: caching of packages where upstream provides no last-modified header now works.

• fixed issue8: only http/https archives are allowed and other schemes (such as ftp) are silently skipped

• added support for REST DELETE methods of projects and versions on an index

• added “argcomplete” support for tab completion on options (thanks to Anthon van der Neut)

0.9.2

• fix /USER/INDEXNAME root views to contain only latest in-stage packages

• make +api calls return bases so that “devpi use” can show them

0.9.1

• return 404 for submits to root/pypi

• properly sorted release file links on stage indexes

• “push” method on indexes for transfering release files to another pypi index

• properly handle urls from indexes with ~ and other special chars

• fix root/pypi and root/dev page serving in various cases

0.9.0

• implement more precise pypi.python.org CDN/caching invalidation technique, using the most recent PyPI API (“X-PYPI-LAST-SERIAL” on simple pages and xmlrpc.list_packages_with_serial()). also simplify background tasks to become only one async task doing both changelog checking and triggering updates.

• use a filesystem based storage mechanism instead of Redis

• prevent automatic decoding of gzip files in case of content encoding

• XXX preliminarily introduce new int/dev, int/prod indexes where int/dev inherits packages from both int/prod and ext/pypi.

• XXX introduce preliminary support for client-side “devpi” workflow tool

• allow uploads to int/dev

• if no crontab exists for a user, simply create one instead of erroring out. Thanks Andi Albrecht.

• internal refactoring for better organisation of redis access

0.8.5

• re-fix issue6: tests and fixes for django-debug-toolbar where recursive scraping was accidentally triggered

• remove fine-grained http caching for now because caching on the index level seems enough. This avoids an issues that occured when installing icalendar and also some offline/online state change issues.

• added a note to README for how to upgrade –gendeploy installs

• remove general dependency on virtualenv which is only needed for –gendeploy. Adapt docs accordingly.

• remove dependency on pip by shifting the relevant scraping bits directly to _pip.py

0.8.4

• use pip’s link parser rather than beautifulsoup to benefit from link parsing code tested out and maintained in the wild. Adapt README.

• skip a test if crontab command is not present

  (thanks Markus Zapke-Gruendemann)

• release 0.8.3 is not useable

0.8.2

• fix issue6 - some edge cases for link parsing uncovered by BeautifulSoup and CouchApp installs. Thanks Anton Baklanov.

• fix issue5 - require minimal versions for deps, thanks Andi Albrecht

• remove superflous include lines in MANIFEST.in

0.8.1

• fix: change gendeployed supervisord.conf to not autostart processes on “devpi-ctl” invocations. “devpi-ctl help” would autostart the processes after a shutdown which is not very intuitive. This is actually compatible with the documentation.

• refactor –gendeploy related code to be in gendeploy.py

• fix: fixate path of devpi-server in gendeployed configuration to point to the freshly installed devpi-server. also add a note to the README.

0.8

• introduce “–gendeploy=TARGETDIR” for generating a virtualenv directory with supervisor-based configuration in TARGETDIR/etc and a TARGETDIR/bin/devpi-ctl helper to control the running of devpi-server and redis-server processes.

• fix issue4: keep the “changelog” thread active across network/reachability errors. Thanks Laurent Brack.

• use argparse instead of optparse, simplify and group options

• fix python2.6 and simplify logging configuration

0.7

• Initial release