Simple pypi package
Simple python package index
How simpypi works
The heart of simpypi is simpypi.wsgi.SimPyPI, a simple WSGI web app that accepts a uploaded python package distribution and places it according to its name and version metadata appropriate to the simple index protocol .
For security, SimPyPI returns straight HTTP 400 s for invalid requests. This could probably be improved.
SimPyPI itself does not serve this directory. The directory should be served by a fileserver that will generate directory indices (as apprpropriate to http://guide.python-distribute.org/contributing.html#the-simple-index-protocol ) such as Apache or FileServer . simpypi.factory.factory does provide a factory to make a WSGI app that wraps FileServer in middleware and serves the simple index under /index/ and the SimPyPI app at /. Additionally, a simpypi command line program is provided that front-ends this.
Currently simpypi only works on source distributions (that is, packages made with python setup.py sdist).
I installed an instance of simpypi at http://k0s.org:8080 for demonstration purposes with a package index at http://k0s.org:8080/index/ . For testing purposes, I made a script, upload_mobase.py that uploads the mozbase packages to simpypi. So the index is now populated with them.
You can upload packages with curl
> wget http://pypi.python.org/packages/source/P/PyYAML/PyYAML-3.10.tar.gz > curl -F 'package=@PyYAML-3.10.tar.gz' http://k0s.org:8080/
You can easy_install mozbase from http://k0s.org:8080/index/ . If a package’s dependencies can be found from the simpypi package index, they will also be installed from the index:
> virtualenv.py tmp New python executable in tmp/bin/python Installing setuptools............done. Installing pip...............done. > cd tmp/ (tmp)│easy_install -i http://k0s.org:8080/index/ mozrunner Searching for mozrunner Reading http://k0s.org:8080/index/mozrunner/ Best match: mozrunner 5.1 Downloading http://k0s.org:8080/index/mozrunner/mozrunner-5.1.tar.gz Processing mozrunner-5.1.tar.gz Running mozrunner-5.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-gqerOV/mozrunner-5.1/egg-dist-tmp-Qyx3Cr Adding mozrunner 5.1 to easy-install.pth file Installing mozrunner script to /home/jhammel/tmp/bin Installed /home/jhammel/tmp/lib/python2.7/site-packages/mozrunner-5.1-py2.7.egg Processing dependencies for mozrunner Searching for mozprofile>=0.1 Reading http://k0s.org:8080/index/mozprofile/ Best match: mozprofile 0.1 Downloading http://k0s.org:8080/index/mozprofile/mozprofile-0.1.tar.gz Processing mozprofile-0.1.tar.gz Running mozprofile-0.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-4Im6x0/mozprofile-0.1/egg-dist-tmp-9Jp5TR Adding mozprofile 0.1 to easy-install.pth file Installing mozprofile script to /home/jhammel/tmp/bin Installed /home/jhammel/tmp/lib/python2.7/site-packages/mozprofile-0.1-py2.7.egg Searching for mozprocess Reading http://k0s.org:8080/index/mozprocess/ Best match: mozprocess 0.1b2 Downloading http://k0s.org:8080/index/mozprocess/mozprocess-0.1b2.tar.gz Processing mozprocess-0.1b2.tar.gz Running mozprocess-0.1b2/setup.py -q bdist_egg --dist-dir /tmp/easy_install-KU6AiF/mozprocess-0.1b2/egg-dist-tmp-4j5CMP Adding mozprocess 0.1b2 to easy-install.pth file Installed /home/jhammel/tmp/lib/python2.7/site-packages/mozprocess-0.1b2-py2.7.egg Searching for mozinfo Reading http://k0s.org:8080/index/mozinfo/ Best match: mozinfo 0.3.3 Downloading http://k0s.org:8080/index/mozinfo/mozinfo-0.3.3.tar.gz Processing mozinfo-0.3.3.tar.gz Running mozinfo-0.3.3/setup.py -q bdist_egg --dist-dir /tmp/easy_install-JaKeaz/mozinfo-0.3.3/egg-dist-tmp-xWojez Adding mozinfo 0.3.3 to easy-install.pth file Installing mozinfo script to /home/jhammel/tmp/bin Installed /home/jhammel/tmp/lib/python2.7/site-packages/mozinfo-0.3.3-py2.7.egg Searching for ManifestDestiny>=0.5.4 Reading http://k0s.org:8080/index/ManifestDestiny/ Best match: ManifestDestiny 0.5.4 Downloading http://k0s.org:8080/index/ManifestDestiny/ManifestDestiny-0.5.4.tar.gz Processing ManifestDestiny-0.5.4.tar.gz Running ManifestDestiny-0.5.4/setup.py -q bdist_egg --dist-dir /tmp/easy_install-2blF3S/ManifestDestiny-0.5.4/egg-dist-tmp-R3KZde Adding ManifestDestiny 0.5.4 to easy-install.pth file Installing manifestparser script to /home/jhammel/tmp/bin Installed /home/jhammel/tmp/lib/python2.7/site-packages/ManifestDestiny-0.5.4-py2.7.egg Finished processing dependencies for mozrunner
Note that all of the packages come from the k0s.org installation and not from http://pypi.python.org/ .
Running the Tests
The tests directory contains doctests and the test-runner, test.py . These tests illustrate basic functionality and protect from regressions if they are run before code is committed. tests-require.txt contains dependencies that should be installed to run the tests. Paste TestApp , though this could be transitioned to WebTest . virtualenv is used for isolating python environments.
To run the tests, do:
While simpypi is pretty simple, just because it is only 100 lines of code doesn’t mean that it is the right 100 lines of code. The following issues could be addressed:
- the temporary package should be dealt with entirely in memory, ideally. Currently we write to a file and move it.
- simpypi use pkginfo.sdist to read the data from a source distribution. Instead, the uploaded package should probably be unpacked and python setup.py sdist run and the resulting package put in the appropriate place. This will allow archives without PKG-INFO (e.g. http://hg.mozilla.org/build/talos/archive/tip.tar.gz ) to be uploaded as well as at least partially correct for the fact that currently simpypi only works for uploaded source distributions.