A cheeseshop clone (PyPI server) written in pyramid
Pyshop is a private repository for python packages.
The aim is to split private projects in distinct private packages and keep a setup.py clean and working, by declaring all dependancies, exactly as public packages on PyPI.
Pyshop also mirror packages from PyPI safety (using ssl by checking certificate).
Pyshop use clear and simple ACL to manage privilleges:
- an installer group that can only download release file
- a developer group that can download/upload release file and browse the website and
- an admin group that have developer privilleges and accounts management.
So, every users, including “pip”, must be authenticated by login and password.
$ virtualenv pyshop $ cd pyshop (pyshop)$ source bin/activate (pyshop)$ pip install pyshop (pyshop)$ cp pyshop.sample.ini pyshop.ini (pyshop)$ vim pyshop.ini # change the pyshop.cookie_key setting (pyshop)$ pyshop_install pyshop.ini (pyshop)$ pserve pyshop.ini start --log-file=pyshop.log
You shoud edit the pyshop.ini file in order to configure the pyshop.cookie_key, the host:port that host the service. When the pyshop is running visit the web application, http://localhost:8000/ by default, to check all is fine.
For production usage, you should create accounts with the “developer” group. Visit http://localhost:8000/pyshop/user with the admin account to create accounts. You also should use an https reverse proxy. Python packaging core use basic authentication: it send user/password in clear.
Configuring your environment to use that new pyshop
Here is all configuration files for usual python tools you have to edit for simplify the usage of pyshop.
Configuration used by pip. This is a user file, you can set a developper or the pip generic account.
[global] # when mirroring a package, # pyshop retrieve informations from PyPI and # store them in its DB. # Be patient, it is not so long. default-timeout = 120 timeout = 120 [install] index-url = http://pip:changeme@localhost:8000/simple/
Note: If you are using a WSGI server that kill requests in a timeout, like uWSGI, set an appropriate timout for this service too.
A setup.cfg file is used by the “python setup.py develop” to install dependancies. You should use a generic account with have installer privilleges only, shared by every developper.
This file is a “per project file” at the root of the package.
[easy_install] index-url = http://pip:changeme@localhost:8000/simple/
This should work now:
python setup.py develop
Configuration used by setuptools to upload package. Every developper should have it’s own account to upload package.
[distutils] index-servers = pyshop [pyshop] username: admin # or create an account in pyshop admin interface password: changeme repository: http://localhost:8000/simple/
This should works now:
python setup.py sdist upload -v -r pyshop /pypi/pypiserver
Developper can’t add other accounts to give them upload right to their project. This can be done in the database or in the pyshop shell by an administrator.
$ pyshop_shell pyshop.ini In : pkg = Package.by_name(session, u'pyshop') In : pkg.owners.append(User.by_login(session, u'admin')) In : session.commit()
- Use requests for xmlrpc queries too.
- validate certificate if https is used
- unified proxies configuration (use environment vars)
- SAWarning/DeprecationWarning removed
- Fix mirroring link for external files
- Mirror the download_url of the release file
- Handle hiphen and underscore [fizyk]
- Enhance mimetime handling [fizyk]
- Bugfixes and DeprecationWarning removed
- Fix package version comparison
- Handle bdist_wheel format
To handle the wheel format, (some package like Twisted 13 use it), for previous install, you must run a migration script like this.
$ pyshop_migrate development.ini 0.7.5
The sqlite database file will be altered, YOU MAY backup it before run the script.
- Fix local package usage (broken since 0.7.1) [fizyk]
- Remove all certificates and extra handling for PyPI validation as PyPI now uses a certificate that can be validated without these. [disko]
- Remove unused certificates for pypi validation
- Securize download from pypi by forcing https and validate certificate pypi.python.org certificate chain is embed in the pyshop package
- Fix package order on web page
- Satanize version number on upload. This is configurable with settings pyshop.upload.satanize and pyshop.upload.satanize.regex
- Settings pyshop.satanize and pyshop.satanize.regex have been renamed to pyshop.mirror.satanize and pyshop.mirror.satanize.regex
- Fix first connection of the web application
- Fix the usage of http proxy (forcing request version)
- Add Link to display all release versions
- Improve navigation
- Fix ugly version number sorting
- Fix release file upgrade (allow developper to override release file)
- Rename user views to account
- Add view to let the connected user to update his account
- The setting pyshop.satanize.reg has been renamed to pyshop.satanize.regex
- The setting cookie_key has been renamed to pyshop.cookie_key
- Fix bug on package upload. don’t close the stream while writing it.
- Add basic tests on packages view
- work with pip, setuptools
- mirror packages
- upload packages
- secure access with login/password
- create/update accounts
- tests for python 2.7 only
- compatible with python 2.6