devpi-server 5.3.0

devpi-server: reliable private and pypi.org caching server

devpi-server: server for private package indexes and PyPI caching

PyPI cache

You can point pip or easy_install to the root/pypi/+simple/ index, serving as a transparent cache for pypi-hosted packages.

User specific indexes

Each user (which can represent a person, project or team) can have multiple indexes and upload packages and docs via standard twine or setup.py invocations. Users and indexes can be manipulated through devpi-client and a RESTful HTTP API.

Index inheritance

Each index can be configured to merge in other indexes so that it serves both its uploads and all releases from other index(es). For example, an index using root/pypi as a parent is a good place to test out a release candidate before you push it to PyPI.

Good defaults and easy deployment

Get started easily and create a permanent devpi-server deployment including pre-configured templates for nginx and process managers.

Separate tool for Packaging/Testing activities

The complementary devpi-client tool helps to manage users, indexes, logins and typical setup.py-based upload and installation workflows.

See https://doc.devpi.net on how to get started and further documentation.

Support

If you find a bug, use the issue tracker at Github.

For general questions use the #devpi IRC channel on freenode.net or the [e-mail] mailing list.

For support contracts and paid help contact merlinux.eu.

Changelog

5.3.0 (2019-12-03)

Features

• fix #732: add --unix-socket-perms option to set Unix socket permissions when a Unix socket is in use, defaults to ‘600’ if not specified, ignored if --unix-socket is not used.

• proxy requests from the replica to the master will use number of seconds from the new --proxy-timeout option as their timeout value.

Bug Fixes

• ignore obsolete pypi_whitelist index setting when sent by devpi-client and remove it when an index config is updated.

5.2.0 (2019-10-18)

Deprecations and Removals

• The --export option is deprecated, use the new devpi-export command instead.

• The --gen-config option is deprecated, use the new devpi-gen-config command instead.

• The --import option is deprecated, use the new devpi-import command instead.

• The --init option is deprecated, use the new devpi-init command instead.

• The --passwd option is deprecated, use the new devpi-passwd command instead.

Features

• fix #253: add --unix-socket option to listen on a unix socket. When this is used the --host and --port options are forbidden.

• Add new option mirror_use_external_urls for mirrors which when true redirects to the original release URL when the release isn’t locally cached yet. No new releases are stored locally from that point on. The metadata is still stored in the database.

• Add devpi-fsck command which compares the hash of files with the hash in the database.

• Metrics for the sqlite storage cache are exposed in the JSON output of the +status view.

• Storage options can now be specified as a dict in yaml config files.

Bug Fixes

• fix #403: extract auth credentials from --master-url to prevent them from leaking in logs and +status.

• fix #545: provide proper error message when there is an exception during push to an external repository.

• fix #686: prevent name clashes for toxresults by including the timestamp in the filename.

• fix #722: prevent bogus “failed to check mirror whitelist” warnings.

• Fix possible race condition when writing files.

• Fix possible assertion error if importing multiple changes in a replica fails in the middle and fetching a single change set is tried as fallback.

• For plugins the offline attribute of mirror stage instances now works independently of the --offline-mode command line option. This only applies to _perstage methods.

• Files created in a transaction are written directly to temporary files instead of being kept in memory until commit.

• Unnecessary database writes where the final value didn’t change are now prevented.

Other Changes

• The timeout when fetching the list of remote projects for a mirror index is set to a minimum of 30s by default and to 60s when running as replica. Other fetches of mirrors still use the timeout specified via --request-timeout.

5.1.0 (2019-08-05)

Features

• Allow stage customizer plugins to filter projects and versions.

• Replicas will use the multiple changelog endpoint added in devpi-server 4.9.0 to reduce the number of requests necessary to synchronize state.

5.0.0 (2019-06-28)

Deprecations and Removals

• fix #518: There are no URLs on PyPI anymore that need to be scraped or crawled, so the code for that was removed.

• removed support for long deprecated acl_upload and bases mirror index option. They were only kept for compatibility with devpi-client <= 2.4.1.

• the --start, --stop, --status and --log options are deprecated. Use --gen-config to create example configuration files for various process managers.

• removed long deprecated pypi_whitelist index option. It was only kept for compatibility with devpi-client <= 2.4.1.

• deprecated Python 2.7 support. This is the last major version supporting Python 2.7. For upgrading to Python 3.x you have to export your data using your current setup with Python 2.7 and import it in a new installation with Python 3.x.

Features

• fix #249: unknown keys for index configuration now result in an error instead of being silently ignored.

• fix #625: project registration is now optional. A file upload with twine or setuptools will automatically register the project.

• fix #636: support ignore_bases argument for project listings.

• support :AUTHENTICATED: for permissions. This resolves to any user which is logged in, regardless of username or groups.

• added experimental support for stage customizers to let plugins add index types with customized behaviour. See BaseStageCustomizer in model.py for the API and devpiserver_get_stage_customizer_classes for the registration.

• support no_projects argument for index json requests. The list of projects will not be added to the result.

• when credentials for the user are rejected, the error message now says so instead of claiming the user could not be found.

Other Changes

• boolean values can now only be set via the following values: ‘false’, ‘no’, ‘true’, ‘yes’ and actual booleans in the REST API. Before any string not matching ‘false’ and ‘no’ was converted into boolean true.

• the default logging configuration now outputs to stdout instead of stderr.

• major releases don’t require an export/import cycle anymore except when explicitly announced. You should always make a backup though! When upgrading to devpi-server 5.0.0 you can keep the state as is and even downgrade to the last 4.9.x release if necessary. Don’t forget to backup before upgrades!

• the server secret isn’t automatically persisted for new installations. A server restart invalidates login tokens. An existing installation will still use it’s stored secret, but log a warning. Use --secretfile to explicitly specify a persistent secret file.

• the --storage option is now required when a storage plugin like devpi-postgresql is in use. It’s recommended to use a configuration file for devpi-server to have everything in one place (see --configfile).

• for the --logger-cfg yaml loading we now use safe_load of ruamel.yaml instead of load from pyyaml.

4.9.0 (2019-04-26)

Features

• implement #93: When creating a user, the password hash can be set directly with pwhash. Upon database initialization allow setting root user password with --root-passwd and the password hash with --root-passwd-hash options. Thanks to Andreas Palsson.

• decouple devpi server version from database version to enable major releases that do not require export import of data

• support --hard-links option during import for releases and doc zips.

• added new endpoint to download multiple changelog entries at once. This will be used for faster replication in the future.

• add option --replica-file-search-path to point to existing files. If a match is found it will be copied locally instead of fetched from the master. These files could be from a previous replication attempt or separately copied/restored.

• add --hard-links support for replicas together with the --replica-file-search-path option. When a matching file is found it’s hard linked instead of writing a copy.

Bug Fixes

• fix multiple triggering of mirror project names initialization.

• fix updating time stamp of mirror project name cache when no project names have changed. This makes subsequent fetches actually use the cache instead of always fetching the data again from the mirror.

• use timeout when waiting for data from master in replica on mirror simple pages.

Other Changes

• slightly improved replica performance by removing unnecessary DB read and using fewer transactions.