pypicloud 1.3.12

Private PyPI backed by S3

PyPI Cloud

WARNING: Pypicloud has transitioned to maintenance mode. For details, see #325

Build:

Documentation:

PyPI:

This package is a Pyramid web app that provides a PyPI server where the packages are stored on Amazon’s Simple Storage Service (S3), Google’s Cloud Storage (GCS) or Azure’s Blob Storage.

Quick Start

pip install pypicloud[server]
pypicloud-make-config -t server.ini
pserve server.ini

Go to http://localhost:6543/ to view the web interface.

For more detailed step-by-step instructions, check out the getting started section of the docs.

Docker

There is a docker image if you’re into that sort of thing: https://github.com/stevearc/pypicloud-docker

Changelog

If you are upgrading an existing installation, read the instructions

1.3.12 - 2022/12/29

• Fix errors with the GCS backend

• Fix error when ‘us-east-1’ storage region

• Reduce unnecessary HTTP redirects

1.3.11 - 2022/9/20

• Sort package names from redis backend

• Deprecate pypi.allow_delete and pypi.allow_overwrite for pypi.allow_delete_groups and pypi.allow_overwrite_groups

1.3.10 - 2022/9/3

• Add uploader to package metadata

• Normalize whitespace in S3 metadata

1.3.9 - 2022/8/22

• Stream files to storage backend for lower memory usage

1.3.8 - 2022/8/11

• Add Redis Cluster support

1.3.7 - 2022/7/13

• Fix login issues in the web interface

1.3.6 - 2022/7/7

• Allow / to serve packages the same as /simple

1.3.5 - 2022/6/20

• Handle all errors when fetching from upstream

1.3.4 - 2022/4/30

• Fix files storage backend on Windows

1.3.3 - 2021/11/12

• Add db.poolclass to configure SQLAlchemy connection pooling

1.3.2 - 2021/10/16

• Fix exception in JSON endpoint

1.3.1 - 2021/10/12

• Remove trailing slash from JSON scraper

1.3.0 - 2021/10/9

• Allow config options to be overridden by environment variables

1.2.4 - 2021/6/10

• Fix missing permissions for non-admin users

1.2.3 - 2021/6/9

• Add Pyramid>=2.0 to dependencies

1.2.2 - 2021/6/8

• Upgrade to Pyramid 2.0

• Remove the SQL index from package summary field (will take effect when you rebuild your cache, but a rebuild is not required)

1.2.1 - 2021/5/18

• Fix a XSS vulnerability

• Remove storage limit of package summary (will take effect when you rebuild your cache, but a rebuild is not required unless you hit this issue)

1.2.0 - 2021/3/1

• Add more package info to JSON API

• Stop normalizing metadata for Azure

• Provide Azure credentials via environment variable

• Pin the Pyramid version to avoid deprecation

• Dropping support for Python 3.5 and 3.6 due to difficulty with cryptography library

1.1.7 - 2020/11/16

• Fix a datetime crash when reloading the cache

• Fix a logic error with db.graceful_reload

1.1.6 - 2020/11/7

• Fix content-type when streaming packages

• JSON scraper doesn’t throw exceptions if it receives a HTTP error

• Add config option for GCS IAM signing email

1.1.5 - 2020/9/19

• Add pypi.allow_delete to disable deleting packages

1.1.4 - 2020/9/13

• Fix concurrency bugs in GCS backend

1.1.3 - 2020/8/17

• Fix metadata storage issue with some S3-compatible backends

• Command line arg to generate password hash from stdin

1.1.2 - 2020/7/23

• Fix error when package in local storage but not in fallback repository

1.1.1 - 2020/6/14

• Fix an exception when pypi.use_json_scraper = false

• Allow passing in auth.signing_key as an environment variable

• Add some documentation about the DynamoDB cache

1.1.0 - 2020/5/31

• Drop support for Python 2

• Add support for package hashes

1.0.16 - 2020/5/20

• Add support for Microsoft Azure Blob storage

1.0.15 - 2020/5/8

• Add requests as a dependency

1.0.14 - 2020/5/7

• Fix a bug with reloading Redis cache

• More graceful handling of non-package files in GCS

• Support for requires_python metadata

• Add pypi.use_json_scraper setting for configuring

• Change default value of storage.redirect_urls to True

• Add auth.scheme setting to customize password hashing algorithm

• SIGNIFICANTLY LOWERED default password hashing rounds. Read about why in the docs

1.0.13 - 2020/1/1

• Add option to use IAM signer on GCS

1.0.12 - 2019/12/11

• Change default fallback url from http://pypi.python.org to https://pypi.org

• Add pypi.disallow_fallback option to disable fallback for specific packages

• Fix automatic bucket creation for all S3 regions

1.0.11 - 2019/4/5

• Add ability to stream files through pypicloud

• Support spaces in auth.ldap.admin_value values

1.0.10 - 2018/11/26

• Strip non-ASCII characters from summary for S3 backend

• Fix an issue with production log format

• Add auth.ldap.fallback to use config file configure groups and permissions with LDAP access backend

1.0.9 - 2018/9/6

• Fix: Exception during LDAP reconnect

• Fix: LDAP on Python 3 could not detect admins

• Feature: New pypi.auth.admin_group_dn setting for LDAP (for when memberOf is unavailable)

1.0.8 - 2018/8/27

• Feature: Google Cloud Storage support

1.0.7 - 2018/8/14

• Feature: /health endpoint checks health of connection to DB backends

• Feature: Options for LDAP access backend to ignore referrals and ignore multiple user results

• Fix: Exception when storage.cloud_front_key_file was set

• Fix: Bad redirect to the fallback url when searching the /json endpoint

• Deprecation: pypi.fallback_url has been deprecated in favor of pypi.fallback_base_url

1.0.6 - 2018/6/11

• Fix: Support auth.profile_name passing in a boto profile name

• Fix: Uploading package with empty description using twine crashes DynamoDB backend

• Fix: Config file generation for use with docker container (using %(here)s was not working)

• Use cryptography package instead of horrifyingly old and deprecated pycrypto

• Add storage.public_url to S3 backend

1.0.5 - 2018/4/24

• Fix: Download ACL button throws error in Python 3

• New access backend: AWS Secrets Manager

• Add storage.storage_class option for S3 storage

• Add db.tablenames option for DynamoDB cache

• Reduce startup race conditions on empty caches when running multiple servers

1.0.4 - 2018/4/1

• Fix: Fix SQL connection issues with uWSGI

• Miscellaneous python 3 fixes

1.0.3 - 2018/3/26

• Fix: uWSGI hangs in python 3

• Fix: Crash when using ppc-migrate to migrate from S3 to S3

• Add warnings and documentation for edge case where S3 bucket has a dot in it

• Admin can create signup tokens

1.0.2 - 2018/1/26

• Fix: Hang when rebuilding Postgres cache

• Fix: Some user deletes fail with Foreign Key errors

• Fix: Incorrect parsing of version for wheels

• Configuration option for number of rounds to use in password hash

• Make request errors visible in the browser

• Add a Create User button to admin page

• SQL access backend defaults to disallowing anonymous users to register

1.0.1 - 2017/12/3

• Support for LDAP anonymous bind

• Fix a crash in Python 3

1.0.0 - 2017/10/29

• Python3 support thanks to boto3

• Removing stable/unstable version from package summary

• Changing and removing many settings

• Performance tweaks

• graceful_reload option for caches, to refresh from the storage backend while remaining operational

• Complete rewrite of LDAP access backend

• Utilities for hooking into S3 create & delete notifications to keep multiple caches in sync

NOTE Because of the boto3 rewrite, many settings have changed. You will need to review the settings for your storage, cache, and access backends to make sure they are correct, as well as rebuilding your cache as per usual.

0.5.6 - 2017/10/29

• Add storage.object_acl for S3

0.5.5 - 2017/9/9

• Allow search endpoint to have a trailing slash

0.5.4 - 2017/8/10

• Allow overriding the displayed download URL in the web interface

• Bump up the DB size of the version field (SQL-only)

0.5.3 - 2017/4/30

• Bug fix: S3 uploads failing from web interface and when fallback=cache

0.5.2 - 2017/4/22

• Bug fix: The /pypi path was broken for viewing & uploading packages

• Update docs to recommend /simple as the install/upload URL

• Beaker session sets invalidate_corrupt = true by default

0.5.1 - 2017/4/17

• Bug fix: Deleting packages while using the Dynamo cache would sometimes remove the wrong package from Dynamo

0.5.0 - 2017/3/29

Upgrade breaks: SQL caching database. You will need to rebuild it.

• Feature: Pip search works now

0.4.6 - 2017/4/17

• Bug fix: Deleting packages while using the Dynamo cache would sometimes remove the wrong package from Dynamo

0.4.5 - 2017/3/25

• Bug fix: Access backend now works with MySQL family

• Bug fix: Return http 409 for duplicate upload to work better with twine

• Bug fix: Show upload button in interface if default_write = everyone

• Confirm prompt before deleting a user or group in the admin interface

• Do some basica sanity checking of username/password inputs

0.4.4 - 2016/10/5

• Feature: Add optional AWS S3 Server Side Encryption option

0.4.3 - 2016/8/2

• Bug fix: Rebuilding cache always ends up with correct name/version

• Feature: /health endpoint (nothing fancy, just returns 200)

0.4.2 - 2016/6/16

• Bug fix: Show platform-specific versions of wheels

0.4.1 - 2016/6/8

• Bug fix: LDAP auth disallows empty passwords for anonymous binding

• Config generator sets pypi.default_read = authenticated for prod mode

0.4.0 - 2016/5/16

Backwards incompatibility: This version was released to handle a change in the way pip 8.1.2 handles package names. If you are upgrading from a previous version, there are detailed instructions for how to upgrade safely.

0.3.13 - 2016/6/8

• Bug fix: LDAP auth disallows empty passwords for anonymous binding

0.3.12 - 2016/5/5

• Feature: Setting auth.ldap.service_account for LDAP auth

0.3.11 - 2016/4/28

• Bug fix: Missing newline in config template

• Feature: pypi.always_show_upstream for tweaking fallback behavior

0.3.10 - 2016/3/21

• Feature: S3 backend setting storage.redirect_urls

0.3.9 - 2016/3/13

• Bug fix: SQL cache works with MySQL

• Feature: S3 backend can use S3-compatible APIs

0.3.8 - 2016/3/10

• Feature: Cloudfront storage

• Bug fix: Rebuilding cache from storage won’t crash on odd file names

0.3.7 - 2016/1/12

• Feature: /packages endpoint to list all files for all packages

0.3.6 - 2015/12/3

• Bug fix: Settings parsed incorrectly for LDAP auth

0.3.5 - 2015/11/15

• Bug fix: Mirror mode: only one package per version is displayed

0.3.4 - 2015/8/30

• Add docker-specific option for config creation

• Move docker config files to a separate repository

0.3.3 - 2015/7/17

• Feature: LDAP Support

• Bug fix: Incorrect package name/version when uploading from web

0.3.2 - 2015/7/7

• Bug fix: Restore direct links to S3 to fix easy_install

0.3.1 - 2015/6/18

• Bug fix: pypi.allow_overwrite causes crash in sql cache

0.3.0 - 2015/6/16

• Fully defines the behavior of every possible type of pip request. See Fallbacks for more detail.

• Don’t bother caching generated S3 urls.

0.2.13 - 2015/5/27

• Bug fix: Crash when mirror mode serves private packages

0.2.12 - 2015/5/14

• Bug fix: Mirror mode works properly with S3 storage backend

0.2.11 - 2015/5/11

• Bug fix: Cache mode will correctly download packages with legacy versioning

• Bug fix: Fix the fetch_requirements endpoint

• Bug fix: Incorrect expire time comparison with IAM roles

• Feature: ‘mirror’ mode. Caches packages, but lists all available upstream versions.

0.2.10 - 2015/2/27

• Bug fix: S3 download links expire incorrectly with IAM roles

• Bug fix: fallback = cache crashes with distlib 0.2.0

0.2.9 - 2014/12/14

• Bug fix: Connection problems with new S3 regions

• Usability: Warn users trying to log in over http when session.secure = true

0.2.8 - 2014/11/11

• Bug fix: Crash when migrating packages from file storage to S3 storage

0.2.7 - 2014/10/2

• Bug fix: First download of package using S3 backend and pypi.fallback = cache returns 404

0.2.6 - 2014/8/3

• Bug fix: Rebuilding SQL cache sometimes crashes

0.2.5 - 2014/6/9

• Bug fix: Rebuilding SQL cache sometimes deadlocks

0.2.4 - 2014/4/29

• Bug fix: ppc-migrate between two S3 backends

0.2.3 - 2014/3/13

• Bug fix: Caching works with S3 backend

0.2.2 - 2014/3/13

• Bug fix: Security bug in user auth

• Bug fix: Package caching from pypi was slightly broken

• Bug fix: ppc-migrate works when migrating to the same storage type

0.2.1 - 2014/3/12

• Bug fix: Pre-existing S3 download links were broken by 0.2.0

0.2.0 - 2014/3/12

Upgrade breaks: caching database

• Bug fix: Timestamp display on web interface

• Bug fix: User registration stores password as plaintext

• Feature: ppc-migrate, command to move packages between storage backends

• Feature: Adding support for more than one package with the same version. Now you can upload wheels!

• Feature: Allow transparently downloading and caching packages from pypi

• Feature: Export/Import access-control data via ppc-export and ppc-import

• Feature: Can set default read/write permissions for packages

• Feature: New cache backend: DynamoDB

• Hosting all js & css ourselves (no more CDN links)

• Obligatory miscellaneous refactoring

0.1.0 - 2014/1/20

• First public release