Skip to main content

An opinionated CLI tool for Python monorepo MGMT.

Project description

Scream

Python 2.7 Python 3.6 License: MIT

What you do when you see a bunch of python... packages.

or...

An opinionated CLI tool for Python monorepo MGMT.


It's objective is to ease the creation, testing, and deploying of multiple python packages in a single repository. To ensure non-overlapping names with PYPI, this tool forces you to use namespace packages. Namespaces are defined according to python's pkgutil-style.

Why use Scream?

  • Scream holds all packages to the same standards

    • Enforces consistent styling.
    • Enforces consistent testing.
    • Enforces consistent documentation.
  • Uses tox to setup virtualenvs for isolated testing across python versions.

  • Pre-commit hooks to help prevent those gosh darn mistakes.

Problems with most monorepos

  1. Testing & CI/CD pipeline can become slow with many packages.

    The other available solutions are:

    These tools are extremely powerful, but sometimes are overkill, and introduce a fair amount of overhead to manage.

  2. Installing intra-monorepo package dependencies is hard with a private repositories.

    The available solutions are:

    • Have all your packages distributed publically.
    • Host a private PYPI index.
    • pip install using dependency links.

How scream works

Scream aims to cause as little overhead as possible for managing your monorepo. No fancy third party configuration or private PYPI repositories.

Scream uses the existing python packaging requirements to resolve intra-monorepo dependencies, and git to detect what's changed since the parent branch.

For example:

cat setup.cfg.

[metadata]
name = company_packageA
version = 0.0.1
description = Your package description!

[options]
packages = find:
install_requires =
    company_packageB

If you make a change to company_packageA then run tests...

> scream test --dry-run

The following packages have changes compared since branch: `master`:
        company_packagea

Packages that require testing:
        company_packagea
        company_packageb

Commands

  • scream new <package_name> - Creates new template package.
  • scream test [--dry-run][--all] - Tests packages and package dependents that have changed.
  • scream install <package_name> - Installs a package.
  • scream build - Builds a python wheel and bundles it with all it's dependencies as wheels.

Quickstart

By default packages are tested against python 3.6.x, which means you have it available on your PATH. If you about different versions, please see the configuration options.

> mkdir mymonorepo
> cd mymonorepo

> scream init
Done!
Create a new package with `scream new <namespace>.<package_name>`

> scream new com.packagea
Created project `com.packagea`

> scream new com.packageb
Created project `com.packageb`

> scream test --all
Running tests...

> coverage report

Using your monorepo packages

The two common ways you would install packages from this monorepo are:

  1. Using scream build in your CI tool to build packages and ship them to your machines.
  2. Standard pip install individual packages to any machine from private or public github repos.

If your repository is public, you can simply install a subpackage anywhere using:

pip install 'git+ssh://git@github.com/ORG/REPO.git@master#subdirectory=examplea'

If your repository is private, you need a few extra steps to make sure packages that depend on other packages in this monorepo can be installed.

  1. Specify dependency_links in the setup.cfg for each 'local' dependency:
dependency_links =
    git+ssh://git@github.com/ORG/REPO.git@master#egg=examplea-0#subdirectory=subpackages/examplea
  1. pip install as before, but specifiy the flag --process-dependency-links
pip install 'git+ssh://git@github.com/ORG/REPO.git@master#subdirectory=examplea' --process-dependency-links

Configuration

  1. In your packages setup.cfg the variable python_requires determines which versions of python your package will be tested against.
  • Ex. python_requires = 2.7, 3.6, 3.7

    Note: the python versions you intend to test must be available on your path.

Upcoming Features

  • Auto version handling.
  • Pre commit hook to flake8.
  • Pre commit hook to avoid committing a large file.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

scream-0.0.12.tar.gz (17.4 kB view hashes)

Uploaded Source

Built Distribution

scream-0.0.12-py2.py3-none-any.whl (21.7 kB view hashes)

Uploaded Python 2 Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page