Skip to main content

pyprojen

A minimal, Python-centric port of projen.

Apache 2.0 License Commit activity Projen debut demo


Overview

Project templates are great... except that they promote "code duplication at scale" and all the technical debt that come with that.

projen (the tool this project is based on) represents a new generation of project template tools. projen helps you manage the "day 2 problem" for project templates, to avoid "template drift".

projen also helps you treat project configuration as an abstraction, whose implementation details can be changed/improved, without disrupting the developer workflow. Elad Ben-Israel, creator of projen, AWS CDK, and Winglang gave a talk on this at CDK day 2020.

  1. ⚡️ Agility. Centrally push unit-tested updates to all existing projects created from your templates in minutes, not months.
  2. 🧱 Modularity. Define composable, parameterized "components" that you can add/remove from your project as needed.

Real Examples

pyprojen is a minimal port of projen's core functionality to Python.

Specifically, it gets you

  1. ⚡️ Agility
  • Examples:
    • 10x the speed of CI for all repos overnight by making CI steps run in parallel instead of serially
    • Update a team linting rule in pyproject.toml, ruff.toml, etc.
    • Add auth steps to publish to or install from a private PyPI registry in CI or Dockerfiles
    • Completely change CI systems with minimal disruption, e.g. switch from Bitbucket Pipelines to GitHub Actions to AWS CodeBuild and back
  1. 🧱 Modularity
  • Examples:
    1. define an opinionated PythonPackage component, and
    2. layer on top a
      • FastAPIApp,
      • StreamlitApp,
      • CdkApp,
      • PulumiApp,
      • AirflowDag,
      • DagsterDag,
      • BentoMLService,
      • AwsLambdaPythonFunction,
      • etc.
    3. Add (or remove) as many of these to your repo as you like, whenever you like, and find these packages instantly set up with CI, linting, formatting, tests, packaging, publishing, deploying, etc.
      • For example, you might incrementally develop a "mini data science app monorepo" with
        1. a MetaflowDag that trains a model
        2. served in a FastAPI app
        3. which you can interact with via a Streamlit app
        4. both of which are deployed by an AwdCdk app
      • all of these components would ideally be instantly set up with CI, linting, formatting, testing, packaging, publishing, deployment, etc. the moment they are added. And "un set up" with those if they are removed.

Q&A

How does (py)projen compare to copier or cookiecutter?

Click to expand

copier is a reaction to cookiecutter, built to allow template updates to be propagated to existing projects.

  1. Migration. The migration process for (py)projen is a single CLI command (python .pyprojenrc.py). Whereas the migration process for copier is a bit more manual and prone to errors.

  2. Composable components. copier is more like cookiecutter in that it uses Jinja templates to generate a certain set of files. (py)projen lets you define re-usable components. You can add arbitrary numbers of these components to your project with different parameters and remove them just as easily.

That said, although cookiecutter and copier are more limited, they are also simpler.

What is the difference between pyprojen and projen?

Click to expand

1. Fundamentally, projen offers a few things:

  1. A Project, Component, and Construct abstraction that lets you define reusable components that you can push updates to.
  2. Primitive components like TextFile, YamlFile, JsonFile, MarkdownFile, etc. that you can compose to build "higher-level components".
  3. A library of opinionated, higher-level components like PythonProject, TypescriptProject, DockerCompose, GithubWorkflow, ...
  4. An opinionated "task runner" system (think Makefile/Justfile, poetry scripts, etc.) to define project-related commands.
  5. A projen new command which creates the initial .projenrc.py config file for your project

2. pyprojen implements [1] and [2] from the list above (the unopinionated parts).

It is up to you to create your own components with your own opinions on things like

  • when, if, and how to manage virtual environments, e.g. uv, pip, conda, etc.
  • which linter/formatter to use, e.g. ruff, pre-commit, etc.
  • how to structure single- and multi-package repos (monorepos) and CI for them

Coming from tools like cookiecutter or copier, many people/teams prefer than using off-the-shelf templates or components.

3. pyprojen is not a drop-in replacement for projen, but it tries to get close.

If you write components in Python using pyprojen, it should be easy to move them over to the projen Python bindings if you decide to.

Should I use pyprojen or projen?

Click to expand

TL;DR Bias towards projen, unless you

  1. Want a Python-first dev experience, and
  2. Prefer to fully-define your own project template/components rather than using projen's existing project templates, higher-level components, or task runner system

projen is a larger project and is primarily maintained by developers at AWS. projen,

But to develop with projen, you either need to write TypeScript, or use generated Python bindings that invoke TypeScript.

If you are familiar with writing AWS CDK in Python, developing with projen in Python is a similar experience, because they both use Python bindings generated from TypeScript using the JSII project.

This means:

  1. Not all internals of the original TS/JS code is exposed in the TS bindings, e.g. private attributes. You can unexectedly hit dead ends when attributes or methods that are available in TS are simply not available in Python.
  2. Step debugging is limited. The bindings are thin wrappers around a tool that invokes the original TypeScript/JavaScript code
  3. Errors raised by python bindings are cryptic and difficult to parse.
  4. Autocompletion is poor
  5. You need to have NodeJS installed on your system and in CI
  6. The JSII is a bit slow. (seconds not milliseconds)

📌 Since projen's Python bindings and pyprojen have a nearly identical API, you could switch to projen if you want to join the larger TypeScript-based ecosystem.

Quick start (TODO)

pip install pyprojen
from pyprojen import ...

Developing/Contributing

System requirements

You will need the following installed on your machine to develop on this codebase

  • make AKA cmake, e.g. sudo apt-get update -y; sudo apt-get install cmake -y
  • Python 3.7+, ideally using pyenv to easily change between Python versions
  • git

# clone the repo
git clone https://github.com/<your github username>/pyprojen.git

# install the dev dependencies
make install

# run the tests
make test

Metadata

Release files for pyprojen 0.0.4

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for pyprojen 0.0.4
File Size Uploaded Provenance
pyprojen-0.0.4.tar.gz 27.0 kB Without provenance Details

Built distribution (wheel)

Table of built distributions (wheels) for pyprojen 0.0.4
File Interpreter ABI Platform
pyprojen-0.0.4-py3-none-any.whl Python 3 none any Details

Total release size: 56.8 kB

Release files / pyprojen-0.0.4.tar.gz

Download URL pyprojen-0.0.4.tar.gz
Size 27.0 kB
Tags Source
SHA-256 checksum
How to use checksums
bf6de0a82958f7ca07447a234dee2e8987134362b3551681c6f74c33cecb29d1
BLAKE2b-256 checksum
How to use checksums
d5c1902ceca28ee98bd9a98cf226f3882d60672030d89e735ae6d2331c61394b
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.1.1 CPython/3.8.18

Release files / pyprojen-0.0.4-py3-none-any.whl

Download URL pyprojen-0.0.4-py3-none-any.whl
Size 29.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
fd676f2dbefa260ea2d89ec49e820862f8eca81a2bcff3c10fbcb61cc3006d23
BLAKE2b-256 checksum
How to use checksums
8e7641d071711cd6a276fcde5230bfefa053d8ba92910b549c162ab84b6bedb7
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.1.1 CPython/3.8.18

Security for pyprojen 0.0.4

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page