Python Compose, a tool for spinning up many long-running Python services simultaneously.
Project description
Python Compose
Python Compose is a command line tool and library for spinning up many long-running python applications in isolated Python environments. Inspired by Docker Compose, Python Compose is meant to provide similar functionality without having to relying on Docker containers, networking etc.
Currently, Python Compose supports the following environments:
In the future, we wish to support:
Installation
Use the package manager pip to install python-compose
.
pip install python-compose
Currently, Python Compose supports venv
, pyenv-virtualenv
and conda
environments. The pyenv-virtualenv
and conda
managers are not installed with Python Compose. To install them, follow instructions from the pyenv-virtualenv and conda documentation.
Concepts
Unit
A unit
is an object which represents how to create, run and clean up a service for a specific type of Python environment handler. Examples can be found in the python_compose/unit directory.
YAML Specification
Python Compile can currently only be instantiated from a YAML file with specification derived from the internal spec.py Python file. A Python Compose YAML file expects a top-level key, units
which maps to a list of units
i.e. Python scripts to run. Each unit's environment is described by the key unit_type
. A unit will have different arguments. Some required, some are optional. Again, these definitions are in spec.py
Alternatives Comparison
Python Compose isn't the only tool in the world for this task. It happens to be a convenient one if you're working in a Python only environment and want to minimize external tools and dependencies. Below is a chart comparing Python Compose versus other solutions.
Tool | Operating Systems | Virtual Networking | Ability to Run Non-Python Programs | Multiple Python Version Support | Non-Python Dependencies Required |
---|---|---|---|---|---|
Python Compose | Linux/MacOS | ❌ | With Conda or subprocess | ✅ | ❌ |
Python Multiprocessing | Linux/MacOS/Windows | ❌ | With subprocess | ❌ | ❌ |
Docker Compose | Linux/MacOS/Windows | ✅ | ✅ | ✅ | ✅ |
Systemd Unit Files | Linux | ❌ | ✅ | ✅ | ✅ |
Bazel | Linux/MacOS/Windows | ❌ | ✅ | ✅ | ✅ |
Example
We provide many examples of how to use Python Compose as both a library and as a command line tool in the examples
directory. Below we provide what we will assume is the most Docker Compose-like use case, starting many backend services from a single config file.
In a directory, create a Python script, httpd.py
that starts an HTTP service:
import argparse
from http.server import BaseHTTPRequestHandler, HTTPServer
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("port", type=int)
args = parser.parse_args()
print(args)
server_address = ("", args.port)
httpd = HTTPServer(server_address, BaseHTTPRequestHandler)
httpd.serve_forever()
Then, create a yaml file, config.yaml
to start this backend service in multiple environments:
units:
- unit_type: "venv"
name: "httpd_0"
env_dir: "./.envs"
script_path: "./httpd.py"
script_args: ["8080"]
- unit_type: "venv"
name: "httpd_1"
env_dir: "./.envs"
script_path: "./httpd.py"
script_args: ["8081"]
- unit_type: "pyenv-virtualenv"
name: "httpd_2"
py_version: "3.10"
script_path: "./httpd.py"
script_args: ["8082"]
- unit_type: "conda"
name: httpd_3
script: ["python3", "../conda/httpd.py", "8083"]
To spin up all containers, run
python3 -m python-compose config.yaml
To exit, press <Ctrl-C>
.
Contributing
Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.
Please make sure to update tests as appropriate.
Setting up Your Development Environment.
Developers should first install all needed development dependencies:
pip3 install -e '.[dev]'
Then make all desired changes
Adding a new Unit
To add a new unit
, you will want to first add the unit to python_compose/unit. Then, add an example to the examples directory and finally update spec.py.
Testing
To test Python Compile, run pytest
after installation of all dev dependencies.
License
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file python_compose-1.2.0.tar.gz
.
File metadata
- Download URL: python_compose-1.2.0.tar.gz
- Upload date:
- Size: 12.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: python-httpx/0.24.0
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | fb7668b55f0f242c570b0280e0e54923b54b6518a33ded53dbe455a7b708426e |
|
MD5 | d8e9ff776b6f397fed9129cc6f1182e9 |
|
BLAKE2b-256 | a0199067335c67d27af9a3aa3597b1940a62fd9e11b465e6931744969d695d55 |
File details
Details for the file python_compose-1.2.0-py3-none-any.whl
.
File metadata
- Download URL: python_compose-1.2.0-py3-none-any.whl
- Upload date:
- Size: 12.4 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: python-httpx/0.24.0
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9865e22770711c9fe285baab45c7211a70d20cd6c1af0c69e6784b59fede74fb |
|
MD5 | 54fab7a22aa1944c358d56170a263676 |
|
BLAKE2b-256 | 52e1fac1f54cdf039b30ffc4ac12330d3032f97a153f4a9d3c318681d1db58ba |