BuildFlow, is an open source framework for building large scale systems using Python. All you need to do is describe where your input is coming from and where your output should be written, and BuildFlow handles the rest.
Project description
⚒️ BuildFlow
Overview
BuildFlow, is an open source framework for building large scale systems using Python. All you need to do is describe where your input is coming from and where your output should be written, and BuildFlow handles the rest. No configuration outside of the code is required.
Key Features (all provided out-of-the-box):
- Automatic resource creation / management (Infrastructure as Code) powered by Pulumi (Infrastructure from Code)
- Automatic parallelism & concurrency powered by Ray
- Dynamic autoscaling: scale up during high traffic / reduce costs during low traffic
- Schema validation powered by Python dataclasses and type hints
Installation
pip install buildflow
Extra Dependencies
Pulumi Installation
BuildFlow uses Pulumi to manage resources used by your BuildFlow Nodes and Processors. To install Pulumi visit: https://www.pulumi.com/docs/install/
Installing Pulumi unlocks:
- allows BuildFlow to manage resource creation and destruction
- full access to Pulumi API / CLI
- fine-grained control over Pulumi Stacks & Resources
Quick Links
- Docs: https://www.buildflow.dev/docs
- Walkthroughs: https://www.buildflow.dev/docs/walkthroughs/realtime-image-classification
- Discord: https://discordapp.com/invite/wz7fjHyrCA
- Contribute: https://www.buildflow.dev/docs/contribute
Code Health Checks
We use black and ruff with pre-commit hooks to perform health checks. To setup these locally:
- Clone the repo
- Install the
dev
dependencies like `python -m pip install .[dev] - Check if pre-commit is installed correctly by running
pre-commit --version
- Setup pre-commit to run before every commit on staged changes by running
pre-commit install
- Pre-commit can also be ran manually as
pre-commit run --all-files
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for buildflow-0.2.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e29a008628c4a999a2e05ca32c362748f0257c930ff6138dcb6267370b0d924b |
|
MD5 | 4526c3f28d61ff60645b34d8c1e53d92 |
|
BLAKE2b-256 | ab0ed081c055fb2b79799d8fe9c78e254af579c4d2dade14a84c445402ea8b77 |