ccflow (Composable Configuration Flow) is a collection of tools for workflow configuration, orchestration, and dependency injection.
It is intended to be flexible enough to handle diverse use cases, including data retrieval, validation, transformation, and loading (i.e. ETL workflows), model training, microservice configuration, and automated report generation.
The framework provides:
- a way to manage hierarchical, strongly typed configurations and the relationships between them through composition
- a way to associate user-defined functions with configurations, and in doing so, to define and name configurable workflow graphs
- a way to manage dependency injection and inversion of control for objects in these graphs
- flexibility in how to interact with configurations and workflows, including files/command line, native python/Jupyter notebook, Airflow/job scheduler, REST API, etc (in progress)
It heavily leverages pydantic, and users are expected to implement their own configuration and workflow building blocks by implementing pydantic models.
It also integrates closely with hydra for file-based configuration and command line interaction, but can also be used natively from Python without it.
This library was partially inspired by this blog post by Suneeta Mall (@suneeta-mall).
We have taken these ideas a step further by introducing the concept of the ModelRegistry, which allows for the configs to be managed without hydra, and also allows us to implement dependency injection.
We aim to provide additional (and optional) tools for workflow orchestration on top of the configuration framework.
Documentation
Our wiki is organized along the four kinds of documentation:
- Tutorials — hands-on lessons, from First Steps through building ETL applications to a configurable calculator driven from the CLI.
- How-to Guides — recipes for specific tasks like configuring complex values, caching, and retries.
- Reference — the core types, built-in models, and options.
- Explanation — the design goals, core concepts, and why ccflow leans on Hydra.
Installation
ccflow can be installed via pip or conda, the two primary package managers for the Python ecosystem.
To install ccflow via pip, run this command in your terminal:
pip install ccflow
To install ccflow via conda, run this command in your terminal:
conda install ccflow -c conda-forge
Community
- Contribute to
ccflowand help improve the project
License
This software is licensed under the Apache 2.0 license. See the LICENSE file for details.
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file ccflow-0.9.1.tar.gz.
File metadata
- Download URL: ccflow-0.9.1.tar.gz
- Upload date:
- Size: 385.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/7.0.0 CPython/3.12.13
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
455b6f6659f4231c6eb145ec5ba74cff3d1e59046ab926eb713983ee1c1ca876
|
|
| MD5 |
c61f6499cb732e2faf379b1e55eb90b9
|
|
| BLAKE2b-256 |
fcf96d6cdeb2e1e3f1df5c2f27ac4576e744c2dc84806d1eff8dc51cfcdec6a1
|
File details
Details for the file ccflow-0.9.1-py3-none-any.whl.
File metadata
- Download URL: ccflow-0.9.1-py3-none-any.whl
- Upload date:
- Size: 465.3 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/7.0.0 CPython/3.12.13
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
b08779a0f293709ae875803fa19c59f717ab55d7b469fc10155f3e590a0ddafc
|
|
| MD5 |
cef5274307b63f29ffe7a1cf68fc837c
|
|
| BLAKE2b-256 |
78591058426a7916bc988e9f7822cd4f3c004a6b21f314b56d19d6b19f05b806
|
