This release is a pre-release and may not be stable for production use.
CogFlow
CogFlow is a modular, SDK-first machine learning workflow management system built on Kubeflow Pipelines, MLflow, Kubernetes, and MinIO.
It provides a clean Python API for:
- building production-grade ML pipelines
- managing datasets and components
- orchestrating federated learning workflows
- enforcing consistent error handling and validation
CogFlow is designed for real infrastructure, not notebooks only.
Why CogFlow?
Modern ML platforms are powerful but fragmented:
- Kubeflow Pipelines → great orchestration, weak ergonomics
- MLflow → experiment tracking, limited workflow control
- Kubernetes → powerful, but verbose and error-prone
- Federated learning → no standard orchestration layer
CogFlow bridges these gaps by providing:
- a stable Python SDK
- safe lazy-loading of heavy dependencies
- unified error handling
- infrastructure-aware abstractions
- zero circular imports
Core Features
🧩 Pipeline Orchestration
- Lazy-loaded Kubeflow Pipelines client
- Safe pipeline compilation and submission
- Runtime environment injection
- Kubernetes service lifecycle management
📦 Component Management
- YAML-based component registry
- MinIO-backed component storage
- Automatic component registration
- Runtime-safe component loading
📊 Dataset Management
- Dataset registration and metadata retrieval
- Secure dataset downloads
- Silent deletes with strict error semantics
- Pluggable storage backends
🤝 Federated Learning
- Auto-generated FL pipelines
- Dynamic pipeline signatures
- Connector-based and dataspace-based workflows
- Region-aware scheduling with node selectors
🧠 Unified Error Handling
- Strongly typed error hierarchy
- Context-aware exception wrapping
- API-ready error serialization
- Zero silent failures
Architecture Overview
CogFlow follows a layered SDK architecture:
cogflow/ ├── core/ │ ├── pipelines/ # Kubeflow orchestration & FL pipelines │ ├── datasets/ # Dataset lifecycle management │ ├── components/ # Component registry & YAML handling │ └── models/ # (future extension) │ ├── utils/ │ ├── common.py # UUIDs, paths, Kubernetes helpers │ ├── network.py # HTTP utilities with retry & streaming │ ├── storage.py # MinIO client abstraction │ └── exceptions.py # Unified error framework │ ├── config.py └── api.py
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file cogflow-3.0.0b11.tar.gz.
File metadata
- Download URL: cogflow-3.0.0b11.tar.gz
- Upload date:
- Size: 366.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
c9185c0a80ac18a090ac6b52d9cb95ea0cb88499539500457f336c15d65d5c48
|
|
| MD5 |
b7b0b03c6227ac1b8782d71830002b02
|
|
| BLAKE2b-256 |
980f16951819952e7976b516a194b4166206faac9331dc0fde7a63cb8142276e
|
Provenance
The following attestation bundles were made for cogflow-3.0.0b11.tar.gz:
Publisher:
release.yml on HIRO-MicroDataCenters-BV/cogflow
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
cogflow-3.0.0b11.tar.gz -
Subject digest:
c9185c0a80ac18a090ac6b52d9cb95ea0cb88499539500457f336c15d65d5c48 - Sigstore transparency entry: 2336878689
- Sigstore integration time:
-
Permalink:
HIRO-MicroDataCenters-BV/cogflow@0506e2bf9bc5f7135523ca67761abde3f266b897 -
Branch / Tag:
refs/tags/v3.0.0b11 - Owner: https://github.com/HIRO-MicroDataCenters-BV
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
release.yml@0506e2bf9bc5f7135523ca67761abde3f266b897 -
Trigger Event:
push
-
Statement type:
File details
Details for the file cogflow-3.0.0b11-py3-none-any.whl.
File metadata
- Download URL: cogflow-3.0.0b11-py3-none-any.whl
- Upload date:
- Size: 421.0 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
3caf85ce8b350e09a48f573ed2f76400f74cf63cb527226d0637cfe288764b57
|
|
| MD5 |
374c0c3648d094b9b03329ad877bd374
|
|
| BLAKE2b-256 |
d037869bf634ee5fa2fbf1c8cdbf993ec06c3fbc98b746a3c42757baadf4b7e6
|
Provenance
The following attestation bundles were made for cogflow-3.0.0b11-py3-none-any.whl:
Publisher:
release.yml on HIRO-MicroDataCenters-BV/cogflow
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
cogflow-3.0.0b11-py3-none-any.whl -
Subject digest:
3caf85ce8b350e09a48f573ed2f76400f74cf63cb527226d0637cfe288764b57 - Sigstore transparency entry: 2336878704
- Sigstore integration time:
-
Permalink:
HIRO-MicroDataCenters-BV/cogflow@0506e2bf9bc5f7135523ca67761abde3f266b897 -
Branch / Tag:
refs/tags/v3.0.0b11 - Owner: https://github.com/HIRO-MicroDataCenters-BV
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
release.yml@0506e2bf9bc5f7135523ca67761abde3f266b897 -
Trigger Event:
push
-
Statement type:
