Ubunye (oo-BOON-yeh): isiZulu for "unity"
One framework. Every pipeline. Any environment.
Docs • Docs map • Try it • Community
A data pipeline moves data from one place to another (a database to a file, a REST API to a data warehouse) and usually changes it on the way. Most of the work is plumbing: connections, credentials, a framework's quirks, the same read, transform, write scaffold again and again.
Ubunye Engine writes that plumbing for you. You describe what to read and write in a short YAML file and put your change to the data in a normal Python class. Ubunye does the connections, the compute engine (Apache Spark, or pandas on a laptop with no Java) and the read and write loop. The same folder runs on your laptop today and on a cluster tomorrow, with no code changes.
What you get:
- One standard layout. Every project is use case, pipeline, task, run by one CLI. The words are explained in Concepts.
- A record of every run. Each run can leave a run record with a hash of every row it read and wrote, so two runs can be compared exactly.
- Checks before you run.
ubunye planandubunye doctorsay what would fail, and why, before any data moves. Errors say what went wrong and suggest the fix. - Open plugins. Connectors, backends, model registries and run record stores are added from outside, with no engine edits.
- Types that ship. The package carries its type information, and a type checker guards every merge.
Try it
On a laptop, with no Java and no cloud account:
pip install "ubunye-engine[pandas]"
ubunye init -d pipelines -u demo -p starter -t filter_adults
ubunye plan -d pipelines -u demo -p starter -t filter_adults --backend pandas
ubunye run -d pipelines -u demo -p starter -t filter_adults --backend pandas --lineage
ubunye lineage list -d pipelines -u demo -p starter -t filter_adults
init makes a task folder with a sample CSV, plan checks it without moving data, run
writes Parquet and keeps a run record, and lineage list shows that record. The test
suite runs these four commands exactly as written. What each one did, and how to run the
same folder on Spark: the Quickstart.
Docs
The full docs live at ubunye-ai-ecosystems.github.io/ubunye_engine. These are the pages most people need:
| You want to | Read |
|---|---|
| Install it | Installation |
| Run a first task | Quickstart |
| Learn the words (use case, pipeline, task, profile, run record) | Concepts |
Write a config.yaml |
Config overview, then Inputs and outputs |
| Check the data a task writes | Expectations |
| Read what a run did (the run record) | Lineage commands and the run record |
| Run on pandas or Spark | Execution backends |
| Look up a command | CLI reference |
| Call it from Python or a notebook | Python API |
| Add a language model step | Language model steps |
| Copy a working example | Examples |
| Read from or write to a system | Connectors |
| Deploy it | Deployment |
| Fix something that went wrong | Troubleshooting, then Errors |
| Move from 0.8 to 0.9 | Upgrading |
| See what changed | Changelog |
Worked examples, grouped by where they run (Databricks, a laptop, Docker, Kubernetes), live in ubunye-examples.
Proof
One task, unchanged, gave the same rows on pandas (no Java), local Spark, Kubernetes,
AWS Glue, GCP Dataproc, Azure Container Apps and Databricks: the same data hash in all
seven, compared from their run records by ubunye prove. See the
latest results.
What Ubunye is not
- Not an agent framework: use LangChain or CrewAI for that.
- Not an orchestrator: use Airflow, Prefect or Dagster (
ubunye exportwrites an Airflow DAG or a Databricks job for you). - Not a compute engine: it runs on Spark, or on pandas for laptop-sized data.
Ubunye is the standard layer between your data sources and your applications. It makes the plumbing boring so you can focus on what matters.
Not yet: data drift detection is a pattern you write in a task, not a built-in feature. EMR Serverless is built but has not run the proving workload, and Hadoop/YARN is not tested. Neither is claimed until a run says so.
Get involved
New connectors, bug fixes, typos, or just telling us what you are building: all help.
- Report a bug
- Request a feature
- Read the contributing guide
- Star the repo if you find it useful; it helps more than you'd think
License
Built with 🇿🇦 by Ubunye AI Ecosystems
Metadata
Release files for ubunye-engine 0.9.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| ubunye_engine-0.9.2.tar.gz | 444.6 kB | With provenance | Details for ubunye_engine-0.9.2.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| ubunye_engine-0.9.2-py3-none-any.whl | Python 3 | none | any | Details for ubunye_engine-0.9.2-py3-none-any.whl |
Total release size: 970.7 kB
Release files / ubunye_engine-0.9.2.tar.gz
| Download URL | ubunye_engine-0.9.2.tar.gz |
|---|---|
| Size | 444.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
c5391c119ce411c82c4900efc91c4eb176b1a5cc160fdd91a092946a497dc22d
|
|
BLAKE2b-256 checksum How to use checksums |
90d094d419836f10a852f8d5800b8b51ddad91a4614fe29962aceafaed56720f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 10, 2026.
Transparency logRelease files / ubunye_engine-0.9.2-py3-none-any.whl
| Download URL | ubunye_engine-0.9.2-py3-none-any.whl |
|---|---|
| Size | 526.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
445480149e94b814e468928cb7bedb9383fafff68d94f63e866ac4e0d5f34167
|
|
BLAKE2b-256 checksum How to use checksums |
a2fa557867fb2732f9c18596e4c77134704debac380a76c0b2e4604ff636c6c8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 10, 2026.
Transparency logSecurity for ubunye-engine 0.9.2
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for ubunye_engine-0.9.2.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 10, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.