mabel

mabel is a Data Engineering platform designed to run in serverless environments.

mabel just runs when you need it, scaling to zero, making it efficient and ideal for deployments to platforms like Kubernetes, GCP Cloud Run, AWS Fargate and Knative.

Documentation GitHub Wiki
Bug Reports GitHub Issues
Feature Requests GitHub Issues
Source Code GitHub
Discussions GitHub Discussions

Focus on What Matters

We've built mabel to enable Data Analysts to write complex data engineering tasks quickly and easily, so they could get on with doing what they do best.

from mabel import Reader

data = Reader(dataset="test_data")
print(data.count())

Key Features

On-the-fly compression
Low-memory requirements, even with terabytes of data
Indexing and partitioning of data for fast reads
Cursors for tracking reading position between processes
Partial SQL DQL (Data Query Language) support
Schema and data_expectations validation

Installation

From PyPI (recommended)

pip install --upgrade mabel

From GitHub

pip install --upgrade git+https://github.com/mabel-dev/mabel

Guides

How to Read Data

Dependencies

orjson for JSON (de)serialization
orso for data Schemas
zstandard for real-time on disk compression
LZ4 for real-time in memory compression

There are a number of optional dependencies which are usually only required for specific features and functionality. These are listed in tests/requirements.txt.

Integrations

mabel comes with adapters for the following data services:

	Service
	Google Cloud Storage
	MinIO
	AWS S3
	Azure Blob Storage
	Local Storage

Mabel is extensible with adapters for other data services as required.

Deployment and Execution

mabel supports running on a range of platforms, including:

	Platform
	Docker
	Kubernetes
	Windows (1)
	Linux (2)
	Mac (3)

Notice 1 - Some non-core features are not available on Windows.
Notice 2 - Tested on Debian (WSL) and Ubuntu.
Notice 3 - Tested on Apple Silicon Macs.

How Can I Contribute?

All contributions, bug reports, bug fixes, documentation improvements, enhancements, and ideas are welcome.

If you have a suggestion for an improvement or a bug, raise a ticket or start a discussion.

Want to help build mabel? See the contribution guidance.

License

Apache 2.0

Metadata

Release files for mabel 0.6.31

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for mabel 0.6.31
File	Size	Uploaded
mabel-0.6.31.tar.gz	96.6 kB	Sep 20, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for mabel 0.6.31
File	Interpreter	ABI	Platform	Reset
mabel-0.6.31-py3-none-any.whl 95.1 kB Sep 20, 2026	Python 3	none	any	Details

Total release size: 191.6 kB

Release files / mabel-0.6.31.tar.gz

Download URL	mabel-0.6.31.tar.gz
Size	96.6 kB
Tags	Source
SHA-256 checksum How to use checksums	`082eb0629a83c1aedd768c6ab00cb5d443841d3a9d51b2b9f06666f1a3be50db`
BLAKE2b-256 checksum How to use checksums	`e429d6603e0baf03e54a865dff58bfab8c72f2a696b8f5e5d894f2208bc6e2aa`
Upload date	Sep 20, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/7.0.0 CPython/3.13.14`

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/mabel-dev/mabel
Publishing commit: github.com/mabel-dev/mabel/tree/7835f098c6286e399da3ba5365c86730048c6d7f

Workflow

Publishing configuration: github.com/mabel-dev/mabel/blob/7835f098c6286e399da3ba5365c86730048c6d7f/.github/workflows/release.yaml
Publishing logs: github.com/mabel-dev/mabel/actions/runs/35506395155/attempts/1

Artifact

Subject: mabel-0.6.31.tar.gz
SHA-256 checksum: 082eb0629a83c1aedd768c6ab00cb5d443841d3a9d51b2b9f06666f1a3be50db Verifying attestations

Release files / mabel-0.6.31-py3-none-any.whl

Download URL	mabel-0.6.31-py3-none-any.whl
Size	95.1 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`9476639c0b3f60775f15e1594d4ffcb14414a69d692cf1ba87a96967503b6125`
BLAKE2b-256 checksum How to use checksums	`42850d41c86dbd42dfb4b7eb2df21d96c760b7143a7c2f940d25138a72f4ff4f`
Upload date	Sep 20, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/7.0.0 CPython/3.13.14`

Provenance

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/mabel-dev/mabel
Publishing commit: github.com/mabel-dev/mabel/tree/7835f098c6286e399da3ba5365c86730048c6d7f

Workflow

Publishing configuration: github.com/mabel-dev/mabel/blob/7835f098c6286e399da3ba5365c86730048c6d7f/.github/workflows/release.yaml
Publishing logs: github.com/mabel-dev/mabel/actions/runs/35506395155/attempts/1

Artifact

Subject: mabel-0.6.31-py3-none-any.whl
SHA-256 checksum: 9476639c0b3f60775f15e1594d4ffcb14414a69d692cf1ba87a96967503b6125 Verifying attestations