mabel 0.4.3

Python Data Libraries

MABEL IS IN ALPHA - USAGE IS STILL STABLIZING

mabel is a platform for authoring data processing systems.

Features

• Programatically define data pipelines
• Immutable datasets
• On-the-fly compression
• Automatic version tracking of processing operations
• Trace messages through the pipeline (random sampling)
• Automatic retry of operations

Documentation

See the wiki

How Do I Get It?

From PyPI (recommended)

pip install --upgrade mabel

From GitHub

pip install --upgrade git+https://github.com/joocer/mabel

Dependencies

• UltraJSON (AKA ujson) is used where orjson is not available. orjson is the preferred JSON library but is not available on all platforms and environments so ujson is a dependency to ensure a performant JSON library with broad support is available.
• dateutil
• zstandard

There are a number of optional dependencies which are usually only required for specific features and functionality. These are listed in the requirements-optional.txt file which is used for testing. The key exception is orjson which is the preferred JSON library but not available on all platforms.

Contributing

Want to help build mabel? See the contribution guidance

What Platforms Does It Support?

mabel comes with adapters for the following services, or is tested to run on the following platforms:

	Service	Support
	Google Cloud Storage	Read/Write
	MinIO	Read/Write
	MongoDB	Read
	MQTT	Read
	Docker	Hosting
	Kubernetes	Hosting
	Raspberry Pi	Hosting (1)

Linux, MacOS and Windows (2) also supported.

Adapters for other data services can be written.

1 - Raspbian fully functional with alternate JSON libraries
2 - Multi-Processing not available on Windows

License

Apache 2.0