A scalable, fast, ACID-compliant Data Catalog powered by Ray.
Project description
DeltaCAT
DeltaCAT is a Pythonic Data Catalog powered by Ray.
Its data storage model allows you to define and manage fast, scalable, ACID-compliant data catalogs through git-like stage/commit APIs, and has been used to successfully host exabyte-scale enterprise data lakes.
DeltaCAT uses the Ray distributed compute framework together with Apache Arrow for common table management tasks, including petabyte-scale change-data-capture, data consistency checks, and table repair.
Getting Started
Install
pip install deltacat
Running Tests
pip3 install virtualenv
virtualenv test_env
source test_env/bin/activate
pip3 install -r requirements.txt
pytest
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
deltacat-1.1.13.tar.gz
(208.5 kB
view hashes)
Built Distribution
deltacat-1.1.13-py3-none-any.whl
(293.3 kB
view hashes)
Close
Hashes for deltacat-1.1.13-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cfb6a976cae1fb0896ea0f1bd28a13233f3bd07ee68e5b106204041818d6f8b2 |
|
MD5 | d756c5a4b7d27136ffab7b0cc42fd451 |
|
BLAKE2b-256 | 8c275e5c40510c2ec187a6831000c0db4c6ddf1e07faaa232085261e497f0757 |