Skip to main content

`datapipe` is a realtime incremental ETL library for Python application

Project description

Datapipe

Datapipe is a real-time, incremental ETL library for Python with record-level dependency tracking.

The library is designed for describing data processing pipelines and is capable of tracking dependencies for each record in the pipeline. This ensures that tasks within the pipeline receive only the data that has been modified, thereby improving the overall efficiency of data handling.

https://datapipe.dev/

Development

At the moment these branches are active:

  • master - current development state, will be promoted to 0.13.x series release once ready
  • v0.13 - current stable version
  • v0.11 - legacy stable version (v0.12 was skipped due reasons)

Version compatibility

At the moment, the datapipe library is under active development. Versions: v0.*.*

It should be expected that each minor version is not backward compatible with the previous one. That is, v0.7.0 is not compatible with v0.6.1. Dependencies should be fixed to the exact minor version.

After stabilization and transition to the major version v1.*.*, the common rules will apply: all versions with the same major component are compatible.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

datapipe_core-0.14.0.tar.gz (42.5 kB view details)

Uploaded Source

Built Distribution

datapipe_core-0.14.0-py3-none-any.whl (52.6 kB view details)

Uploaded Python 3

File details

Details for the file datapipe_core-0.14.0.tar.gz.

File metadata

  • Download URL: datapipe_core-0.14.0.tar.gz
  • Upload date:
  • Size: 42.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.8.3 CPython/3.10.12 Linux/5.15.153.1-microsoft-standard-WSL2

File hashes

Hashes for datapipe_core-0.14.0.tar.gz
Algorithm Hash digest
SHA256 69f8dc297c63af96b8e11c9bbcc4c64c38ba400f8c2d58f965273fd9e311f4ee
MD5 7cb148ac79d8e6201095f5be970878df
BLAKE2b-256 55b8ed7bfc75a15b6bb607dac8b362fb4a1ec1633a4823dca9664b7f4decb594

See more details on using hashes here.

File details

Details for the file datapipe_core-0.14.0-py3-none-any.whl.

File metadata

  • Download URL: datapipe_core-0.14.0-py3-none-any.whl
  • Upload date:
  • Size: 52.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.8.3 CPython/3.10.12 Linux/5.15.153.1-microsoft-standard-WSL2

File hashes

Hashes for datapipe_core-0.14.0-py3-none-any.whl
Algorithm Hash digest
SHA256 ef8dde7d0be054dc02097744347b96f1e8bef7ff927ae7c33bd50c8a5e3e8722
MD5 c0c209c8b888ce1aa64f44c7a74aebb5
BLAKE2b-256 921a0823645dab63a1ffe5456ccf35e91a21a6be1b01d5b24946da634cd32811

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page