A pandas extension built for OpenLineage
Project description
pandas-lineage
BEWARE: This project is in very early stages (as of 2022-09-12)
pandas-lineage is intended to extend the functionality of I/O and standard transform operations on a pandas dataframe to emit OpenLineage RunEvents. I am starting just with read/write operations emiting RunEvents with schema facets.
Badges:
Installation
pip install pandas-lineage
Development Documentation
Examples:
- marquez-examples
- contains getting started code and a script for running Marquez locally in Docker
- mock-api-example
- contains getting started code and a simple Flask API for sending lineage events to which will just always return a 200 status code
References:
- :green_heart: Marquez :green_heart:
- :green_heart: OpenLineage :green_heart:
- :green_heart: Pandas :green_heart:
Contributing:
I have not created any sort of contribution guide yet, but I don't want that to stop anyone! If you are interested in contributing, fork this repository and open a PR. As this becomes more feature-rich/useful, we will establish a contributors workflow. For now, please just use the pre-commit hooks.
Notes:
- The pandas-lineage directory structure (for now) will mirror the directory structure of pandas for the components that it is extending.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for pandas_lineage-0.0.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 83abf8040ac574014ca8ebdee1fb5b01c7c4939dfd777b0a9eeb4fdceb964bdd |
|
MD5 | 8dd8b4f1777ebf66577ee7e21b05a19a |
|
BLAKE2b-256 | 4df39bc8cd9859562e13c55954548219f3bffe22e25aab051cfb2dc4c449ec65 |