What is Data.Rentgen?
Data.Rentgen is a Data Motion Lineage service, compatible with OpenLineage specification.
Currently we support consuming lineage from:
Apache Spark
Apache Airflow
Apache Hive
Apache Flink
dbt
SyncMaster (proprietary integration, part of MWS Data Bridge)
StarRocks (proprietary integration, part of MWS Data Engine)
Note: service is under active development, so API can be unstable.
Goals
Collect lineage events produced by OpenLineage clients & integrations.
Store operation-grained events for better detalization.
Provide API for fetching both job/run ↔ dataset lineage and dataset ↔ dataset lineage.
Features
Support consuming large amounts of lineage events, use Apache Kafka as event buffer.
Store data in tables partitioned by event timestamp, to speed up lineage graph resolution.
Lineage graph is build with user-specified time boundaries.
Lineage graph can be build with different granularity. e.g. merge all individual Spark commands into Spark applicationId or Spark applicationName.
Column-level lineage support.
Authentication support.
Non-goals
This is not a Data Catalog. DataRentgen doesn’t track dataset schema change, owner and so on. Use Datahub or OpenMetadata instead.
Static Data Lineage like view → table is not supported.
Limitations
OpenLineage have integrations with Trino, Debezium and some other lineage sources. DataRentgen support may be added later.
DataRentgen parses only limited set of OpenLineage facets, and doesn’t store custom facets. This can be changed in future.
Documentation
Screenshots
Lineage graph
Dataset-level lineage graph
Dataset column-level lineage graph
Job-level lineage graph
Run-level lineage graph
Hierarchy graph
Datasets
Runs
Spark application
Spark run
Spark command
Hive query
Airflow DagRun
Airflow TaskInstance
Release files for data-rentgen 0.7.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| data_rentgen-0.7.2.tar.gz | 174.1 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| data_rentgen-0.7.2-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 506.2 kB
Release files / data_rentgen-0.7.2.tar.gz
| Download URL | data_rentgen-0.7.2.tar.gz |
|---|---|
| Size | 174.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
02848436c9a441978ba47a0b5f4154766a5b1f39c4ac229e6d93c6c9fbf222ec
|
|
BLAKE2b-256 checksum How to use checksums |
9429632b378c7223ac9e4a6ae518bd6f5aa3445ad8839704604ce652a9c07a33
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 11, 2026.
Transparency logRelease files / data_rentgen-0.7.2-py3-none-any.whl
| Download URL | data_rentgen-0.7.2-py3-none-any.whl |
|---|---|
| Size | 332.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
51c84e6db761b0bc257a42a4158fc4426a6bddfb4e30fedee65471a89632dd81
|
|
BLAKE2b-256 checksum How to use checksums |
bd85bdcd11cba6067300e57efe25ce56a71014de4956b522f09416782add8caf
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 11, 2026.
Transparency log
