pyspark-data-provenance

Python package in this monorepo for enabling Spark data provenance features in PySpark jobs.

This project is managed with uv.

Quick Start

From the repository root:

cd pyspark-data-provenance
make init-uv-python
make init
make check
make test
make run

Project Layout

pyspark-data-provenance/
	src/pyspark_data_provenance/
	tests/
	main.py
	pyproject.toml
	Makefile

Common Commands

make init - Create/refresh lock file and sync dependencies
make update - Upgrade dependencies and sync environment
make check - Run lint and type checks
make test - Run unit tests
make build - Build package wheel
make publish - Publish package (PyPI)

PyPI Package

The Python package is published on PyPI:

https://pypi.org/project/pyspark-data-provenance/

You can install it with pip:

pip install pyspark-data-provenance

Before publishing to production PyPI with make publish, set your token:

export UV_PUBLISH_TOKEN=<your-pypi-token>

For details, see:

https://docs.astral.sh/uv/guides/package/#publishing-your-package

CI/CD

Repository-level workflows are provided for:

CI: lint, type-check, and tests on pushes/PRs affecting this package
Publish: build and publish on release

Metadata

Release files for pyspark-data-provenance 0.0.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distribution (wheel)

Table of built distributions (wheels) for pyspark-data-provenance 0.0.2
File	Interpreter	ABI	Platform	Reset
pyspark_data_provenance-0.0.2-py3-none-any.whl 2.8 kB Mar 11, 2026 Without provenance	Python 3	none	any	Details

Release files / pyspark_data_provenance-0.0.2-py3-none-any.whl

Download URL	pyspark_data_provenance-0.0.2-py3-none-any.whl
Size	2.8 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`942ab194ef3158de30b37489e2a5569dbdd4c3dd4ef01607bc2a92cafc31aac1`
BLAKE2b-256 checksum How to use checksums	`45a3c9ce70eb7828e5bdf83a85988f98dafa428274095b316d98013903de7eb0`
Upload date	Mar 11, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`uv/0.10.9 {"installer":{"name":"uv","version":"0.10.9","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}`

Security for pyspark-data-provenance 0.0.2

Report project as malware

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.