Skip to main content

DataComPy

PyPI - Python Version Code style: black PyPI version Anaconda-Server Badge PyPI - Downloads

DataComPy is a package to compare two Pandas DataFrames. Originally started to be something of a replacement for SAS's PROC COMPARE for Pandas DataFrames with some more functionality than just Pandas.DataFrame.equals(Pandas.DataFrame) (in that it prints out some stats, and lets you tweak how accurate matches have to be). Then extended to carry that functionality over to Spark Dataframes.

Quick Installation

pip install datacompy

or

conda install datacompy

Installing extras

If you would like to use Spark or any other backends please make sure you install via extras:

pip install datacompy[spark]
pip install datacompy[dask]
pip install datacompy[duckdb]
pip install datacompy[polars]
pip install datacompy[ray]

In-scope Spark versions

Different versions of Spark play nicely with only certain versions of Python below is a matrix of what we test with

Spark 3.1.3 Spark 3.2.3 Spark 3.3.4 Spark 3.4.2 Spark 3.5.0
Python 3.8 ✅ ✅ ✅ ✅ ✅
Python 3.9 ✅ ✅ ✅ ✅ ✅
Python 3.10 ✅ ✅ ✅ ✅ ✅
Python 3.11 ❌ ❌ ❌ ✅ ✅
Python 3.12 ❌ ❌ ❌ ❌ ❌

:::{note} At the current time Python 3.12 is not supported by Spark and also Ray within Fugue. :::

Supported backends

  • Pandas: (See documentation)
  • Spark: (See documentation)
  • Polars (Experimental): (See documentation)
  • Fugue is a Python library that provides a unified interface for data processing on Pandas, DuckDB, Polars, Arrow, Spark, Dask, Ray, and many other backends. DataComPy integrates with Fugue to provide a simple way to compare data across these backends. Please note that Fugue will use the Pandas (Native) logic at its lowest level (See documentation)

Contributors

We welcome and appreciate your contributions! Before we can accept any contributions, we ask that you please be sure to sign the Contributor License Agreement (CLA).

This project adheres to the Open Source Code of Conduct. By participating, you are expected to honor this code.

Roadmap

Roadmap details can be found here

Metadata

Release files for datacompy 0.11.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for datacompy 0.11.1
File Size Uploaded Provenance Actions
datacompy-0.11.1.tar.gz 56.8 kB Without provenance Details for datacompy-0.11.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for datacompy 0.11.1
File Interpreter ABI Platform Actions
datacompy-0.11.1-py3-none-any.whl Python 3 none any Details for datacompy-0.11.1-py3-none-any.whl

Total release size: 98.4 kB

Release files / datacompy-0.11.1.tar.gz

Download URL datacompy-0.11.1.tar.gz
Size 56.8 kB
Tags Source
SHA-256 checksum
How to use checksums
cfb5e871a7f4a8d9dec70f8e9e0f331d13b16183624b4960dd425ac6f1a0338e
BLAKE2b-256 checksum
How to use checksums
5b2d70fc642bc93b557b5265d8ba4b2336bf03e22550da54dd6289820d4c9a8f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.0.0 CPython/3.9.18

Release files / datacompy-0.11.1-py3-none-any.whl

Download URL datacompy-0.11.1-py3-none-any.whl
Size 41.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
0ec8c7272cec4c2731e7a502936cad6a7ae611b28d3454dd57696ba6e9926215
BLAKE2b-256 checksum
How to use checksums
d41879adc5c43cd4cb49471c122b5504019bd1f50077cd578eb7ec4ad3e363a2
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.0.0 CPython/3.9.18

Security for datacompy 0.11.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.11.1 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page