df-enrich
Pandas DataFrame accessor for schema-driven validation, derived columns, lookups, and profiling—powered by df-eval.
Features
- 🔍 Validation: Schema-driven validation using Pandera
- 🔧 Derivation: Create derived columns using expressions (via df-eval Engine)
- 📊 Profiling: Generate data profiles with ydata-profiling
- 🔗 Lookups: Enrich data with external sources
- ⛓️ Chaining: Fluent API for method chaining
- 📝 Provenance: Track operations in DataFrame.attrs
- 🎯 Type Casting: Convenient dtype conversions
Installation
Using uv (recommended):
uv pip install df-enrich
Using pip:
pip install df-enrich
With optional dependencies:
# For documentation
uv pip install "df-enrich[docs]"
# For testing
uv pip install "df-enrich[test]"
Quick Start
import pandas as pd
import pandera as pa
from df_enrich import EnrichAccessor
# Create a DataFrame
df = pd.DataFrame({"a": [1, 2, 3], "b": [4, 5, 6]})
# Validate with Pandera
schema = pa.DataFrameSchema({
"a": pa.Column(int),
"b": pa.Column(int)
})
df_validated = df.enrich.validate(schema)
# Derive new columns
df_enriched = df.enrich.derive({"c": "a + b", "d": "a * b"})
# Chain operations
result = (df.enrich
.validate(schema)
.enrich.derive({"total": "a + b"})
.enrich.cast({"total": "int32"}))
Usage
Validation
import pandera as pa
class MySchema(pa.DataFrameModel):
col1: int
col2: float
df.enrich.validate(MySchema)
Derive Columns
# From dictionary
df.enrich.derive({"total": "price * quantity"})
# From YAML string
df.enrich.derive("""
total: "price * quantity"
discount: "total * 0.1"
""")
# From YAML file
df.enrich.derive("derivations.yaml")
Profiling
# Generate profile report
profile = df.enrich.profile(engine="ydata")
# Lazy profiling
profile = df.enrich.profile(engine="ydata", lazy=True)
Lookups
# Lookup from DataFrame
prices = pd.DataFrame({"price": [10, 20, 30]})
df.enrich.lookup(prices, dst="price")
# Custom resolver
def my_resolver(df, src, dst):
df[dst] = fetch_from_api(df)
return df
df.enrich.lookup(None, dst="data", resolver=my_resolver)
Configuration
# Configure registries, secrets, etc.
df.enrich.config(registry_url="https://api.example.com")
Development
Setup
# Clone repository
git clone https://github.com/elphick/df-enrich.git
cd df-enrich
# Install uv
curl -LsSf https://astral.sh/uv/install.sh | sh
# Create virtual environment and install dependencies
uv venv
uv pip install -e ".[test,docs]"
Running Tests
uv run pytest tests/ -v --cov=df_enrich
Building Documentation
cd docs
uv run sphinx-build -b html . _build/html
Contributing
Contributions are welcome! Please feel free to submit a Pull Request.
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
df_enrich-0.1.1.tar.gz
(89.2 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file df_enrich-0.1.1.tar.gz.
File metadata
- Download URL: df_enrich-0.1.1.tar.gz
- Upload date:
- Size: 89.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
db458037835ab3e21fd4196271811888ac87f6fc5ba8036f6050258443b2182c
|
|
| MD5 |
50704b6662ca274f7a91da5b22350783
|
|
| BLAKE2b-256 |
5f80c475519b1ca2ee906d168510c838fd554be628e3cb0efc434aac316f0c27
|
Provenance
The following attestation bundles were made for df_enrich-0.1.1.tar.gz:
Publisher:
publish_to_pypi.yml on elphick/df-enrich
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
df_enrich-0.1.1.tar.gz -
Subject digest:
db458037835ab3e21fd4196271811888ac87f6fc5ba8036f6050258443b2182c - Sigstore transparency entry: 2341737095
- Sigstore integration time:
-
Permalink:
elphick/df-enrich@631f6d5701cf4f044490f5c641fd6cc66fe131c8 -
Branch / Tag:
refs/heads/main - Owner: https://github.com/elphick
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish_to_pypi.yml@631f6d5701cf4f044490f5c641fd6cc66fe131c8 -
Trigger Event:
workflow_dispatch
-
Statement type:
File details
Details for the file df_enrich-0.1.1-py3-none-any.whl.
File metadata
- Download URL: df_enrich-0.1.1-py3-none-any.whl
- Upload date:
- Size: 7.1 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
7f80285f1e88dc0b6bef4483d05ebf52a95f5e37c9b8af3cb023a772cb46f71e
|
|
| MD5 |
94fc3f5e072edeb0e1808be5102d01d3
|
|
| BLAKE2b-256 |
0cdeaa410136b91b1253df7c2ad57f49e2a1d01a33479900a22a621a2e352aeb
|
Provenance
The following attestation bundles were made for df_enrich-0.1.1-py3-none-any.whl:
Publisher:
publish_to_pypi.yml on elphick/df-enrich
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
df_enrich-0.1.1-py3-none-any.whl -
Subject digest:
7f80285f1e88dc0b6bef4483d05ebf52a95f5e37c9b8af3cb023a772cb46f71e - Sigstore transparency entry: 2341737100
- Sigstore integration time:
-
Permalink:
elphick/df-enrich@631f6d5701cf4f044490f5c641fd6cc66fe131c8 -
Branch / Tag:
refs/heads/main - Owner: https://github.com/elphick
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish_to_pypi.yml@631f6d5701cf4f044490f5c641fd6cc66fe131c8 -
Trigger Event:
workflow_dispatch
-
Statement type:
