DataForge Profile
High-performance data profiling for Python, powered by Rust.
Installation
pip install dataforge-profile
Usage
import pandas as pd
from dataforge_profile import profile
# Profile a pandas DataFrame
df = pd.DataFrame({
"id": range(1_000_000),
"name": [f"user_{i}" for i in range(1_000_000)],
"email": [f"user{i}@example.com" for i in range(1_000_000)],
"age": [25] * 1_000_000,
})
result = profile(df)
# Access column statistics
print(result["columns"]["id"]["mean"]) # ~500000
print(result["columns"]["email"]["inferred_type"]) # "email"
Benchmark
Profiling 1M rows:
| Tool | Time | Memory |
|---|---|---|
| ydata-profiling | ~12s | ~500MB |
| dataforge-profile | ~0.14s | ~50MB |
~85x faster, 10x less memory
Requirements
- Python 3.9+
- pandas >= 1.5 or pyarrow >= 12.0
License
Apache 2.0
1 maintainer
Metadata
1 maintainer
Release files for dataforge-profile 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| dataforge_profile-0.1.0-cp39-cp39-macosx_11_0_arm64.whl | CPython 3.9 | CPython 3.9 | macOS 11.0+ ARM64 | Details |
Release files / dataforge_profile-0.1.0-cp39-cp39-macosx_11_0_arm64.whl
| Download URL | dataforge_profile-0.1.0-cp39-cp39-macosx_11_0_arm64.whl |
|---|---|
| Size | 628.0 kB |
| Tags | CPython 3.9 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
c763e4eb8aae3ea3ab87baf502f1dc624d61c64b7d0c4c36b0b26c3ac2b48003
|
|
BLAKE2b-256 checksum How to use checksums |
b2c35356c550bf16c43d4fd6a1b9a9adeb229fe29ab650566152c4181f708cba
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.9.6
|
