Data Enrichment Agent
The Data Enrichment Agent is a Python-based tool that automatically generates and applies feature engineering transformations to enhance your datasets. It analyzes the structure and content of your data, then uses a Large Language Model (LLM) to suggest and create meaningful new features for machine learning and analytics.
Features
- Automated Feature Engineering: Generate new features using LLM-powered suggestions
- Intelligent Data Profiling: Automatic analysis of input data characteristics
- Multiple Data Formats: Support for CSV, Excel, JSON, and Parquet files
- Configurable Parameters: Customize enrichment behavior and thresholds
- Production-Ready: Type hints, logging, and error handling throughout
- Extensible: Built on a modular framework that allows for easy extension
Feature Engineering Capabilities
The agent can generate various types of features:
-
Time-Based Features
- Date part extraction
- Time differences
- Business day calculations
-
Numeric Transformations
- Polynomial features
- Binning and discretization
- Mathematical transformations
-
Categorical Encodings
- One-hot encoding
- Frequency encoding
- Target encoding
-
Interaction Features
- Arithmetic combinations
- Ratio features
- Conditional features
Prerequisites
- uv – package & environment manager
For quick setup on macOS/Linux:curl -LsSf https://astral.sh/uv/install.sh | sh
- Git
Installation
-
Clone the repository
git clone https://github.com/stepfnAI/data_enrichment_agent.git cd data_enrichment_agent git switch dev
-
Set up the virtual environment and install dependencies
uv venv --python=3.10 venv source venv/bin/activate uv pip install -e ".[dev]"
-
Clone and install the blueprint dependency
cd .. git clone https://github.com/stepfnAI/sfn_blueprint.git cd sfn_blueprint git switch dev uv pip install -e . cd ../data_enrichment_agent
-
export the environment variables
export OPENAI_API_KEY="your_openai_api_key"
Architecture
The Data Enrichment Agent is built with a modular architecture:
data_enrichment_agent/
├── agent.py # Main agent implementation
├── models.py # Pydantic models for data structures
├── utils.py # Helper functions and utilities
├── config.py # Configuration management
├── constants.py # Constants and templates
└── cli.py # Command-line interface
Configuration
The agent can be configured using the EnrichmentConfig class. Here are the available configuration options:
from data_enrichment_agent.models import EnrichmentConfig
config = EnrichmentConfig(
model_name="gpt-4.1-mini", # LLM model to use
model_temperature=0.1, # Temperature for LLM responses
model_max_tokens=2000, # Maximum tokens for LLM responses
ai_provider="openai", # AI provider to use
ai_task_type="feature_suggestions_generator", # Task type for AI requests
Basic Usage
python examples/basic_usage.py
Testing
Run the test suite using pytest:
# Run all tests
pytest tests/ -s
# Run specific test
pytest tests/test_agent.py -s
# Run with coverage report
pytest --cov=data_enrichment_agent tests/ -s
Contributing
Contributions are welcome! Please follow these steps:
- Create a feature branch (
git checkout -b feature/amazing-feature) - Commit your changes (
git commit -m 'Add some amazing feature') - Push to the branch (
git push origin feature/amazing-feature)
Metadata
Release files for data-enrichment-agent 0.1.12
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| data_enrichment_agent-0.1.12.tar.gz | 18.6 kB | Without provenance | Details for data_enrichment_agent-0.1.12.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| data_enrichment_agent-0.1.12-py3-none-any.whl | Python 3 | none | any | Details for data_enrichment_agent-0.1.12-py3-none-any.whl |
Total release size: 36.0 kB
Release files / data_enrichment_agent-0.1.12.tar.gz
| Download URL | data_enrichment_agent-0.1.12.tar.gz |
|---|---|
| Size | 18.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
e1533b794fd963558678b453f347b028b9791707a7188983527547e064d9967d
|
|
BLAKE2b-256 checksum How to use checksums |
d4db87106879c955139386b84f68e41bc59ae621f601d1139d2830d8627780d8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.9.6
|
Release files / data_enrichment_agent-0.1.12-py3-none-any.whl
| Download URL | data_enrichment_agent-0.1.12-py3-none-any.whl |
|---|---|
| Size | 17.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
18071d4783d7afdd860a2b41706c513e5f3b2e7b5194af3f155b2bc96bf6bdab
|
|
BLAKE2b-256 checksum How to use checksums |
ee39de8935d417a8192e8640deab4ce7a9d6faed8a470b2395c1d2c4a4605d49
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.9.6
|
Security for data-enrichment-agent 0.1.12
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.