Schema Description Agent
The Schema Description Agent is a Python-based tool that automatically generates descriptions for tables and their columns. It analyzes the structure and content of a data file, and then uses a Large Language Model (LLM) to produce accurate and concise documentation.
Features
- Statistical Analysis: Automatically calculates key statistics for each column, such as row count, column count, duplicate rows, missing cells, and more.
- AI-Powered Descriptions: Leverages LLMs to generate human-readable descriptions for tables and columns based on the statistical analysis.
- Configurable: Easily configure the AI provider, model, and other parameters.
- Extensible: Built on a modular framework (
sfn_blueprint) that allows for easy extension and integration.
Installation
Prerequisites
- uv – package & environment manager
Please refer to the official installation guide for the most up-to-date instructions.
For quick setup on macOS/Linux, you can currently use:curl -LsSf https://astral.sh/uv/install.sh | sh
- Git
Steps
-
Clone the repository:
git clone https://github.com/stepfnAI/schema_description_agent.git cd schema_description_agent git switch review
-
Create virtual environment and install dependencies:
uv sync --extra dev source .venv/bin/activate
-
Clone and install the blueprint dependency: The agent requires the
sfn_blueprintlibrary. Clone it into a sibling directory.cd ../ git clone https://github.com/stepfnAI/sfn_blueprint.git cd sfn_blueprint git switch dev uv pip install -e .
-
Return to the agent directory:
cd ../schema_description_agent
-
** set environment variables:**
export OPENAI_API_KEY='your_openai_api_key'
Configuration
You can configure the agent in two ways: using a .env file for project-specific settings or by exporting environment variables for more dynamic, shell-level control. Settings loaded via export will take precedence over those in a .env file.
Available Settings
The following table details the configuration options available:
| Environment Variable | Description | Default |
|---|---|---|
OPENAI_API_KEY or ANTHROPIC_API_KEY |
(Required) Your OpenAI API key. | None |
ai_provider_schema_description |
The AI provider to use for generating schema descriptions. | openai |
model_name_schema_description |
The specific AI model to use for schema descriptions. | gpt-4o |
temperature_schema_description |
AI model temperature (e.g., 0.0 to 2.0). |
0.3 |
max_tokens_schema_description |
Maximum tokens for the AI response. | 4000 |
Method 1: Using a .env File (Recommended)
For consistent configuration within your project, create a file named .env in the root directory and add your settings. This method is ideal for storing API keys and project-wide defaults.
- Create a file named
.envin the root of your project. - Add the key-value pairs for the settings you wish to override.
Example .env file:
# .env
# --- Required Settings ---
# Provide the API key for the provider you select below.
# For this example, we are using Anthropic.
ANTHROPIC_API_KEY="sk-your-anthropic-api-key-here"
# --- Optional Overrides for the Schema Description Agent ---
# Switch the AI provider to Anthropic
AI_PROVIDER_SCHEMA_DESCRIPTION="anthropic"
# Use a different model from the new provider
MODEL_NAME_SCHEMA_DESCRIPTION="claude-3-haiku-20240307"
# Use a higher temperature for potentially more descriptive responses
TEMPERATURE_SCHEMA_DESCRIPTION=0.7```
Testing
To run the tests, use the following command from the root of the schema_description_agent directory:
# Run all tests
pytest tests/ -s
# test agent
pytest tests/test_agent.py -s
# test agent with sample data
pytest tests/test_agent_with_data.py -s
Usage
Here is a simple example of how to use the agent:
python examples/basic_usage.py
from schema_description_agent import SchemaDescriptionAgent, SchemaDescriptionConfig
# Create a custom configuration
config = SchemaDescriptionConfig(
ai_provider_schema_description="anthropic",
model_name_schema_description="claude-3-opus-20240229",
temperature_schema_description=0.5
)
# Create an instance of the agent with the custom configuration
agent = SchemaDescriptionAgent(config=config)
Metadata
Release files for schema-description-agent 0.1.6
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| schema_description_agent-0.1.6.tar.gz | 14.6 kB | Without provenance | Details for schema_description_agent-0.1.6.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| schema_description_agent-0.1.6-py3-none-any.whl | Python 3 | none | any | Details for schema_description_agent-0.1.6-py3-none-any.whl |
Total release size: 26.0 kB
Release files / schema_description_agent-0.1.6.tar.gz
| Download URL | schema_description_agent-0.1.6.tar.gz |
|---|---|
| Size | 14.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
76761718bac2d52b05ce4750bd2e56831bb41943a804a9babf7e99f3deede68d
|
|
BLAKE2b-256 checksum How to use checksums |
f663f3f69af636eef01894f381ca53ef0174f1e67801bfe9b29f139035ccea20
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.11
|
Release files / schema_description_agent-0.1.6-py3-none-any.whl
| Download URL | schema_description_agent-0.1.6-py3-none-any.whl |
|---|---|
| Size | 11.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
44d2c4502007a2be31dd2208adf293c8233d6d369f730700d65b4556ede57954
|
|
BLAKE2b-256 checksum How to use checksums |
aa6cb23da7573ca253498c4bf0a42591196a89c01a894de2cbed5de370a1f7dc
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.11
|
Security for schema-description-agent 0.1.6
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.