delm 1.0.0

DELM:Data Extraction with Language Models

Data Extraction with Language Models

DELM is a Python toolkit for extracting structured data from unstructured text using language models.

📖 Full Documentation

Features

• Multiple input formats: TXT, HTML, MD, DOCX, PDF, CSV, Excel, Parquet, Feather
• Flexible schemas: Simple key-value → nested objects → multiple schemas
• Multiple LLM providers: OpenAI, Anthropic, Google, Groq, Together AI, Fireworks AI
• Cost management: Automatic cost tracking, caching, and budget limits
• Built for scale: Batch processing with parallel execution and checkpointing

Installation

pip install delm

Quick Start

Define your extraction schema and extract structured data in just a few lines:

from delm import DELM, Schema, ExtractionVariable

# Define what to extract
schema = Schema.simple(
    variables_list=[
        ExtractionVariable(
            name="company",
            description="Company name mentioned",
            data_type="string",
            required=True,
        ),
        ExtractionVariable(
            name="price",
            description="Price value if mentioned",
            data_type="number",
            required=False,
        ),
    ]
)

# Initialize and extract
delm = DELM(
    schema=schema,
    provider="openai",
    model="gpt-4o-mini",
)

# Extract from any supported file format
results = delm.extract("data/earnings_calls.txt")
print(results)

# Check costs
print(delm.get_cost_summary())

Schema Types

DELM supports three schema types for different extraction needs:

Simple Schema

Extract key-value pairs from text:

schema = Schema.simple(
    variables_list=[
        ExtractionVariable(name="author", data_type="string"),
        ExtractionVariable(name="date", data_type="date"),
    ]
)

Nested Schema

Extract lists of structured objects:

schema = Schema.nested(
    container_name="products",
    variables_list=[
        ExtractionVariable(name="name", data_type="string"),
        ExtractionVariable(name="price", data_type="number"),
        ExtractionVariable(name="features", data_type="[string]"),
    ]
)

Multiple Schemas

Extract multiple different schemas simultaneously:

schema = Schema.multiple({
    "companies": Schema.nested(
        container_name="companies",
        variables_list=[...],
    ),
    "products": Schema.nested(
        container_name="products",
        variables_list=[...],
    ),
})

Supported Data Types

Type	Description	Example
string	Text values	"Apple Inc."
number	Floating-point	150.5
integer	Whole numbers	2024
boolean	True/False	true
date	Date strings	"2025-09-15"
[string]	List of strings	["oil", "gas"]
[number]	List of numbers	[100, 200]

Advanced Features

Custom Prompts

delm = DELM(
    schema=schema,
    provider="openai",
    model="gpt-4o-mini",
    prompt_template="""You are a financial data extraction expert.

Extract the following information:
{variables}

Text to analyze:
{text}""",
)

Process CSV/Structured Data

delm = DELM(
    schema=schema,
    provider="openai",
    model="gpt-4o-mini",
    target_column="transcript_text",  # Column containing text to process
)

results = delm.extract("earnings_data.csv")

Cost Tracking & Limits

delm = DELM(
    schema=schema,
    provider="openai",
    model="gpt-4o-mini",
    track_cost=True,
    max_budget=10.0,  # Stop if cost exceeds $10
)

results = delm.extract("data.txt")
summary = delm.get_cost_summary()
print(f"Total cost: ${summary['total_cost']:.2f}")

Batch Processing

delm = DELM(
    schema=schema,
    provider="openai",
    model="gpt-4o-mini",
    batch_size=50,      # Process 50 records per batch
    max_workers=5,      # Use 5 parallel workers
)

results = delm.extract("large_dataset.csv")

Configuration Options

For a complete list of configuration options, see the documentation.

Common parameters:

• provider: LLM provider ("openai", "anthropic", "google", etc.)
• model: Model name ("gpt-4o-mini", "claude-3-sonnet-20240229", etc.)
• temperature: Generation temperature (default: 0.0)
• batch_size: Records per batch (default: 10)
• max_workers: Concurrent workers (default: 1)
• track_cost: Enable cost tracking (default: True)
• max_budget: Maximum cost limit in dollars (default: None)
• target_column: Column name for CSV/tabular data (default: None)

Documentation

📖 Full Documentation

Learn more about:

• Getting Started Guide
• Schema Design
• Text Processing & Filtering
• Cost Management
• API Reference

File Format Support

Format	Extensions	Additional Dependencies
Text	.txt	None
HTML/Markdown	.html, .htm, .md	beautifulsoup4
Word	.docx	python-docx
PDF	.pdf	marker-pdf
CSV	.csv	pandas
Excel	.xlsx	openpyxl
Parquet	.parquet	pyarrow
Feather	.feather	pyarrow

Contributing

We welcome contributions! Please see our documentation for guidelines.

License

This project is licensed under the MIT License - see the LICENSE.md file for details.

Acknowledgments

• Built on Instructor for structured outputs
• Uses Marker for PDF processing
• Developed at the Center for Applied AI at Chicago Booth