structx-llm 0.4.0

Structured data extraction from text using LLMs and dynamic model generation

structx

Advanced structured data extraction from any document using LLMs with multimodal support.

structx is a powerful Python library for extracting structured data from any document or text using Large Language Models (LLMs). It features an innovative multimodal PDF processing pipeline that converts any document to PDF and uses instructor's vision capabilities for superior extraction quality.

✨ Key Features

🎯 Advanced Document Processing

• � Multimodal PDF Pipeline: Converts any document (TXT, DOCX, etc.) to PDF for optimal extraction
• 🖼️ Vision-Enabled Extraction: Native instructor multimodal support for PDFs and images
• 🔄 Smart Format Detection: Automatic processing mode selection for best results
• 📊 Universal File Support: CSV, Excel, JSON, Parquet, PDF, DOCX, TXT, Markdown, and more

🚀 Intelligent Data Extraction

• 🔄 Dynamic Model Generation: Create type-safe Pydantic models from natural language queries
• 🎯 Automatic Schema Inference: Intelligent schema generation and refinement
• 📊 Complex Data Structures: Support for nested and hierarchical data
• 🔄 Natural Language Refinement: Improve models with conversational instructions

⚡ Performance & Reliability

• 🚀 High-Performance Processing: Multi-threaded and async operations
• 🔄 Robust Error Handling: Automatic retry mechanism with exponential backoff
• 📈 Token Usage Tracking: Detailed step-by-step metrics for cost monitoring
• � Flexible Configuration: Configurable extraction using OmegaConf
• 🔌 Multiple LLM Providers: Support through litellm integration

Installation

# Core package with basic extraction capabilities
pip install structx-llm

📄 Enhanced Document Processing (Recommended)

For the best experience with all document types including advanced multimodal PDF processing:

# Complete document processing support
pip install structx-llm[docs]

# Individual components
pip install structx-llm[pdf]   # PDF processing with multimodal support
pip install structx-llm[docx]  # Advanced DOCX conversion via docling

🔧 What Each Extra Provides

• [docs]: Complete multimodal document processing pipeline

  • PDF conversion from any document type
  • Instructor multimodal vision support
  • Advanced DOCX processing via docling
  • Enhanced extraction quality

• [pdf]: PDF-specific processing

  • Multimodal PDF support via instructor
  • PDF generation capabilities
  • Basic PDF text extraction fallback

• [docx]: Advanced DOCX support

  • Document conversion via docling
  • Structure preservation
  • Markdown-based processing pipeline

Quick Start

Basic Text Extraction

from structx import Extractor

# Initialize extractor
extractor = Extractor.from_litellm(
    model="gpt-4o-mini",
    api_key="your-api-key",
    max_retries=3,      # Automatically retry on transient errors
    min_wait=1,         # Start with 1 second wait
    max_wait=10         # Maximum 10 seconds between retries
)

# Extract from text
result = extractor.extract(
    data="System check on 2024-01-15 detected high CPU usage (92%) on server-01.",
    query="extract incident date and details"
)

# Access results
print(f"Extracted {result.success_count} items")
print(result.data[0].model_dump_json(indent=2))

📄 Document Processing with Multimodal Support

# Process PDF documents directly with vision capabilities
result = extractor.extract(
    data="financial_report.pdf",      # Direct multimodal processing
    query="extract revenue figures, profit margins, and key financial metrics"
)

# Convert DOCX to PDF and process with multimodal support
result = extractor.extract(
    data="contract.docx",             # Auto-converted via docling → PDF → multimodal
    query="extract parties, dates, payment terms, and key obligations"
)

# Process any text file with enhanced PDF conversion
result = extractor.extract(
    data="meeting_notes.txt",         # Converted to styled PDF → multimodal
    query="extract action items, deadlines, and responsible parties"
)

📊 Token Usage Monitoring

# Check token usage for cost monitoring
usage = result.get_token_usage()
if usage:
    print(f"Total tokens: {usage.total_tokens}")
    print(f"By step: {[(s.name, s.tokens) for s in usage.steps]}")

🚀 Why Multimodal PDF Processing?

The innovative multimodal approach provides significant advantages over traditional text-based extraction:

• 📄 Context Preservation: Full document layout and structure are maintained
• 🎯 Higher Accuracy: Vision models can interpret tables, charts, and complex layouts
• 🔄 No Chunking Issues: Eliminates problems with information split across chunks
• 📊 Universal Format: Any document type becomes processable through PDF conversion
• 🖼️ Visual Understanding: Handles documents with visual elements, formatting, and structure

📚 Documentation

For comprehensive documentation, examples, and guides, visit our documentation site.

• Getting Started
• Basic Extraction
• Unstructured Text Processing
• Async Operations
• Multiple Queries
• Custom Models
• Token Usage Tracking
• API Reference

Examples

Check out our example gallery for real-world use cases,

📁 Supported File Formats

📊 Structured Data (Direct Processing)

• CSV: Comma-separated values with custom delimiters
• Excel: .xlsx/.xls with sheet selection and custom options
• JSON: JavaScript Object Notation with nested support
• Parquet: Columnar storage format for large datasets
• Feather: Fast binary format for data frames

📄 Unstructured Documents (Multimodal Pipeline)

Format	Extensions	Processing Method	Quality
PDF	.pdf	Direct multimodal processing	⭐⭐⭐⭐⭐
Word	.docx, .doc	Docling → Markdown → PDF → Multimodal	⭐⭐⭐⭐⭐
Text	.txt, .md, .py, .log, .xml, .html	Styled PDF → Multimodal	⭐⭐⭐⭐

🔄 Processing Modes

• Multimodal PDF (default): Best quality, preserves layout and context
• Simple Text: Fallback mode with chunking for memory-constrained environments
• Simple PDF: Basic PDF text extraction without vision capabilities

Contributing

Contributions are welcome! Please read our Contributing Guidelines for details.

License

This project is licensed under the MIT License - see the LICENSE file for details.