file-parse-by-bajirao 0.1.0

Universal Document Processor for LLM Processing - extracts text, tables, numeric data, and metadata from multiple file formats

Universal Document Processor

Universal Document Processor for LLM Processing - extracts text, tables, numeric data, and metadata from PDF, DOCX, XLSX, CSV, images, HTML, JSON, Markdown, and text files with advanced OCR, intelligent table detection, and numeric extraction.

Features

• Multi-format Support: PDF, DOCX, XLSX, CSV, HTML, JSON, Markdown, images, and text files
• Advanced OCR: Multi-engine OCR with automatic fallback (Tesseract, EasyOCR, PaddleOCR)
• Intelligent Table Detection: Multiple extraction methods with deduplication
• Numeric Data Extraction: Automatic extraction and analysis of numeric metrics
• LLM-Ready Output: Formatted context for direct use with LLMs
• URL Support: Process documents directly from URLs
• Comprehensive Metadata: File type, page count, word count, confidence scores, and more

Installation

Basic Installation

pip install file_parse_by_bajirao

With Optional OCR Engines

pip install file_parse_by_bajirao[ocr]

With Optional Table Extractors

pip install file_parse_by_bajirao[tables]

With All Optional Dependencies

pip install file_parse_by_bajirao[all]

System Dependencies

For OCR functionality, you may need to install Tesseract:

• Windows: Download from GitHub
• macOS: brew install tesseract
• Linux: sudo apt-get install tesseract-ocr

For PDF processing, you may need Poppler:

• Windows: Download from poppler-windows
• macOS: brew install poppler
• Linux: sudo apt-get install poppler-utils

Quick Start

from file_parse_by_bajirao import UniversalDocumentProcessor

# Initialize processor
processor = UniversalDocumentProcessor()

# Process a document (file path, bytes, or URL)
result = processor.process('document.pdf')

# Get LLM-ready context
llm_context = result.to_llm_context()
print(llm_context)

# Access individual components
print(result.text)
print(result.tables)
print(result.numeric_data)
print(result.metadata)

# Export to JSON
result.to_json("output.json")

Usage Examples

Process Different File Types

from file_parse_by_bajirao import UniversalDocumentProcessor

processor = UniversalDocumentProcessor()

# PDF file
result = processor.process('document.pdf')

# DOCX file
result = processor.process('document.docx')

# XLSX file
result = processor.process('spreadsheet.xlsx')

# CSV file
result = processor.process('data.csv')

# Image file (with OCR)
result = processor.process('scanned_document.png')

# From URL
result = processor.process('https://example.com/document.pdf')

# From bytes
with open('document.pdf', 'rb') as f:
    result = processor.process(f.read())

Access Extracted Data

# Text content
print(result.text)

# Tables
for table in result.tables:
    print(f"Table {table.table_id} on page {table.page_number}")
    print(table.to_markdown())

# Numeric data
if result.numeric_data:
    print("Key Metrics:", result.numeric_data.key_metrics)
    print("Statistics:", result.numeric_data.statistics)

# Metadata
print(f"File type: {result.metadata.file_type}")
print(f"Pages: {result.metadata.page_count}")
print(f"Words: {result.metadata.word_count}")
print(f"Tables found: {result.metadata.table_count}")
print(f"Confidence: {result.metadata.confidence_score:.0%}")

LLM Context Formatting

# Full context with all features
context = result.to_llm_context(
    include_tables=True,
    include_metadata=True,
    include_numeric=True
)

# Limited context (metadata only)
context = result.to_llm_context(
    include_tables=False,
    include_metadata=True,
    include_numeric=False
)

# With length limit
context = result.to_llm_context(max_length=10000)

Export Results

# Export to JSON file
result.to_json("output.json")

# Get JSON string
json_str = result.to_json()

# Convert to dictionary
data_dict = result.to_dict()

API Reference

UniversalDocumentProcessor

Main processor class.

Methods

• process(source: Union[str, bytes, io.BytesIO]) -> ProcessedDocument
  • Process a document from file path, bytes, or URL
  • Returns: ProcessedDocument object

ProcessedDocument

Result object containing all extracted data.

Attributes

• text: str - Extracted text content
• tables: List[TableData] - Extracted tables
• numeric_data: Optional[NumericData] - Numeric data and statistics
• metadata: DocumentMetadata - Document metadata
• errors: List[str] - Processing errors
• warnings: List[str] - Processing warnings

Methods

• to_dict() -> Dict - Convert to dictionary
• to_json(filepath: Optional[str] = None) -> str - Export as JSON
• to_llm_context(...) -> str - Format for LLM consumption

TableData

Extracted table structure.

Attributes

• table_id: str - Unique table identifier
• page_number: int - Page number where table was found
• headers: List[str] - Table headers
• rows: List[List[str]] - Table rows
• confidence: float - Extraction confidence (0-1)
• extraction_method: str - Method used for extraction

Methods

• to_dict() -> Dict - Convert to dictionary
• to_markdown() -> str - Convert to markdown table

NumericData

Extracted numeric data and statistics.

Attributes

• key_metrics: Dict[str, float] - Key numeric metrics
• statistics: Dict[str, Any] - Statistical summaries
• numeric_tables: List[pd.DataFrame] - Numeric-only tables
• extraction_method: str - Extraction method used

DocumentMetadata

Document metadata and processing information.

Attributes

• file_type: str - File type
• file_size: int - File size in bytes
• page_count: int - Number of pages
• text_length: int - Length of extracted text
• word_count: int - Word count
• has_tables: bool - Whether tables were found
• table_count: int - Number of tables found
• has_numeric: bool - Whether numeric data was found
• numeric_count: int - Number of numeric items
• extraction_methods: List[str] - Methods used for extraction
• processing_time: float - Processing time in seconds
• confidence_score: float - Overall confidence score (0-1)

Supported File Formats

Format	Extension	Features
PDF	.pdf	Text extraction, OCR, table extraction
Word	.docx, .doc	Text and table extraction
Excel	.xlsx	Sheet and table extraction
CSV	.csv	Tabular data extraction
HTML	.html	Text and table extraction
JSON	.json	Structured data extraction
Markdown	.md	Text and table extraction
Images	.png, .jpg, .jpeg, .bmp, .tiff, .gif, .webp	OCR text extraction
Text	.txt	Plain text extraction

Requirements

Core Dependencies

• PyMuPDF (fitz)
• python-docx
• Pillow
• pytesseract
• pdf2image
• pandas
• numpy
• opencv-python
• openpyxl
• beautifulsoup4
• lxml
• requests

Optional Dependencies

• OCR: easyocr, paddleocr
• Tables: pdfplumber, camelot-py, tabula-py

License

Apache Software License 2.0

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

Author

Bajirao S. Sali

Support

For issues, questions, or contributions, please visit the GitHub repository.