PyStreamXL

Process massive Excel files with constant memory. No more crashes.

Stream through millions of rows without loading the whole file into memory. Works with complex Excel workbooks—multiple sheets, formulas, merged cells—at a fraction of the cost.

30-Second Start

from pystreamxl import Stream

# Stream a massive Excel file (constant memory)
with Stream("sales_data_2024.xlsx") as stream:
    for row in stream.rows():
        print(f"Sale: ${row['amount']}")
        # Memory = size of ONE row, no matter how big the file

Why PyStreamXL?

The Problem:

• Excel files over 100MB crash when you load them
• Pandas reads entire file into memory (kills your server)
• ETL pipelines can't handle large workbooks
• Processing big spreadsheets is slow and unreliable

The Solution:

• Stream rows one at a time (constant memory usage)
• Process files of any size
• Maintain Excel structure (formulas, formatting, sheets)
• Simple, familiar Python API

Key Features

• Streaming: Read files row-by-row with O(1) memory usage
• Multi-Sheet: Handle workbooks with 100+ sheets
• Formula Support: Preserve Excel formulas (or evaluate them)
• Data Types: Detect and preserve types (dates, numbers, text)
• Fast: 100K+ rows per second
• Filters: Skip rows matching criteria
• Export: Write processed data to CSV, Parquet, or new Excel

Real-World Use Cases

ETL Pipelines:

# Process 10GB Excel file in a stream
with Stream("huge_dataset.xlsx") as stream:
    for row in stream.rows(sheet="Sales"):
        if row['amount'] > 1000:
            send_to_warehouse(row)

Data Validation:

# Check data quality without loading whole file
with Stream("upload.xlsx") as stream:
    errors = []
    for i, row in enumerate(stream.rows()):
        if not is_valid(row):
            errors.append(f"Row {i}: {row}")

Format Conversion:

# Convert Excel to Parquet (memory-efficient)
with Stream("data.xlsx") as stream:
    stream.export("data.parquet", format="parquet")

Performance

vs. Pandas (loads entire file):

Installation

pip install pystreamxl
# or with uv
uv pip install pystreamxl

Documentation

• Quick Start — Stream your first file
• Advanced — Formulas, formatting, multi-sheet
• Performance Tips — Optimize for your use case
• Examples — Real-world workflows

License

Proprietary License - Free to use with explicit attribution. See LICENSE.

PyStreamXL v2.0.0 | Constant-memory Excel streaming | Python 3.10+