fastload

fastload is a lightweight Python library for loading large CSV and Excel files in chunks, with automatic encoding and delimiter detection, and progress bars.

Features

• Chunked reading – process huge files without memory overflow.
• Auto‑detect encoding (via chardet) – handles UTF‑8, CP1251, etc.
• Auto‑detect delimiter – supports ,, ;, \t, |.
• Progress bars with tqdm.
• Excel support for .xlsx and .xlsm (via openpyxl).
• File‑type auto‑detection with load_file().

Installation

pip install fastload

For development:

pip install -e .

Quick Start

CSV

from fastload import load_csv
for chunk in load_csv('huge_file.csv', chunk_size=10000):
    process(chunk)

Excel

from fastload import load_excel
for chunk in load_excel('data.xlsx', sheet_name='Sheet1'):
    process(chunk)

Auto‑detect

from fastload import load_file
for chunk in load_file('data.csv'):
    process(chunk)

Requirements

• Python 3.8+
• Dependencies: openpyxl, tqdm, chardet

License

MIT – see LICENSE

Author

Stepan Bogdanov – GitHub

Contributing

Contributions are welcome – open an issue or submit a pull request.