loadfile 0.1.0

Load tabular data from any storage backend (GCS, S3, Azure, local) and any common format into a pandas DataFrame with a single call.

loadfile

Load tabular data from any storage backend (Google Cloud Storage, AWS S3, Azure, local disk, …) and any common format into a pandas DataFrame with a single call.

from loadfile import load

df = load("gs://my-bucket/data.parquet")
df = load("data/local.csv")
df = load("s3://bucket/export.json", storage_options={"anon": True})

The storage backend is chosen automatically from the URL prefix (powered by fsspec). The file format is detected from the extension (or forced with format=).

The function is also exported as load_data (an alias of load) for compatibility with the original helper.

Install

pip install loadfile            # core: pandas + fsspec + pyarrow (local files)
pip install "loadfile[gcs]"     # Google Cloud Storage (gcsfs)
pip install "loadfile[s3]"      # AWS S3 (s3fs)
pip install "loadfile[azure]"   # Azure (adlfs)
pip install "loadfile[excel]"   # .xlsx support (openpyxl)
pip install "loadfile[all]"     # everything

Supported formats

Format	Extensions
Parquet	.parquet
CSV	.csv (and .csv.gz, .csv.bz2…)
JSON / JSON Lines	.json, .jsonl, .ndjson
Excel	.xlsx, .xls
Feather	.feather
ZIP	.zip (wrapping any of the above)

Usage

Format override and reader options

Any extra keyword arguments are forwarded to the underlying pandas reader:

load("export.tsv", format="csv", sep="\t")
load("book.xlsx", sheet_name="Sheet2")
load("big.csv", usecols=["id", "value"])

Credentials

Pass backend options via storage_options (forwarded to fsspec):

load("gs://bucket/data.parquet", storage_options={"token": "/path/key.json"})
load("s3://bucket/data.csv", storage_options={"key": "...", "secret": "..."})

ZIP archives

A .zip may contain any mix of supported formats. Each member is loaded with the reader matching its own extension.

# Single data file in the zip -> a DataFrame
df = load("archive.zip")

# Multiple data files -> a dict {member_name: DataFrame}
frames = load("archive.zip")               # {"a.csv": ..., "b.parquet": ...}

# Pick one member -> a DataFrame
df = load("archive.zip", filename="a.csv")

# Pick a subset -> a dict of just those members
frames = load("archive.zip", filename=["a.csv", "b.parquet"])

Non-data members (e.g. readme.txt) and nested zips are ignored. A zip with no supported files raises EmptyArchiveError; a missing requested member raises MemberNotFoundError.

Development

pip install -e ".[dev,all]"
pytest

License

MIT