zcsv

Blazing-fast drop-in replacement for Python's csv module, powered by Rust.

import zcsv as csv — same API, 4-6x faster.

Why zcsv?

Python's built-in csv module is implemented in C but still creates Python objects for every field of every row. For a 100K-row file with 50 columns, that's 5 million string allocations just to iterate.

zcsv eliminates this. The Rust core parses CSV via SIMD instructions into a single contiguous buffer. The Python-facing Row object is a zero-copy cursor — it holds a pointer and an index, nothing more. Python strings are created only when you access a field, and only for the fields you actually use.

Features you won't find elsewhere

• CSV Injection Protection — safe=True escapes =, +, -, @, \t, \r (OWASP best practice)
• RFC 4180 Strict Mode — strict=True validates field counts, quoting rules, with line numbers in errors
• Delimiter Autodetection — frequency analysis in Rust (, ; \t | :)
• Encoding Autodetection — UTF-8, UTF-16 (LE/BE), Latin-1, BOM handling
• Automatic Type Inference — zcsv.read() returns typed list[dict] (int, float, bool, str)

Install

pip install zcsv

From source (requires Rust toolchain):

git clone https://github.com/Seinarukiro2/zcsv.git
cd zcsv
pip install maturin
maturin develop --release

Quick Start

Drop-in replacement

import zcsv as csv

# Exactly like stdlib — but 4x faster
with open("data.csv") as f:
    for row in csv.reader(f):
        print(row[0], row[1])

# DictReader — 6x faster
with open("data.csv") as f:
    for row in csv.DictReader(f):
        print(row["name"], row["age"])

# Writer — 1.4x faster
with open("out.csv", "w", newline="") as f:
    w = csv.writer(f)
    w.writerow(["name", "age"])
    w.writerows([["Alice", "30"], ["Bob", "25"]])

zcsv extensions

import zcsv

# Read with automatic type inference
data = zcsv.read("data.csv")
# [{"name": "Alice", "age": 30, "active": True}, ...]

# Batch reading for large files
for batch in zcsv.read_batches("huge.csv", batch_size=10_000):
    process(batch)

# Write with CSV injection protection (safe=True by default)
zcsv.write("out.csv", data)

API Reference

zcsv.reader(csvfile, **kwargs)

Returns a cursor iterator. Each iteration advances to the next row. Access fields with row[0] (by index) or row.to_list() for a full list.

with open("data.csv") as f:
    for row in zcsv.reader(f):
        name = row[0]          # lazy — creates Python string only now
        last = row[-1]         # negative indexing works
        print(len(row))        # field count
        print(repr(row))       # ['Alice', '30', 'NYC']

Storing rows: The cursor reuses the same object. To collect rows, use snapshot():

with open("data.csv") as f:
    rows = [row.snapshot() for row in zcsv.reader(f)]
    # or: [row.to_list() for row in zcsv.reader(f)]

Parameters: delimiter, quotechar, strict (RFC 4180 validation)

zcsv.DictReader(f, fieldnames=None, **kwargs)

Same cursor pattern with dict-like access:

with open("data.csv") as f:
    for row in zcsv.DictReader(f):
        row["name"]           # by key
        row[0]                # also by index
        row.keys()            # column names
        row.values()          # all values
        row.items()           # (key, value) pairs
        row.get("x", "N/A")  # with default
        "name" in row         # membership test

zcsv.read(path, **kwargs) -> list[dict]

Read entire file with automatic type inference.

zcsv.read("data.csv",
    delimiter=None,       # None = autodetect
    has_header=True,
    schema={"id": int, "price": float},  # override types
    skip_rows=0,
    max_rows=None,
    columns=["name", "age"],  # select columns
    null_values=["", "NA", "null", "None"],
    encoding=None,        # None = autodetect
    strict=False,         # RFC 4180 validation
    n_threads=None,       # parallel type conversion
)

zcsv.write(path, data, **kwargs)

zcsv.write("out.csv", data,
    delimiter=",",
    safe=True,    # CSV injection protection (default: True)
    strict=False,
)

zcsv.writer(csvfile, **kwargs) / zcsv.DictWriter(csvfile, fieldnames, **kwargs)

Stdlib-compatible streaming writer. safe=False by default (stdlib compat).

zcsv.read_batches(path, batch_size=10_000, **kwargs)

Memory-efficient iterator yielding list[dict] batches.

Architecture

Python API        ┌─ reader() ─── cursor Row (zero-copy, lazy strings)
                  ├─ DictReader() ─── cursor with field names
                  ├─ writer() / DictWriter() ─── raw FFI serialization
                  ├─ read() ─── type inference + parallel conversion
                  └─ write() ─── CSV injection protection

Rust Core         ┌─ simd-csv ─── SIMD-accelerated CSV parsing
(PyO3 + FFI)      ├─ SharedData ─── single Vec<u8> buffer for all rows
                  ├─ memmap2 ─── memory-mapped I/O for large files
                  ├─ rayon ─── parallel column type conversion
                  ├─ encoding_rs ─── charset detection + conversion
                  └─ fast_pyobjects ─── raw CPython FFI (PyUnicode_New, PyList_SET_ITEM)

Key design decisions

• Zero-copy Row: All CSV data lives in one contiguous Vec<u8>. Row is Arc<SharedData> + u32 — 12 bytes, no per-row heap allocation. Python strings created only on field access via raw PyUnicode_New.
• Cursor pattern: reader.__next__() returns self with Py_INCREF (~10ns) instead of allocating a new object (~900ns).
• String dedup cache: Repeated values (countries, categories, booleans) are cached. Auto-disables after 200 samples if hit rate < 20%.
• GIL release: File I/O, SIMD parsing, type inference, CSV serialization all run with GIL released.

Benchmarks

100,000 rows, Python 3.13, Apple Silicon M4:

License

MIT