py-chunks

Part of chunk-engine — one Rust engine, three byte-identical SDKs (py-chunks · js-chunks · rs-chunks). Full documentation, playground and benchmarks: chunkengine.dev

PyPI Python License

The Python binding for chunk-engine. Turn any of 36 document formats into typed, structure-aware chunks for RAG — parsing and chunking run in a compiled Rust core, not a stack of Python dependencies.

Install

pip install py-chunks

Python 3.9+, and no runtime dependencies. The Rust engine ships compiled inside the wheel: it parses PDF itself, and vendors PDFium for the one job that needs a rasteriser — rendering a scanned PDF's pages when the file carries no embedded page image of its own.

Quick start

from py_chunks import get_chunks, stream_chunks, get_markdown

# Batch — works for every supported format
chunks = get_chunks("report.pdf", mode="semantic")

for chunk in chunks:
    print(chunk["content_type"])   # "heading", "table", "semantic", …
    print(chunk["content"])
    print(chunk["metadata"])       # format- and mode-specific

# Streaming — one chunk at a time
for chunk in stream_chunks("large.pdf", mode="section"):
    handle(chunk)

# Markdown conversion
md = get_markdown("report.docx")

Every chunk is a dict with content, content_type, and metadata.

📖 Chunking modes · Supported formats · Output schema · Metadata reference

Input sources

get_chunks / stream_chunks auto-detect the source:

Source Example
Path (str / Path) get_chunks("report.pdf")
bytes / bytearray / memoryview get_chunks(data, filename="report.pdf")
File-like (BytesIO, open file) get_chunks(BytesIO(data), filename="doc.md")
FastAPI / Starlette UploadFile get_chunks(upload_file)
HTTP(S) / S3 pre-signed URL get_chunks("https://bucket.s3…/f.pdf?sig=…")

Explicit helpers are also exported — get_chunks_from_path, get_chunks_from_bytes, get_chunks_from_fileobj, get_chunks_from_upload, get_chunks_from_s3_presigned_url, plus the matching stream_chunks_from_*.

A filename is required for bytes and unnamed file objects — dispatch is by extension. get_markdown accepts paths, bytes, and file objects, but not URLs.

Signatures

get_chunks(
    source, *,
    filename: str | None = None,
    mode: str = "default",
    window_size: int = 3,           # sliding_window
    overlap: int = 1,               # sliding_window (must be < window_size)
    sentences_per_chunk: int = 3,   # sentence
    paragraphs_per_page: int = 15,  # page_aware
    list_images: bool = False,
) -> list[dict] | ChunksResult

stream_chunks(source, *, filename=None, mode="default", ...) -> Iterator[dict]

get_markdown(source, *, filename=None, list_images=False) -> str | MarkdownResult

Images

from py_chunks import get_chunks

result = get_chunks("report.docx", list_images=True)   # -> ChunksResult
result.chunks   # text chunks + content_type="image" chunks
result.images   # {"7fdc906103e95537.png": b"...", ...}  <16-hex content hash>.<ext>

get_markdown(..., list_images=True) returns a MarkdownResult with .markdown (carrying ![](hash.ext) refs) and .images. Not available on stream_chunks.

Format-specific chunkers

For parameters the unified API doesn't expose (rows_per_chunk, max_chunk_chars, sheet_names, delimiter, encoding) or for per-call timing, import the format module directly. Each returns (chunks, timing) where timing is {"rust_ms": …, "python_ms": …}:

from py_chunks.chunkers.xlsx import chunk_xlsx, stream_chunk_xlsx
from py_chunks.chunkers.csv  import chunk_csv

chunks, timing = chunk_xlsx("data.xlsx", mode="row", rows_per_chunk=5)
chunks, timing = chunk_csv("data.csv", mode="row", delimiter="\t")

Modules: csv, doc, docx (the OOXML Word family), eml (also .mbox), epub, html, ipynb, json (also .jsonl/.ndjson), md, msg, odf (.odt/.odp), pdf, ppt, pptx (the OOXML PowerPoint family), rtf, tsv, txt, xlsx (the whole spreadsheet family).

Errors

Exception When
FileNotFoundError Path doesn't exist
ValueError Unsupported extension, invalid mode/parameter, missing filename
TypeError Unsupported source type, or an async upload.read()
RuntimeError Engine-level failure (e.g. a PDF with no text layer)
NotImplementedError Unsupported streaming format/mode combination

📖 Error handling · FastAPI / Flask / Django / Litestar / aiohttp / Celery recipes

Develop

pip install maturin
maturin develop --release    # rebuild the extension after any Rust change
python -m pytest -v
python -m pylint py_chunks   # expected: 10.00/10

License

MIT

Project links

Data verified by PyPI on Sep 1, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.6.4)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for ranjan_kudesia from gravatar.com ranjan_kudesia

GitHub Statistics

Data verified by PyPI on Sep 1, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License

MIT License (MIT)

Requires

Python >=3.9

Provides Extra

dev

Tags

chunking document pdf docx docm dotx dotm doc pptx potx potm ppsx ppsm xlsx xls xlsm xlsb ods xltx xltm csv tsv msg outlook rtf epub ebook ipynb jupyter eml mbox email mime odt odp opendocument libreoffice json jsonl ndjson markdown rust

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

py_chunks-0.6.3.tar.gz (639.7 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

py_chunks-0.6.3-cp39-abi3-win_amd64.whl (5.5 MB view details)

Uploaded CPython 3.9+Windows x86-64

py_chunks-0.6.3-cp39-abi3-manylinux_2_28_x86_64.whl (5.4 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.28+ x86-64

py_chunks-0.6.3-cp39-abi3-manylinux_2_28_aarch64.whl (5.3 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.28+ ARM64

py_chunks-0.6.3-cp39-abi3-macosx_11_0_arm64.whl (5.3 MB view details)

Uploaded CPython 3.9+macOS 11.0+ ARM64

py_chunks-0.6.3-cp39-abi3-macosx_10_12_x86_64.whl (5.4 MB view details)

Uploaded CPython 3.9+macOS 10.12+ x86-64

File details

Details for the file py_chunks-0.6.3.tar.gz.

File metadata

  • Download URL: py_chunks-0.6.3.tar.gz
  • Upload date:
  • Size: 639.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for py_chunks-0.6.3.tar.gz
Algorithm Hash digest
SHA256 600423010fa0bd1e76eba51a85ade789751db66fc9361b57df9af81aef3866bd
MD5 af939bb3edb3e0ef1ffbbb49603e2f0a
BLAKE2b-256 e2e6a6e3459cc167e6c544f9a08e6954876d3fb4bc7d5cee5faa8c4c12f5f3ee

See more details on using hashes here.

Provenance

The following attestation bundles were made for py_chunks-0.6.3.tar.gz:

Publisher: release.yml on RanjanKudesia/py-chunks

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file py_chunks-0.6.3-cp39-abi3-win_amd64.whl.

File metadata

  • Download URL: py_chunks-0.6.3-cp39-abi3-win_amd64.whl
  • Upload date:
  • Size: 5.5 MB
  • Tags: CPython 3.9+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for py_chunks-0.6.3-cp39-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 cb5dab389be85050c16e999ab810dcb7df406948c691d276159241cfef40ad17
MD5 882a1dda39be67c7a2be025ee1b73db7
BLAKE2b-256 11389b9b0e36e982575ace853e7490a0f22420cbecc3ea413b3767c7498f37fa

See more details on using hashes here.

Provenance

The following attestation bundles were made for py_chunks-0.6.3-cp39-abi3-win_amd64.whl:

Publisher: release.yml on RanjanKudesia/py-chunks

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file py_chunks-0.6.3-cp39-abi3-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for py_chunks-0.6.3-cp39-abi3-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 c83451b0b7aa89df40cc37969a14f26884d99da82b68ca8a9004d1d0b27c581f
MD5 a7fd14d04d99d14a329d8c49fc3e0843
BLAKE2b-256 158dc07f955a71f375f3608fc2fe8687c4e623730d0adb5d45a29d6ee2f743c7

See more details on using hashes here.

Provenance

The following attestation bundles were made for py_chunks-0.6.3-cp39-abi3-manylinux_2_28_x86_64.whl:

Publisher: release.yml on RanjanKudesia/py-chunks

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file py_chunks-0.6.3-cp39-abi3-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for py_chunks-0.6.3-cp39-abi3-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 117863564d3d59a1c7c7caba3d491c75ad39c31f116581db94e4e2f7c24c76d8
MD5 d3a96a0c620bd40fb2609c1b7103109b
BLAKE2b-256 08a5ae6c2a95d8a25b11c0e91807b0fc94080caa042ee6acea03b41933cc3f6e

See more details on using hashes here.

Provenance

The following attestation bundles were made for py_chunks-0.6.3-cp39-abi3-manylinux_2_28_aarch64.whl:

Publisher: release.yml on RanjanKudesia/py-chunks

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file py_chunks-0.6.3-cp39-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for py_chunks-0.6.3-cp39-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 561f9d016d3513ceb7dd153d720d474123c4fb201256b0458beef69635835124
MD5 c59b7224d3d255bda349e1c010e36157
BLAKE2b-256 eb4b3cec3b41baa28ff4b57d08cb96f69b646928508bceca9077c74929d44db9

See more details on using hashes here.

Provenance

The following attestation bundles were made for py_chunks-0.6.3-cp39-abi3-macosx_11_0_arm64.whl:

Publisher: release.yml on RanjanKudesia/py-chunks

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file py_chunks-0.6.3-cp39-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for py_chunks-0.6.3-cp39-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 4b4ffbe4083ee64f5dced4e4308c19dfa3f7e7a3890f46a21cfec4271d1cceb9
MD5 bfb73f25b1740326d009c1e3a03dbd2e
BLAKE2b-256 8887ab5005a07b0a881ff3178219c07e825a407b59eec43709e8d2d37ecc9ca5

See more details on using hashes here.

Provenance

The following attestation bundles were made for py_chunks-0.6.3-cp39-abi3-macosx_10_12_x86_64.whl:

Publisher: release.yml on RanjanKudesia/py-chunks

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.6.4

6 files

This release

0.6.3 This release

6 files

0.6.2

6 files

0.6.1

6 files

0.6.0

5 files

0.5.0

5 files

0.4.7

5 files

0.4.6

5 files

0.4.5

5 files

0.4.4

5 files

0.4.3

5 files

0.4.2

5 files

0.4.1

5 files

0.4.0

5 files

0.3.1

5 files

0.3.0

5 files

0.2.4

5 files

0.2.3

5 files

0.2.2

5 files

0.2.1

5 files

0.1.6

3 files

0.1.5

3 files

0.1.0

3 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page