py-chunks

Part of chunk-engine — one Rust engine, three byte-identical SDKs (py-chunks · js-chunks · rs-chunks). Full documentation, playground and benchmarks: chunkengine.dev

PyPI Python License

The Python binding for chunk-engine. Turn any of 36 document formats into typed, structure-aware chunks for RAG — parsing and chunking run in a compiled Rust core, not a stack of Python dependencies.

Install

pip install py-chunks

Python 3.9+, and no runtime dependencies. The Rust engine ships compiled inside the wheel: it parses PDF itself, and vendors PDFium for the one job that needs a rasteriser — rendering a scanned PDF's pages when the file carries no embedded page image of its own.

Quick start

from py_chunks import get_chunks, stream_chunks, get_markdown

# Batch — works for every supported format
chunks = get_chunks("report.pdf", mode="semantic")

for chunk in chunks:
    print(chunk["content_type"])   # "heading", "table", "semantic", …
    print(chunk["content"])
    print(chunk["metadata"])       # format- and mode-specific

# Streaming — one chunk at a time
for chunk in stream_chunks("large.pdf", mode="section"):
    handle(chunk)

# Markdown conversion
md = get_markdown("report.docx")

Every chunk is a dict with content, content_type, and metadata.

📖 Chunking modes · Supported formats · Output schema · Metadata reference

Input sources

get_chunks / stream_chunks auto-detect the source:

Source Example
Path (str / Path) get_chunks("report.pdf")
bytes / bytearray / memoryview get_chunks(data, filename="report.pdf")
File-like (BytesIO, open file) get_chunks(BytesIO(data), filename="doc.md")
FastAPI / Starlette UploadFile get_chunks(upload_file)
HTTP(S) / S3 pre-signed URL get_chunks("https://bucket.s3…/f.pdf?sig=…")

Explicit helpers are also exported — get_chunks_from_path, get_chunks_from_bytes, get_chunks_from_fileobj, get_chunks_from_upload, get_chunks_from_s3_presigned_url, plus the matching stream_chunks_from_*.

A filename is required for bytes and unnamed file objects — dispatch is by extension. get_markdown accepts paths, bytes, and file objects, but not URLs.

Signatures

get_chunks(
    source, *,
    filename: str | None = None,
    mode: str = "default",
    window_size: int = 3,           # sliding_window
    overlap: int = 1,               # sliding_window (must be < window_size)
    sentences_per_chunk: int = 3,   # sentence
    paragraphs_per_page: int = 15,  # page_aware
    list_images: bool = False,
) -> list[dict] | ChunksResult

stream_chunks(source, *, filename=None, mode="default", ...) -> Iterator[dict]

get_markdown(source, *, filename=None, list_images=False) -> str | MarkdownResult

Images

from py_chunks import get_chunks

result = get_chunks("report.docx", list_images=True)   # -> ChunksResult
result.chunks   # text chunks + content_type="image" chunks
result.images   # {"7fdc906103e95537.png": b"...", ...}  <16-hex content hash>.<ext>

get_markdown(..., list_images=True) returns a MarkdownResult with .markdown (carrying ![](hash.ext) refs) and .images. Not available on stream_chunks.

Format-specific chunkers

For parameters the unified API doesn't expose (rows_per_chunk, max_chunk_chars, sheet_names, delimiter, encoding) or for per-call timing, import the format module directly. Each returns (chunks, timing) where timing is {"rust_ms": …, "python_ms": …}:

from py_chunks.chunkers.xlsx import chunk_xlsx, stream_chunk_xlsx
from py_chunks.chunkers.csv  import chunk_csv

chunks, timing = chunk_xlsx("data.xlsx", mode="row", rows_per_chunk=5)
chunks, timing = chunk_csv("data.csv", mode="row", delimiter="\t")

Modules: csv, doc, docx (the OOXML Word family), eml (also .mbox), epub, html, ipynb, json (also .jsonl/.ndjson), md, msg, odf (.odt/.odp), pdf, ppt, pptx (the OOXML PowerPoint family), rtf, tsv, txt, xlsx (the whole spreadsheet family).

Errors

Exception When
FileNotFoundError Path doesn't exist
ValueError Unsupported extension, invalid mode/parameter, missing filename
TypeError Unsupported source type, or an async upload.read()
RuntimeError Engine-level failure (e.g. a PDF with no text layer)
NotImplementedError Unsupported streaming format/mode combination

📖 Error handling · FastAPI / Flask / Django / Litestar / aiohttp / Celery recipes

Develop

pip install maturin
maturin develop --release    # rebuild the extension after any Rust change
python -m pytest -v
python -m pylint py_chunks   # expected: 10.00/10

License

MIT

Project links

Data verified by PyPI on Sep 12, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for ranjan_kudesia from gravatar.com ranjan_kudesia

GitHub Statistics

Data verified by PyPI on Sep 12, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License

MIT License (MIT)

Requires

Python >=3.9

Provides Extra

dev

Tags

chunking document pdf docx docm dotx dotm doc pptx potx potm ppsx ppsm xlsx xls xlsm xlsb ods xltx xltm csv tsv msg outlook rtf epub ebook ipynb jupyter eml mbox email mime odt odp opendocument libreoffice json jsonl ndjson markdown rust

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

py_chunks-0.6.4.tar.gz (646.1 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

py_chunks-0.6.4-cp39-abi3-win_amd64.whl (5.5 MB view details)

Uploaded CPython 3.9+Windows x86-64

py_chunks-0.6.4-cp39-abi3-manylinux_2_28_x86_64.whl (5.4 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.28+ x86-64

py_chunks-0.6.4-cp39-abi3-manylinux_2_28_aarch64.whl (5.3 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.28+ ARM64

py_chunks-0.6.4-cp39-abi3-macosx_11_0_arm64.whl (5.1 MB view details)

Uploaded CPython 3.9+macOS 11.0+ ARM64

py_chunks-0.6.4-cp39-abi3-macosx_10_12_x86_64.whl (5.4 MB view details)

Uploaded CPython 3.9+macOS 10.12+ x86-64

File details

Details for the file py_chunks-0.6.4.tar.gz.

File metadata

  • Download URL: py_chunks-0.6.4.tar.gz
  • Upload date:
  • Size: 646.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for py_chunks-0.6.4.tar.gz
Algorithm Hash digest
SHA256 88c8dfbfef1e93eb312dd575356f040cca0617712d4a0ee58187f726a06b9592
MD5 5564db81331a7cc39ccbcab9b24e5a95
BLAKE2b-256 1227644e7b73dc6eee8e49b008317b1d6d72e5f77e5a243e0a65254d1be65484

See more details on using hashes here.

Provenance

The following attestation bundles were made for py_chunks-0.6.4.tar.gz:

Publisher: release.yml on RanjanKudesia/py-chunks

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file py_chunks-0.6.4-cp39-abi3-win_amd64.whl.

File metadata

  • Download URL: py_chunks-0.6.4-cp39-abi3-win_amd64.whl
  • Upload date:
  • Size: 5.5 MB
  • Tags: CPython 3.9+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for py_chunks-0.6.4-cp39-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 f4ac3e05ef578df5193563b7dd8e5a1adbf4da7a8643019bbeba082d12a3bc69
MD5 cb2c473f8a511c8e8e964b5f101e55ea
BLAKE2b-256 368abd4b53abb0c5f95cbd12564dc3497c15b439f0445f54ba1ecd09acbc212b

See more details on using hashes here.

Provenance

The following attestation bundles were made for py_chunks-0.6.4-cp39-abi3-win_amd64.whl:

Publisher: release.yml on RanjanKudesia/py-chunks

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file py_chunks-0.6.4-cp39-abi3-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for py_chunks-0.6.4-cp39-abi3-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 ebb348f65ffadb617fd4cbc26b93e393bc3b0006a9f55452c8bbfc35250b18a5
MD5 2592e7038c0f57aae09436cca589956c
BLAKE2b-256 440838918e6703b88b2574262b876a354bc0d916e17cd96547fe6349d3362903

See more details on using hashes here.

Provenance

The following attestation bundles were made for py_chunks-0.6.4-cp39-abi3-manylinux_2_28_x86_64.whl:

Publisher: release.yml on RanjanKudesia/py-chunks

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file py_chunks-0.6.4-cp39-abi3-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for py_chunks-0.6.4-cp39-abi3-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 f8de9f871c1b5bbbf1b8bf29d88ef9ee3e8e2921b8945e53f45013cf354d13d8
MD5 17bd3e9bded72a0417777791cb081680
BLAKE2b-256 ab7f2a2ae516b7f2c2de57e3eacc426ba97b67a768245dbd230fa84fa5c8977f

See more details on using hashes here.

Provenance

The following attestation bundles were made for py_chunks-0.6.4-cp39-abi3-manylinux_2_28_aarch64.whl:

Publisher: release.yml on RanjanKudesia/py-chunks

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file py_chunks-0.6.4-cp39-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for py_chunks-0.6.4-cp39-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 6c802e892d2f63541a45f059892276fc0eb8737104f7344653da7b0109456beb
MD5 1cf48cca6449d53bf4430a8862574b03
BLAKE2b-256 5ec8a7cad2d51fc89b7d42e1b13f02de95d03e6c8cf86e55bdde765a0b1a3e85

See more details on using hashes here.

Provenance

The following attestation bundles were made for py_chunks-0.6.4-cp39-abi3-macosx_11_0_arm64.whl:

Publisher: release.yml on RanjanKudesia/py-chunks

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file py_chunks-0.6.4-cp39-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for py_chunks-0.6.4-cp39-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 2c66128b14f49ef9f5b0ee3108b25768e3dfabfe77604206b54fcdd90865f645
MD5 7c87cf0460461525df823d50b909c93c
BLAKE2b-256 9af3b070dc7516dd10fe5e076347a1b15668a8ab66b2f3a84910ff40fd36a2c5

See more details on using hashes here.

Provenance

The following attestation bundles were made for py_chunks-0.6.4-cp39-abi3-macosx_10_12_x86_64.whl:

Publisher: release.yml on RanjanKudesia/py-chunks

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

0.6.4 This release

6 files

0.6.3

6 files

0.6.2

6 files

0.6.1

6 files

0.6.0

5 files

0.5.0

5 files

0.4.7

5 files

0.4.6

5 files

0.4.5

5 files

0.4.4

5 files

0.4.3

5 files

0.4.2

5 files

0.4.1

5 files

0.4.0

5 files

0.3.1

5 files

0.3.0

5 files

0.2.4

5 files

0.2.3

5 files

0.2.2

5 files

0.2.1

5 files

0.1.6

3 files

0.1.5

3 files

0.1.0

3 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page