py-chunks

Part of chunk-engine — one Rust engine, three byte-identical SDKs (py-chunks · js-chunks · rs-chunks). Full documentation, playground and benchmarks: chunkengine.dev

PyPI Python License

The Python binding for chunk-engine. Turn any of 36 document formats into typed, structure-aware chunks for RAG — parsing and chunking run in a compiled Rust core, not a stack of Python dependencies.

Install

pip install py-chunks

Python 3.9+, and no runtime dependencies. The Rust engine ships compiled inside the wheel: it parses PDF itself, and vendors PDFium for the one job that needs a rasteriser — rendering a scanned PDF's pages when the file carries no embedded page image of its own.

Quick start

from py_chunks import get_chunks, stream_chunks, get_markdown

# Batch — works for every supported format
chunks = get_chunks("report.pdf", mode="semantic")

for chunk in chunks:
    print(chunk["content_type"])   # "heading", "table", "semantic", …
    print(chunk["content"])
    print(chunk["metadata"])       # format- and mode-specific

# Streaming — one chunk at a time
for chunk in stream_chunks("large.pdf", mode="section"):
    handle(chunk)

# Markdown conversion
md = get_markdown("report.docx")

Every chunk is a dict with content, content_type, and metadata.

📖 Chunking modes · Supported formats · Output schema · Metadata reference

Input sources

get_chunks / stream_chunks auto-detect the source:

Source Example
Path (str / Path) get_chunks("report.pdf")
bytes / bytearray / memoryview get_chunks(data, filename="report.pdf")
File-like (BytesIO, open file) get_chunks(BytesIO(data), filename="doc.md")
FastAPI / Starlette UploadFile get_chunks(upload_file)
HTTP(S) / S3 pre-signed URL get_chunks("https://bucket.s3…/f.pdf?sig=…")

Explicit helpers are also exported — get_chunks_from_path, get_chunks_from_bytes, get_chunks_from_fileobj, get_chunks_from_upload, get_chunks_from_s3_presigned_url, plus the matching stream_chunks_from_*.

A filename is required for bytes and unnamed file objects — dispatch is by extension. get_markdown accepts paths, bytes, and file objects, but not URLs.

Signatures

get_chunks(
    source, *,
    filename: str | None = None,
    mode: str = "default",
    window_size: int = 3,           # sliding_window
    overlap: int = 1,               # sliding_window (must be < window_size)
    sentences_per_chunk: int = 3,   # sentence
    paragraphs_per_page: int = 15,  # page_aware
    list_images: bool = False,
) -> list[dict] | ChunksResult

stream_chunks(source, *, filename=None, mode="default", ...) -> Iterator[dict]

get_markdown(source, *, filename=None, list_images=False) -> str | MarkdownResult

Images

from py_chunks import get_chunks

result = get_chunks("report.docx", list_images=True)   # -> ChunksResult
result.chunks   # text chunks + content_type="image" chunks
result.images   # {"7fdc906103e95537.png": b"...", ...}  <16-hex content hash>.<ext>

get_markdown(..., list_images=True) returns a MarkdownResult with .markdown (carrying ![](hash.ext) refs) and .images. Not available on stream_chunks.

Format-specific chunkers

For parameters the unified API doesn't expose (rows_per_chunk, max_chunk_chars, sheet_names, delimiter, encoding) or for per-call timing, import the format module directly. Each returns (chunks, timing) where timing is {"rust_ms": …, "python_ms": …}:

from py_chunks.chunkers.xlsx import chunk_xlsx, stream_chunk_xlsx
from py_chunks.chunkers.csv  import chunk_csv

chunks, timing = chunk_xlsx("data.xlsx", mode="row", rows_per_chunk=5)
chunks, timing = chunk_csv("data.csv", mode="row", delimiter="\t")

Modules: csv, doc, docx (the OOXML Word family), eml (also .mbox), epub, html, ipynb, json (also .jsonl/.ndjson), md, msg, odf (.odt/.odp), pdf, ppt, pptx (the OOXML PowerPoint family), rtf, tsv, txt, xlsx (the whole spreadsheet family).

Errors

Exception When
FileNotFoundError Path doesn't exist
ValueError Unsupported extension, invalid mode/parameter, missing filename
TypeError Unsupported source type, or an async upload.read()
RuntimeError Engine-level failure (e.g. a PDF with no text layer)
NotImplementedError Unsupported streaming format/mode combination

📖 Error handling · FastAPI / Flask / Django / Litestar / aiohttp / Celery recipes

Develop

pip install maturin
maturin develop --release    # rebuild the extension after any Rust change
python -m pytest -v
python -m pylint py_chunks   # expected: 10.00/10

License

MIT

Project links

Data verified by PyPI on Aug 8, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.6.4)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for ranjan_kudesia from gravatar.com ranjan_kudesia

GitHub Statistics

Data verified by PyPI on Aug 8, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License

MIT License (MIT)

Requires

Python >=3.9

Provides Extra

dev

Tags

chunking document pdf docx docm dotx dotm doc pptx potx potm ppsx ppsm xlsx xls xlsm xlsb ods xltx xltm csv tsv msg outlook rtf epub ebook ipynb jupyter eml mbox email mime odt odp opendocument libreoffice json jsonl ndjson markdown rust

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

py_chunks-0.6.2.tar.gz (558.3 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

py_chunks-0.6.2-cp39-abi3-win_amd64.whl (5.2 MB view details)

Uploaded CPython 3.9+Windows x86-64

py_chunks-0.6.2-cp39-abi3-manylinux_2_28_x86_64.whl (5.2 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.28+ x86-64

py_chunks-0.6.2-cp39-abi3-manylinux_2_28_aarch64.whl (5.1 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.28+ ARM64

py_chunks-0.6.2-cp39-abi3-macosx_11_0_arm64.whl (5.0 MB view details)

Uploaded CPython 3.9+macOS 11.0+ ARM64

py_chunks-0.6.2-cp39-abi3-macosx_10_12_x86_64.whl (5.2 MB view details)

Uploaded CPython 3.9+macOS 10.12+ x86-64

File details

Details for the file py_chunks-0.6.2.tar.gz.

File metadata

  • Download URL: py_chunks-0.6.2.tar.gz
  • Upload date:
  • Size: 558.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for py_chunks-0.6.2.tar.gz
Algorithm Hash digest
SHA256 34d459379c98748fec87aece486b26d007ceb1aa11f0a13d9381f4b08025c7a2
MD5 48a35f838d7c803a647072863e410edb
BLAKE2b-256 dd460de0e665957bdd285a0d6367a873b44c5c99aab0158c26dfc6b1cdfaa32f

See more details on using hashes here.

Provenance

The following attestation bundles were made for py_chunks-0.6.2.tar.gz:

Publisher: release.yml on RanjanKudesia/py-chunks

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file py_chunks-0.6.2-cp39-abi3-win_amd64.whl.

File metadata

  • Download URL: py_chunks-0.6.2-cp39-abi3-win_amd64.whl
  • Upload date:
  • Size: 5.2 MB
  • Tags: CPython 3.9+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for py_chunks-0.6.2-cp39-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 fd4aaf490a40224c62d3052402f0e322fbd68888407b9ac53b942924284d88e8
MD5 74e7370921ad2e2d6ea26c035fa63ebf
BLAKE2b-256 28af5f55d06223381168b18761e1c49ccfb71878008d554211a671c6e7688fd5

See more details on using hashes here.

Provenance

The following attestation bundles were made for py_chunks-0.6.2-cp39-abi3-win_amd64.whl:

Publisher: release.yml on RanjanKudesia/py-chunks

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file py_chunks-0.6.2-cp39-abi3-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for py_chunks-0.6.2-cp39-abi3-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 23a7e424dd04bc3e611e988020fd4e977e54a158229a09f51b7bacc5fb8d3057
MD5 26020554d910ed5a4a1b0ae492bc65b1
BLAKE2b-256 0da58b98884490b916e11e5b93420c26aea716d5447c041690a6e1f7595fc074

See more details on using hashes here.

Provenance

The following attestation bundles were made for py_chunks-0.6.2-cp39-abi3-manylinux_2_28_x86_64.whl:

Publisher: release.yml on RanjanKudesia/py-chunks

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file py_chunks-0.6.2-cp39-abi3-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for py_chunks-0.6.2-cp39-abi3-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 3e5427a8a2ad68c961ed4cc400ac2f55a2529f38a207e1c13f5f7cb6367956d1
MD5 3313f3136ac50d5a8f9b3b64c456f717
BLAKE2b-256 00b56cd615650d90ac1b6b00dbeecd3b41d14d97e7d2b4ef0edc2cd4797993c1

See more details on using hashes here.

Provenance

The following attestation bundles were made for py_chunks-0.6.2-cp39-abi3-manylinux_2_28_aarch64.whl:

Publisher: release.yml on RanjanKudesia/py-chunks

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file py_chunks-0.6.2-cp39-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for py_chunks-0.6.2-cp39-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 d3405a326864cf9da82232c5978be358832c92f93952a9113dbb0165f850eebe
MD5 cad2814aa69cf9625afe2ac0751a1378
BLAKE2b-256 91a54fc029d024cf06c43e468c82f3a70fc9d032df4d1e223d6e79db579e0126

See more details on using hashes here.

Provenance

The following attestation bundles were made for py_chunks-0.6.2-cp39-abi3-macosx_11_0_arm64.whl:

Publisher: release.yml on RanjanKudesia/py-chunks

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file py_chunks-0.6.2-cp39-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for py_chunks-0.6.2-cp39-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 12027c833d723bfb014e18ec9ecd20f5ef2ed00c163ba10a10bca9af69ad1c05
MD5 49523958f5c4c710dc13e7043fd286f2
BLAKE2b-256 bdaa3d5b70dfc771178272a52361596c5efdcf31054fe0b7c948b6e90ded76a9

See more details on using hashes here.

Provenance

The following attestation bundles were made for py_chunks-0.6.2-cp39-abi3-macosx_10_12_x86_64.whl:

Publisher: release.yml on RanjanKudesia/py-chunks

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.6.4

6 files

0.6.3

6 files

This release

0.6.2 This release

6 files

0.6.1

6 files

0.6.0

5 files

0.5.0

5 files

0.4.7

5 files

0.4.6

5 files

0.4.5

5 files

0.4.4

5 files

0.4.3

5 files

0.4.2

5 files

0.4.1

5 files

0.4.0

5 files

0.3.1

5 files

0.3.0

5 files

0.2.4

5 files

0.2.3

5 files

0.2.2

5 files

0.2.1

5 files

0.1.6

3 files

0.1.5

3 files

0.1.0

3 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page