py-chunks

Part of chunk-engine — one Rust engine, three byte-identical SDKs (py-chunks · js-chunks · rs-chunks). Full documentation, playground and benchmarks: chunkengine.dev

PyPI Python License

The Python binding for chunk-engine. Turn any of 36 document formats into typed, structure-aware chunks for RAG — parsing and chunking run in a compiled Rust core, not a stack of Python dependencies.

Install

pip install py-chunks

Python 3.9+, and no runtime dependencies. The Rust engine ships compiled inside the wheel: it parses PDF itself, and vendors PDFium for the one job that needs a rasteriser — rendering a scanned PDF's pages when the file carries no embedded page image of its own.

Quick start

from py_chunks import get_chunks, stream_chunks, get_markdown

# Batch — works for every supported format
chunks = get_chunks("report.pdf", mode="semantic")

for chunk in chunks:
    print(chunk["content_type"])   # "heading", "table", "semantic", …
    print(chunk["content"])
    print(chunk["metadata"])       # format- and mode-specific

# Streaming — one chunk at a time
for chunk in stream_chunks("large.pdf", mode="section"):
    handle(chunk)

# Markdown conversion
md = get_markdown("report.docx")

Every chunk is a dict with content, content_type, and metadata.

📖 Chunking modes · Supported formats · Output schema · Metadata reference

Input sources

get_chunks / stream_chunks auto-detect the source:

Source Example
Path (str / Path) get_chunks("report.pdf")
bytes / bytearray / memoryview get_chunks(data, filename="report.pdf")
File-like (BytesIO, open file) get_chunks(BytesIO(data), filename="doc.md")
FastAPI / Starlette UploadFile get_chunks(upload_file)
HTTP(S) / S3 pre-signed URL get_chunks("https://bucket.s3…/f.pdf?sig=…")

Explicit helpers are also exported — get_chunks_from_path, get_chunks_from_bytes, get_chunks_from_fileobj, get_chunks_from_upload, get_chunks_from_s3_presigned_url, plus the matching stream_chunks_from_*.

A filename is required for bytes and unnamed file objects — dispatch is by extension. get_markdown accepts paths, bytes, and file objects, but not URLs.

Signatures

get_chunks(
    source, *,
    filename: str | None = None,
    mode: str = "default",
    window_size: int = 3,           # sliding_window
    overlap: int = 1,               # sliding_window (must be < window_size)
    sentences_per_chunk: int = 3,   # sentence
    paragraphs_per_page: int = 15,  # page_aware
    list_images: bool = False,
) -> list[dict] | ChunksResult

stream_chunks(source, *, filename=None, mode="default", ...) -> Iterator[dict]

get_markdown(source, *, filename=None, list_images=False) -> str | MarkdownResult

Images

from py_chunks import get_chunks

result = get_chunks("report.docx", list_images=True)   # -> ChunksResult
result.chunks   # text chunks + content_type="image" chunks
result.images   # {"7fdc906103e95537.png": b"...", ...}  <16-hex content hash>.<ext>

get_markdown(..., list_images=True) returns a MarkdownResult with .markdown (carrying ![](hash.ext) refs) and .images. Not available on stream_chunks.

Format-specific chunkers

For parameters the unified API doesn't expose (rows_per_chunk, max_chunk_chars, sheet_names, delimiter, encoding) or for per-call timing, import the format module directly. Each returns (chunks, timing) where timing is {"rust_ms": …, "python_ms": …}:

from py_chunks.chunkers.xlsx import chunk_xlsx, stream_chunk_xlsx
from py_chunks.chunkers.csv  import chunk_csv

chunks, timing = chunk_xlsx("data.xlsx", mode="row", rows_per_chunk=5)
chunks, timing = chunk_csv("data.csv", mode="row", delimiter="\t")

Modules: csv, doc, docx (the OOXML Word family), eml (also .mbox), epub, html, ipynb, json (also .jsonl/.ndjson), md, msg, odf (.odt/.odp), pdf, ppt, pptx (the OOXML PowerPoint family), rtf, tsv, txt, xlsx (the whole spreadsheet family).

Errors

Exception When
FileNotFoundError Path doesn't exist
ValueError Unsupported extension, invalid mode/parameter, missing filename
TypeError Unsupported source type, or an async upload.read()
RuntimeError Engine-level failure (e.g. a PDF with no text layer)
NotImplementedError Unsupported streaming format/mode combination

📖 Error handling · FastAPI / Flask / Django / Litestar / aiohttp / Celery recipes

Develop

pip install maturin
maturin develop --release    # rebuild the extension after any Rust change
python -m pytest -v
python -m pylint py_chunks   # expected: 10.00/10

License

MIT

Project links

Data verified by PyPI on Aug 7, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.6.4)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for ranjan_kudesia from gravatar.com ranjan_kudesia

GitHub Statistics

Data verified by PyPI on Aug 7, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License

MIT License (MIT)

Requires

Python >=3.9

Provides Extra

dev

Tags

chunking document pdf docx docm dotx dotm doc pptx potx potm ppsx ppsm xlsx xls xlsm xlsb ods xltx xltm csv tsv msg outlook rtf epub ebook ipynb jupyter eml mbox email mime odt odp opendocument libreoffice json jsonl ndjson markdown rust

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

py_chunks-0.6.1.tar.gz (547.4 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

py_chunks-0.6.1-cp39-abi3-win_amd64.whl (5.2 MB view details)

Uploaded CPython 3.9+Windows x86-64

py_chunks-0.6.1-cp39-abi3-manylinux_2_28_x86_64.whl (5.2 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.28+ x86-64

py_chunks-0.6.1-cp39-abi3-manylinux_2_28_aarch64.whl (5.1 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.28+ ARM64

py_chunks-0.6.1-cp39-abi3-macosx_11_0_arm64.whl (5.0 MB view details)

Uploaded CPython 3.9+macOS 11.0+ ARM64

py_chunks-0.6.1-cp39-abi3-macosx_10_12_x86_64.whl (5.2 MB view details)

Uploaded CPython 3.9+macOS 10.12+ x86-64

File details

Details for the file py_chunks-0.6.1.tar.gz.

File metadata

  • Download URL: py_chunks-0.6.1.tar.gz
  • Upload date:
  • Size: 547.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for py_chunks-0.6.1.tar.gz
Algorithm Hash digest
SHA256 9391d70ebb90ec15c7b6d8128cdc95d766eae3d6fbcf6fe9e32cbbde04b02483
MD5 afebfcbe852d4e8f380cf3fc311061cc
BLAKE2b-256 c8dc444be3ad248dfb649d3306e916cc29e446b7a21104e1130524ed7507dbcf

See more details on using hashes here.

Provenance

The following attestation bundles were made for py_chunks-0.6.1.tar.gz:

Publisher: release.yml on RanjanKudesia/py-chunks

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file py_chunks-0.6.1-cp39-abi3-win_amd64.whl.

File metadata

  • Download URL: py_chunks-0.6.1-cp39-abi3-win_amd64.whl
  • Upload date:
  • Size: 5.2 MB
  • Tags: CPython 3.9+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for py_chunks-0.6.1-cp39-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 da1efba0be00c436b82934fbc5407423d74c61537a1f21436e6d87bad1484a62
MD5 ac74f633dac27415422f420d5f13e5ba
BLAKE2b-256 ef5c9c41b41d26bd9400944672d51b97a4455efad7fc143fe92206b7a0928fe1

See more details on using hashes here.

Provenance

The following attestation bundles were made for py_chunks-0.6.1-cp39-abi3-win_amd64.whl:

Publisher: release.yml on RanjanKudesia/py-chunks

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file py_chunks-0.6.1-cp39-abi3-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for py_chunks-0.6.1-cp39-abi3-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 a8e28eb4334e5ca9b64638b5646cd92ca701cbaa2519a0fcb086c1d28a9ea33e
MD5 97f9fd22b0790b8e10993f6b670031f2
BLAKE2b-256 93f39a65c7d193a4e699580543f342e6c7862509b6484eec911584082d58aff2

See more details on using hashes here.

Provenance

The following attestation bundles were made for py_chunks-0.6.1-cp39-abi3-manylinux_2_28_x86_64.whl:

Publisher: release.yml on RanjanKudesia/py-chunks

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file py_chunks-0.6.1-cp39-abi3-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for py_chunks-0.6.1-cp39-abi3-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 afa3fef4ef583a6d8ce15c23a6e1f1faed712ebe71e9e84345f8b601b4eb301b
MD5 882238e4cadaa6147f1d32c8774382ef
BLAKE2b-256 9a1867ef862cb5d16de82c92eca3cf2bb04ed55410546b134055a6d2dee99105

See more details on using hashes here.

Provenance

The following attestation bundles were made for py_chunks-0.6.1-cp39-abi3-manylinux_2_28_aarch64.whl:

Publisher: release.yml on RanjanKudesia/py-chunks

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file py_chunks-0.6.1-cp39-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for py_chunks-0.6.1-cp39-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 71e875a9bfd7e6cc382f207dc33ce570da395753e6f108044e37a0688ac3d18a
MD5 b62b6f2a1516a28340181f41f493009a
BLAKE2b-256 56932725a930ebd824cb751f14aae538d540f8a94a1b65d74e398d6832f6a3c7

See more details on using hashes here.

Provenance

The following attestation bundles were made for py_chunks-0.6.1-cp39-abi3-macosx_11_0_arm64.whl:

Publisher: release.yml on RanjanKudesia/py-chunks

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file py_chunks-0.6.1-cp39-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for py_chunks-0.6.1-cp39-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 ecbbb82aaf7a7513aeb6286349b2c4421730258ec34228874aa9beb733206e15
MD5 f785041477387c4b4530e6336b28ab57
BLAKE2b-256 2fa37cd4b39e012808c90e2e888a5fcc7cf96381960720de234a4e3178d7f497

See more details on using hashes here.

Provenance

The following attestation bundles were made for py_chunks-0.6.1-cp39-abi3-macosx_10_12_x86_64.whl:

Publisher: release.yml on RanjanKudesia/py-chunks

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.6.4

6 files

0.6.3

6 files

0.6.2

6 files

This release

0.6.1 This release

6 files

0.6.0

5 files

0.5.0

5 files

0.4.7

5 files

0.4.6

5 files

0.4.5

5 files

0.4.4

5 files

0.4.3

5 files

0.4.2

5 files

0.4.1

5 files

0.4.0

5 files

0.3.1

5 files

0.3.0

5 files

0.2.4

5 files

0.2.3

5 files

0.2.2

5 files

0.2.1

5 files

0.1.6

3 files

0.1.5

3 files

0.1.0

3 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page