Unichunking
Extract LLM-friendly chunks from any file type.
Supported file types are :
- DOCX & DOCX-like (DOC, ODT)
- PPTX & PPTX-like (PPT, ODP)
- XLSX & XLSX-like (XLS, ODS)
- TXT, MD, CSV
- IPYNB
Installation
To install, run the following command:
python3 -m pip install delos-unichunking
How to use
The main functions are :
extract_subchunksreturns a list of all the text particles in the file.split_chunks_with_overlaptransforms a list of subchunks on a given page into a list of chunks following default or specified parameters for minimum/maximum token size and overlap.build_chunked_pagesreturns a list of "pages", which are lists of formated chunks, following the structure of the document.compute_pagesapproximates the pagination of a file that does not have a native pagination system (such as DOCX) by comparing it to a PDF version.
Specificities
Please note that the package requires a LibreOffice installation to run soffice commands, used during file conversions : for instance, DOC/ODT are first converted to DOCX format and processed as such.
The page numbers computed for DOCX files are an approximation and can be off by a few pages for large files.
Artifical page numbers are used for page-less structures such as TXT files, or to split large XLSX sheets into multiple pages, to follow a tokens-per-page limit.
2 maintainers
Metadata
2 maintainers
Release files for delos-unichunking 1.9.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| delos_unichunking-1.9.1.tar.gz | 191.4 kB | Without provenance | Details for delos_unichunking-1.9.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| delos_unichunking-1.9.1-py3-none-any.whl | Python 3 | none | any | Details for delos_unichunking-1.9.1-py3-none-any.whl |
Total release size: 241.9 kB
Release files / delos_unichunking-1.9.1.tar.gz
| Download URL | delos_unichunking-1.9.1.tar.gz |
|---|---|
| Size | 191.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
fb1f7796fd6b2f7754a213634e07c70f0d05581ce4c4c02983a62c29464480ef
|
|
BLAKE2b-256 checksum How to use checksums |
7f47a41c466acb1823a61cc070a7cff37fe3a74e5c7a8595592c7d6e30a5e0f1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.13.6 {"installer":{"name":"uv","version":"0.13.6","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / delos_unichunking-1.9.1-py3-none-any.whl
| Download URL | delos_unichunking-1.9.1-py3-none-any.whl |
|---|---|
| Size | 50.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
73962c6ee46c8ad9ef9a993ac0bfa036cee9265d7258f823cc5f6f05a04c3856
|
|
BLAKE2b-256 checksum How to use checksums |
c6e6c0e49dc492cb605cedaead29d9dcb6cedaa8c8b38c0a91b11710afa11067
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.13.6 {"installer":{"name":"uv","version":"0.13.6","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Security for delos-unichunking 1.9.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
