ZipText (ZTAR) — High-Efficiency Textual & Binary Reversible Container
Zip Text Adaptive Representation (ZTAR) is a storage-independent framework designed to represent files as compact, reversible textual containers (.ztxt) or high-speed binary containers (.zbin). It prioritizes lossless exact reconstruction ($\operatorname{Decode}(\operatorname{Encode}(X)) = X$), adaptive multi-codec optimization, bounded-memory chunk streaming, and cryptographic chunk integrity.
1. Key Highlights & Priorities
- Exact Lossless Reconstruction: Strict invariant guaranteed by whole-file and per-chunk SHA-256 and CRC32 digests.
- Dual Container Profiles:
- Textual Profile (
.ztxt): Base85 armored ASCII container with plain-text headers. Ideal for systems restricted to text-only transmission (APIs, JSON envelopes, message queues). - Binary Profile (
.zbin): High-speed binary container with zero text expansion penalty. Ideal for dense media (JPEG, PNG, MP4, WebP).
- Textual Profile (
- Adaptive Codec Selection: Dynamically benchmarks candidate compression algorithms chunk-by-chunk:
brotli(Google Brotli high-density compression)zstd(Facebook Zstandard high-throughput compression)lzma(LZMA2 / XZ maximum ratio)zlib(Deflate standard)bz2(Burrows-Wheeler bzip2)lz4(High-speed real-time compression)raw(Identity pass-through for incompressible high-entropy data)
- Bounded Working Memory: Decodes in user-configurable chunk windows (default 1 MiB) without buffering multi-gigabyte files into RAM.
- ZTAR-Bench Standardization: Rigorously benchmarked across the 11 standardized Canterbury Corpus files and synthetic corpora with fact-checked transparency.
- Interactive Documentation Web App: High-tech interactive React documentation app ready for instant deployment to Cloudflare Pages.
2. Global Installation
Install globally via PyPI to access the ziptext CLI from any terminal:
# Standard installation
pip install ziptext
# With all high-speed optional compression engines (Brotli, Zstandard, LZ4)
pip install "ziptext[all]"
For local development or editable installation:
git clone https://github.com/itsjustayush/ZipText-ZTAR-.git
cd ZipText-ZTAR-
pip install -e .[all]
Verify global availability:
ziptext --version
# Output: ziptext 0.2.0
3. Global CLI Reference
Encode Files
# Encode into human-readable text container (.ztxt)
ziptext encode document.json document.ztxt
# Encode into high-speed raw binary container (.zbin)
ziptext encode photo.jpg photo.zbin --profile binary
# Explicitly choose compression codec
ziptext encode dataset.csv dataset.ztxt --codec brotli --chunk-size 1048576
Decode & Recover Files
# Decode container into original file (auto-preserves filename and extension)
ziptext decode document.ztxt
# Decode to custom destination path
ziptext decode document.ztxt recovered_document.json
Verification & Inspection
# Verify CRC32 and SHA-256 integrity without writing to disk
ziptext verify document.ztxt
# Inspect container manifest metadata
ziptext info document.ztxt
# Analyze file type and compression viability
ziptext classify dataset.tar.gz
Corpus Management & ZTAR-Bench
# Download Canterbury Corpus automatically
ziptext corpora download --corpus canterbury --dest fixtures/canterbury
# Run full 4-tier evaluation across Canterbury Corpus
ziptext ztar-bench fixtures/canterbury --output-dir results/canterbury_eval
4. Python SDK API
from ziptext import ZipTextEngine, ContainerWriter, ContainerReader
# 1. High-level one-line roundtrip
engine = ZipTextEngine(chunk_size_bytes=1048576, default_codec="auto")
# Encode
res = engine.encode_file("report.pdf", "report.ztxt", profile="text")
print(f"Encoded {res.original_size}B -> {res.container_size}B using {res.codec_used}")
# Decode with byte-for-byte SHA256 check
dec = engine.decode_file("report.ztxt", "recovered_report.pdf")
assert dec.exact_match is True
# 2. Low-memory streaming writer
with open("large_stream.ztxt", "w", encoding="utf-8") as f:
writer = ContainerWriter(f, original_filename="archive.tar", profile="text")
writer.write_chunk(b"chunk data...", codec="zstd")
writer.finalize(original_size=10485760, whole_file_hash=sha256_hash)
5. Canterbury Corpus Benchmark Results (v0.2.0)
| File Name | Category | Rep A (Raw) | Rep B (Comp) | Best Codec | Rep D (.ztxt) | Rep D (.zbin) | Net Storage Reduction (.ztxt) | Net Storage Reduction (.zbin) | Lossless Pass |
|---|---|---|---|---|---|---|---|---|---|
| alice29.txt | Literature (English) | 152,089 B | 43,202 B | bz2 |
56,052 B | 43,604 B | -63.15% | -71.33% | 100% Exact |
| asyoulik.txt | Play (Shakespeare) | 125,179 B | 39,569 B | bz2 |
51,392 B | 39,972 B | -58.95% | -68.07% | 100% Exact |
| cp.html | HTML Document | 24,603 B | 7,609 B | brotli |
10,478 B | 8,011 B | -57.41% | -67.44% | 100% Exact |
| fields.c | C Source Code | 11,150 B | 3,012 B | brotli |
4,426 B | 3,414 B | -60.30% | -69.38% | 100% Exact |
| grammar.lsp | LISP Source Code | 3,721 B | 1,215 B | brotli |
1,988 B | 1,617 B | -46.57% | -56.54% | 100% Exact |
| kennedy.xls | Spreadsheet Binary | 1,029,744 B | 130,248 B | brotli |
168,676 B | 130,650 B | -83.62% | -87.31% | 100% Exact |
| lcet10.txt | Technical Paper | 426,754 B | 107,641 B | bz2 |
139,588 B | 108,043 B | -67.29% | -74.68% | 100% Exact |
| plrabn12.txt | Poetry (Milton) | 481,861 B | 145,548 B | bz2 |
188,824 B | 145,950 B | -60.81% | -69.71% | 100% Exact |
| ptt5 | CCITT Fax Binary | 513,216 B | 49,759 B | brotli |
65,112 B | 50,161 B | -87.31% | -90.23% | 100% Exact |
| sum | SPARC Executable | 38,240 B | 9,811 B | brotli |
13,386 B | 10,213 B | -64.99% | -73.29% | 100% Exact |
| xargs.1 | UNIX Man Page | 4,227 B | 1,588 B | brotli |
2,526 B | 1,990 B | -40.24% | -52.92% | 100% Exact |
6. Cloudflare Pages Web Documentation
The repository includes a modern, high-tech interactive documentation website located in web/ built with Vite, React, GSAP, and React Bits components (<GradientWaves />, <LetterGlitch />, <TechText />, <MagicBento />, <SpotlightCard />, and <AnimatedContent />).
Deploying to Cloudflare Pages
When connecting this repository to Cloudflare Pages:
- Build command:
npm run build - Build output directory:
web/dist - Root directory:
/(orwebif configuring subfolder directly)
Local Web Preview
npm run build
npm run preview
7. Incident Register & Engineering Forensics
We maintain an immutable log of anomalies and fixes in errors/ (ERR-001 through ERR-012), each verified by automated unit tests in tests/.
Run the automated test suite:
pytest -v
# 39 passed in 13.25s
8. License
MIT License. Designed and engineered for research and production by the ZipText Research Team.
Metadata
Release files for ziptext 0.3.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| ziptext-0.3.0.tar.gz | 47.8 kB | Without provenance | Details for ziptext-0.3.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| ziptext-0.3.0-py3-none-any.whl | Python 3 | none | any | Details for ziptext-0.3.0-py3-none-any.whl |
Total release size: 92.2 kB
Release files / ziptext-0.3.0.tar.gz
| Download URL | ziptext-0.3.0.tar.gz |
|---|---|
| Size | 47.8 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
d069aef0c648bc8d4b8265285aeefda724cb8df46ee66741608d01aca0002911
|
|
BLAKE2b-256 checksum How to use checksums |
55ca47a6d76e375970343e3faf46a1a086e33b84dce0e33fa677798659ec71af
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.15
|
Release files / ziptext-0.3.0-py3-none-any.whl
| Download URL | ziptext-0.3.0-py3-none-any.whl |
|---|---|
| Size | 44.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
7c7c091d426e409a2990af90610232e985888011b500a2d4cfe56a2377a62f06
|
|
BLAKE2b-256 checksum How to use checksums |
0b8d84d3c61e531df5018e58e822e092f0258c4fce0211c993130cd56a0beb17
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.15
|
Security for ziptext 0.3.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.