hayazip 0.2.0

Blazing fast, multi-threaded SIMD ZIP library

Hayazip

日本語 | English

🚀 Blazing Fast, Multi-Threaded SIMD ZIP Library for Rust & Python

hayazip is an ultra-fast ZIP archive library designed from the ground up to leverage modern hardware capabilities. It combines memory-mapped I/O, SIMD-accelerated compression and decompression (via libdeflater), and thread-pool-based parallelism (via rayon) to accelerate both ZIP extraction and ZIP creation.

Features

• Zero-Copy Parsers: Uses memmap2 to map the ZIP file directly into memory, skipping expensive kernel-to-user-space copies.
• SIMD Optimized Compression and Decompression: Powered by libdeflater to leverage AVX2, AVX-512, or NEON depending on the architecture.
• Multi-threaded ZIP Creation and Extraction: Uses rayon to process independent files in parallel.
• Hardware-accelerated CRC32: Validates integrity using hardware instructions through crc32fast.
• Low-footprint Archive Writing: Spools compressed members to temporary files instead of holding the full archive in memory.
• Cross-platform Python Bindings: Built with PyO3 for easy, out-of-the-box integration in any Python environment.

Python Quick Start

Installation

You can install hayazip directly from PyPI with uv or pip. Prebuilt abi3 wheels are published for CPython 3.8+ on Linux, macOS, and Windows, and a source distribution is published as a fallback:

uv add hayazip
# or
pip install hayazip

Usage

Creating and extracting archives in Python is straightforward:

import hayazip

source_dir = "project_files"
archive_path = "project_files.zip"
output_dir = "extracted_files"

hayazip.create_zip(source_dir, archive_path)
hayazip.extract_zip(archive_path, output_dir)
print("Done!")

Rust Quick Start

Add hayazip to your Cargo.toml:

[dependencies]
hayazip = "0.2.0"

Usage

use hayazip::{create_zip, extract};

fn main() {
    let source_dir = "project_files";
    let archive_path = "project_files.zip";
    let output_dir = "extracted_files";

    create_zip(source_dir, archive_path).expect("Archive creation failed");

    if let Err(e) = extract(archive_path, output_dir) {
        eprintln!("Extraction failed: {}", e);
    } else {
        println!("Extraction successful!");
    }
}

Benchmarks

On modern CPUs, hayazip uses libdeflater for SIMD-accelerated DEFLATE and rayon for parallel file processing. Archive creation writes members with bounded worker parallelism and a temporary spool to keep memory usage predictable while still saturating multiple cores.

Build from Source (Python)

To compile from source and install into your local Python environment:

pip install maturin
maturin develop --release

License

MIT