datacortex

Python bindings for DataCortex, a lossless JSON/NDJSON compressor that beats zstd-19 and brotli-11 on every file tested.

Built with Rust via PyO3. Native performance, Python convenience.

Install

pip install datacortex

Requires Python 3.8+. Pre-built wheels available for macOS (ARM).

Quick Start

import datacortex

# Compress JSON bytes
with open("logs.ndjson", "rb") as f:
    data = f.read()

compressed = datacortex.compress(data)
print(f"Ratio: {len(data) / len(compressed):.1f}x")

# Turbo mode: ~30x faster encode, ~2% ratio tradeoff
fast = datacortex.compress(data, turbo=True)

# Decompress (byte-exact)
original = datacortex.decompress(compressed)
assert original == data

API Reference

compress(data, mode="fast", format="auto", level=None, turbo=False)

Compress bytes. Returns compressed bytes in .dcx format.

Args:

• data (bytes): Input data (JSON, NDJSON, or generic text)
• mode (str): "fast" (default), "balanced", or "max"
• format (str): "auto" (default), "json", "ndjson", "generic"
• level (int, optional): zstd level override (fast mode only)
• turbo (bool): Use turbo mode for ~30x faster encode (fast mode only)

Returns: bytes

decompress(data)

Decompress .dcx bytes. Returns the original data, byte-exact.

Args:

• data (bytes): Compressed .dcx data

Returns: bytes

compress_file(input_path, output_path, mode="fast", level=None, turbo=False)

Compress a file to .dcx format.

Args:

• input_path (str): Path to the input file
• output_path (str): Path for the compressed output
• mode (str): "fast", "balanced", or "max"
• level (int, optional): zstd level override (fast mode only)
• turbo (bool): Use turbo mode for ~30x faster encode (fast mode only)

decompress_file(input_path, output_path)

Decompress a .dcx file back to the original.

Args:

• input_path (str): Path to the .dcx file
• output_path (str): Path for the decompressed output

detect_format(data)

Detect the format of input data.

Args:

• data (bytes): Input data to analyze

Returns: str -- "ndjson", "json", "json_array", or "generic"

info(data)

Inspect compressed .dcx data.

Args:

• data (bytes): Compressed .dcx data

Returns: dict with keys: mode, format, original_size, compressed_size, crc32, entropy_coder

Compression Modes

Benchmarks (v0.6.0)

CLI

For command-line usage, install the Rust CLI:

cargo install datacortex-cli

Links

• GitHub
• CLI (crates.io)
• Site

License

MIT