GzipSimilarity

GzipSimilarity provides a dependency-light Swarmauri similarity implementation based on gzip compression and Normalized Compression Distance (NCD).

Features

• Implements SimilarityBase from swarmauri_base.
• Compares text and binary values through shared gzip compressibility.
• Supports deterministic gzip sizing and configurable compression levels.
• Supports symmetric concatenation-order handling.
• Exposes both bounded similarity and raw NCD values.

Installation

uv add swarmauri_similarity_gzip

pip install swarmauri_similarity_gzip

Usage

from swarmauri_similarity_gzip import GzipSimilarity

similarity = GzipSimilarity()

score = similarity.similarity(
    "The quick brown fox",
    "Le renard brun rapide",
)

ncd = similarity.ncd(
    "The quick brown fox",
    "Le renard brun rapide",
)

similarity() returns a score in [0.0, 1.0], where higher values indicate greater shared compressibility. ncd() exposes the underlying Normalized Compression Distance. The result is a compression-based similarity estimate and should not be interpreted as semantic similarity without corpus-specific validation.

For language comparison, use balanced samples, consistent UTF-8 encoding and preprocessing, and sufficiently large inputs to reduce gzip framing effects.

The implementation is based on Benedetto, Caglioti, and Loreto, Language Trees and Zipping.

License

Apache-2.0