Pure Python module providing a variable-length, content-based blocking algorithm
Project description
Chop a file into variable-length, content-based chunks.
Example use: .. code-block:: python
>>> import rolling_checksum_mod >>> with open('/tmp/big-file.bin', 'rb') as file_: >>> for chunk in rolling_checksum_mod.min_max_chunker(file_): >>> # chunk is now a piece of the data from file_, and it will not always have the same length. >>> # Instead, it has the property that if you insert a byte at the beginning of /tmp/big-file.bin, >>> # most of the chunks of the file will remain the same. This can be nice for a deduplicating >>> # backup program. >>> print(len(chunk))
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Close
Hashes for rolling_checksum_py_mod-1.0.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | c9ae51f4824d46088f02e60869ca3d67f969e1dc298b67d748b6cb1573f25f86 |
|
MD5 | 398c232dddd68aae959f219eebfcfe39 |
|
BLAKE2b-256 | 5db78578032ec19cf4e10636d0e4f40e3217242e1e962880ba0e06fed539b12c |