Skip to main content

Cython-augmented Python module providing a variable-length, content-based blocking algorithm

Project description

Chop a file into variable-length, content-based chunks.

Example use: .. code-block:: python

>>> import rolling_checksum_mod
>>> # If you have both rolling_checksum_pyx_mod and rolling_checksum_py_mod installed, the software will
>>> # automatically prefer the pyx version.  Both py and pyx versions require rolling_checksum_py_mod, but
>>> # only the pyx version requires rolling_checksum_pyx_mod.
>>> with open('/tmp/big-file.bin', 'rb') as file_:
>>>     for chunk in rolling_checksum_mod.min_max_chunker(file_):
>>>         # chunk is now a piece of the data from file_, and it will not always have the same length.
>>>         # Instead, it has the property that if you insert a byte at the beginning of /tmp/big-file.bin,
>>>         # most of the chunks of the file will remain the same.  This can be nice for a deduplicating
>>>         # backup program.
>>>         print(len(chunk))

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

rolling_checksum_pyx_mod-1.1.3.tar.gz (60.9 kB view details)

Uploaded Source

File details

Details for the file rolling_checksum_pyx_mod-1.1.3.tar.gz.

File metadata

File hashes

Hashes for rolling_checksum_pyx_mod-1.1.3.tar.gz
Algorithm Hash digest
SHA256 476f4e5d786b5f1071d2dc922f66a20713765365b248fe5c79d6daa795391951
MD5 ed96bd311736a13369d353ec7ec175e0
BLAKE2b-256 b6261b5053e194f016d80915fe36e30da5d924866fcef8d298b227c454cdb67d

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page