Skip to main content

Calculate multiple hash digests for a piece of data in parallel, one algo/thread.

Project description

MtHasher

Calculate multiple hash digests for a piece of data in parallel, one algo/thread.

Based on the code of Peter Wu peter@lekensteyn.nl (https://git.lekensteyn.nl/scripts/tree/digest.py).

Usage

From CLI

Add data over STDIN and/or as arguments and select the desired algorithms:

cat data.txt | python3 -m mthasher -i data2.txt - --sha1 --sha256 -o checksums.txt

At least one algorithm is mandatory and by default the script reads from STDIN and writes to STDOUT.

From Python

The exposed API is the following

  • ALGORITHMS_GUARANTEED: The tuple of the supported algorithms
  • Hasher(): Single-threaded hasher, takes an iterable (e.g. list of algorithms to use)
  • MtHasher() Multi-threaded hasher, takes an iterable (e.g. list of algorithms to use)

Both hashers expose the following API

  • header: tuple of header elements ("filename" and the list of algorithms in the supplied order)
  • algos: tuple of supplied algorithms
  • hash_file(): Takes a filename or a file-like object on bytes, returns the digest tuple in same order as header (the filename is omited)
  • hash_multiple_files():Takes an iterable of filenames or file-like objects on bytes,returns the generator of filename + digest tuples in same order as header, one for every input object

Example

from io import BytesIO

from mthasher import MtHasher

hasher = MtHasher(('sha1', 'md5'))
filename_header, sha1_header, md5_header = hasher.header
sha1_digest, md5_digest = hasher.hash_file('data.txt')
for filename, sha1_digest, md5_digest in hasher.hash_multiple_files(('data.txt', open('data2.txt', 'rb'), '-', BytesIO('bytesstring'))):
    # First the header and then the digests
    print(filename, sha1_digest, md5_digest, sep='\t')

Supported algorithms

  • md5
  • sha1
  • sha224
  • sha256
  • sha384
  • sha512
  • sha3_224
  • sha3_256
  • sha3_384
  • sha3_512
  • blake2b
  • blake2s

License

Licensed under the MIT license http://opensource.org/licenses/MIT

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

mthasher-2.0.4.tar.gz (5.3 kB view details)

Uploaded Source

Built Distribution

mthasher-2.0.4-py3-none-any.whl (6.4 kB view details)

Uploaded Python 3

File details

Details for the file mthasher-2.0.4.tar.gz.

File metadata

  • Download URL: mthasher-2.0.4.tar.gz
  • Upload date:
  • Size: 5.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.1.12 CPython/3.8.3 Linux/5.11.0-1028-gcp

File hashes

Hashes for mthasher-2.0.4.tar.gz
Algorithm Hash digest
SHA256 7ec05afca15e12ccca73238caf44e7aa52d2cae8a2547efe9b3657d0980ada64
MD5 7fd642d35dafdbd3ca10c3201f364267
BLAKE2b-256 7e942ab9c10b2260e8199cf8d9d1051ac14dc81bb22d8713b79ae26c4d6838c4

See more details on using hashes here.

File details

Details for the file mthasher-2.0.4-py3-none-any.whl.

File metadata

  • Download URL: mthasher-2.0.4-py3-none-any.whl
  • Upload date:
  • Size: 6.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.1.12 CPython/3.8.3 Linux/5.11.0-1028-gcp

File hashes

Hashes for mthasher-2.0.4-py3-none-any.whl
Algorithm Hash digest
SHA256 8c37d16755a7f01b45bb08c87bb6f8843235e82ea9611c2abbca0c17de04c693
MD5 bfc3840ad0414c88093051ae0429bfa4
BLAKE2b-256 2d862503c9ab5f572faa175541fb153261debc240101401fc1250186b3e9ba4e

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page