Skip to main content

Library to simplify process of gathering identifiable attributes about files.

Project description

atklite

atklite is a library intended to help easily determine identifiable attributes about files under investigation. With a simple usage syntax, the following information can be determined by calling applications:

  • MD5 hash
  • SHA-1 hash
  • SHA-256 hash
  • CRC32 checksum
  • File size
  • File type and MIME type detection
  • Fuzzy hash (CTPH) via ssdeep
  • First byte values of file
  • Cymru Malware Hash Registry (MHR) status

atklite is primarily intended for use in analyzing malware samples but may be useful for anyone looking for a simple API or command line tool for triaging files.

Additionally, atklite can be used as a binary storage system, utilizing a standard filesystem to store the files in a sharded manner using the first N bytes of the hash that a user chooses to use (MD5, SHA-1, SHA-256 or SHA-512).

Setup

Requirements

The following modules are installed automatically:

Installing prerequisites

Installing the prerequisites on an Debian/Ubuntu system is simple.

First install the prerequisites for python-ssdeep and python-magic:

apt install ssdeep python3-dev libmagic1 libfuzzy-dev libfuzzy2

Installation

atklite is available on PyPI. Install with pip:

python3 -m pip install atklite

If you just want to run the installed command line utility (atk-info), try out pipx:

pipx install atklite

Usage overview

Usage (CLI):

$ atk-info /usr/bin/dash
using binary store at: /home/jdoe/binary_store
-- dash ------------------------------------------------------------------------
  Analysis time: 2023-08-23 07:28:05.016638
  File name:     dash
  File size:     125688
  File type:     ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=f7ab02fc1b8ff61b41647c1e16ec9d95ba5de9f0, for GNU/Linux 3.2.0, stripped
  MIME type:     application/x-pie-executable
  CRC-32:        dea50977
  MD5 hash:      7409ae3f7b10e059ee70d9079c94b097
  SHA1 hash:     42e94914c7800c7063c51d7a17aec3a2069a3769
  SHA256 hash:   4f291296e89b784cd35479fca606f228126e3641f5bcaee68dee36583d7c9483
  Fuzzy hash:    3072:BW795HHUunYzyVSlYV+tqOsDRC1wAtXqW6mfDrEDImbr:BWjHHZ2dtq3RFGqWzbrED7br
  First bytes:   7f45 4c46 0201 0100 0000 0000 0000 0000  .ELF............
  Stored file:   /home/jdoe/binary_store/7/4/0/7409ae3f7b10e059ee70d9079c94b097
  Cymru MHR:     Unknown

Python API (processing data input):

>>> from json import dumps as json_dumps
>>> from atklite import FileAnalysis
>>> with open("/usr/bin/dash", "rb") as f:
...     data = f.read(100)
... 
>>> analysis = FileAnalysis(data=data)
>>> print(json_dumps(analysis.return_analysis(), indent=4))
{
    "time": 1692775973.2529104,
    "isotime": "2023-08-23 07:32:53.252912",
    "md5": "2b17c2d5693b2257c7a1e09f00e5e2aa",
    "sha1": "ff1070796c5d75a284415a6269a42c89b0b385ac",
    "sha256": "13feaf307e36054cf7537965d8343714112a25674a13bc5f591ecb1fb61de65b",
    "sha512": "e658d0487a72e91f2b00099ad2eebecd2c5c7f5963f1adf861b76dc088b569199f43c866dbb1de7af2ad58b312e205c2e3ef3891e51229508059e034176284ab",
    "size": 100,
    "ftype": "ELF 64-bit LSB shared object, x86-64, version 1 (SYSV)",
    "mime_type": "application/x-sharedlib",
    "ssdeep": "3:Bnks//ZlllVrX/1llp/leulZpXltllvllvll/n:BnX//ZtBP/leulZ5",
    "crc32": "0c29b48e",
    "first_bytes": "7f45 4c46 0201 0100 0000 0000 0000 0000 03  .ELF.............",
    "mhr_result": "Unknown"
}

A file known to MHR:

$ atk-info -n 7ee6095ba8c4ed9fe11fbf5e703823e1aeae7f5443027738f55979b27ca57171.dll 
using binary store at: /home/jdoe/binary_store
-- 7ee6095ba8c4ed9fe11fbf5e703823e1aeae7f5443027738f55979b27ca57171.dll --------
  Analysis time: 2023-08-21 02:03:14.620973
  File name:     7ee6095ba8c4ed9fe11fbf5e703823e1aeae7f5443027738f55979b27ca57171.dll
  File size:     135168
  File type:     PE32 executable (DLL) (GUI) Intel 80386, for MS Windows
  MIME type:     application/x-dosexec
  CRC-32:        013a7794
  MD5 hash:      81e56fd3b67ce33ef7150003985be7f4
  SHA1 hash:     6c739fcc6dea8cc65617ff184f1febcd5404143a
  SHA256 hash:   7ee6095ba8c4ed9fe11fbf5e703823e1aeae7f5443027738f55979b27ca57171
  Fuzzy hash:    3072:4ELogSZScYg+E/wmqpFQQT7J/AzMVWWRTBfItV74VZ:LofScb/wmqp+QPJ4zMVWWRTBgkj
  First bytes:   4d5a 9000 0300 0000 0400 0000 ffff 0000  MZ..............
  Cymru MHR:     2023-08-20T00:40:33 11 engines (37%)

The MHR lookup is performed as a TXT record, allowing us to capture the results timestamp and number of engines detecting the file as malicious. atklite estimates a percentage of supported engines (in the above sample, 11/30, resulting in a 37% detection rate).

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

atklite-1.1.5.tar.gz (8.3 kB view details)

Uploaded Source

Built Distribution

atklite-1.1.5-py3-none-any.whl (8.7 kB view details)

Uploaded Python 3

File details

Details for the file atklite-1.1.5.tar.gz.

File metadata

  • Download URL: atklite-1.1.5.tar.gz
  • Upload date:
  • Size: 8.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.10.12

File hashes

Hashes for atklite-1.1.5.tar.gz
Algorithm Hash digest
SHA256 3c77c3d47c1ff6fc4d3ee52b84051bf3d502c1e10001bf3c8cc393c210c27241
MD5 456dc6027cb7e42e7c1c1f7cca28688d
BLAKE2b-256 75fe311a47db462fb353dcf05b2a6bd20c3a81dc34bb2f46f6258783e21fb367

See more details on using hashes here.

File details

Details for the file atklite-1.1.5-py3-none-any.whl.

File metadata

  • Download URL: atklite-1.1.5-py3-none-any.whl
  • Upload date:
  • Size: 8.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.10.12

File hashes

Hashes for atklite-1.1.5-py3-none-any.whl
Algorithm Hash digest
SHA256 ba0cf4b2d3722b62ad1b0be24ffe20aa942f3b36f5726c9ae926061d5c7240f4
MD5 4ae835dd6ca74fd113d0072a645059d3
BLAKE2b-256 8b1d8580a6377a91bdf36b071d9f2d7a627d97634e9b117657ef8a2a19148c97

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page