Skip to main content

chkbit is a lightweight bitrot detection tool.

Project description

chkbit

chkbit is a lightweight bitrot detection tool.

bitrot (a bit flipping in your data) can occur

  • at a low level on the storage media through decay (hdd/sdd)
  • at a high level in the os or firmware through bugs

chkbit is independent of the file system and can help you detect bitrot on you primary system, on backups and in the cloud.

Installation

pip install --user chkbit

Or in its own environment:

pipx install chkbit

Usage

Run chkbit -u PATH to create/update the chkbit index.

chkbit will

  • create a .chkbit index in every subdirectory of the path it was given.
  • update the index with md5 hashes for every file.
  • report bitrot for files that rotted since the last run (check the exit status).
usage: chkbit.py [-h] [-u] [-f] [-q] [-v] PATH [PATH ...]

Checks files for bitrot. See https://github.com/laktak/chkbit-py

positional arguments:
  PATH

optional arguments:
  -h, --help     show this help message and exit
  -u, --update   update indices (without this chkbit will only verify files)
  -f, --force    force update of damaged items
  -q, --quiet    quiet, don't show progress/information
  -v, --verbose  verbose output

Status codes:
  ROT: error, bitrot detected
  EIX: error, index damaged
  old: warning, file replaced by an older version
  add: add to index
  upd: file updated
  ok : check ok
  skp: skipped (see .chkbitignore)
  EXC: internal exception

Repair

chkbit cannot repair bitrot, its job is simply to detect it.

You should

  • backup regularly.
  • run chkbit before each backup.
  • check for bitrot on the backup media.
  • in case of bitrot restore from a checked backup.

Ignore files

Add a .chkbitignore file containing the names of the files/directories you wish to ignore

  • each line should contain exactly one name
  • lines starting with # are skipped

FAQ

Should I run chkbit on my whole drive?

You would typically run it only on content that you keep for a long time (e.g. your pictures, music, videos).

Why is chkbit placing the index in .chkbit files (vs a database)?

The advantage of the .chkbit files is that

  • when you move a directory the index moves with it
  • when you make a backup the index is also backed up

The disadvantage is that you get hidden .chkbit files in your content folders.

How does chkbit work?

chkbit operates on files.

When run for the first time it records a md5 hash of the file contents as well as the file modification time.

When you run it again it first checks the modification time,

  • if the time changed (because you made an edit) it records a new md5 hash.
  • otherwise it will compare the current md5 to the recorded value and report an error if they do not match.

Can I test if chkbit is working correctly?

On Linux/OS X you can try:

Create test and set the modified time:

$ echo foo1 > test; touch -t 201501010000 test
$ chkbit -u .
a ./test
$

a indicates the file was added.

Now update test with a new modified:

$ echo foo2 > test; touch -t 201501010001 test # update test & modified
$ chkbit -u .
u ./test
$

u indicates the file was updated.

Now update test with the same modified to simulate bitrot:

$ echo foo3 > test; touch -t 201501010001 test
$ chkbit -u .
E ./test
chkbit detected bitrot in these files:
./test
error: detected 1 file(s) with bitrot!
$

E indicates an error.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

chkbit-2.0.0.tar.gz (5.7 kB view details)

Uploaded Source

File details

Details for the file chkbit-2.0.0.tar.gz.

File metadata

  • Download URL: chkbit-2.0.0.tar.gz
  • Upload date:
  • Size: 5.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.32.2 CPython/3.7.4

File hashes

Hashes for chkbit-2.0.0.tar.gz
Algorithm Hash digest
SHA256 1b007b9a3e762d5941f9afc79f9ce58ed738fc1ea7f7f41f00a372ca69058a58
MD5 e35990f4821680dea4d8bd70f0f6a905
BLAKE2b-256 f89c3e2b786275b9e0d0d68411e688f4003db31230f130a0df9e0df01c0b641d

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page