Skip to main content

A much faster du that tells you why your quota is full.

Project description

rapiDU

A much faster du that tells you why your quota is full.

CI Python 3.6+ No dependencies MIT License

rapiDU walking a project tree and explaining that it occupies 266.8 MiB to hold 75.5 MiB of data, ranking the same tree by file count, printing a quota table with the age of its snapshot, finding 512 MiB held by a deleted-but-open file descriptor, and catching a freshly written GPFS tree that loses 224 MiB while it settles.

Install

pip install rapidu

Use

rdu                      # this directory: how big, and what is big inside it
rdu /project/mylab       # any other path
rdu ~/scratch -n 20      # list 20 entries instead of 10

rdu -i                   # rank by file count -- what an inode quota limits
rdu -c                   # count files only, no stat: ~8x again on GPFS
rdu -Q                   # the quota table, and the age of its figures
rdu -D                   # space held by files deleted while still open
rdu -a                   # the full audit: quota + /proc scan + reconciliation

Faster, and the same number

Cold GPFS walk: du takes 168.1s against rapiDU's 25.4s on a 792,225-file package cache (6.6x), and 298.5s against 57.4s on a 1,686,589-file project directory (5.2x).

Same total as du, to the byte. That is checked on every commit.

Reading the table

╭───────────────────────────────────────────────────────────────────────────────────╮
│ /project/lab/shared                                                               │
│ 1.4 TiB  ·  5,435 files  ·  4.12s                                                 │
│                                                                                   │
│   ──────────────────────────────────────────────────────                          │
│         size  share                           files  entry                        │
│    661.5 GiB  █████▏░░░░░░░░░░░░   31.9%        350  checkpoints/                 │
│    343.8 GiB  ██▊░░░░░░░░░░░░░░░   16.6%        968  datasets/                    │
│    470.9 GiB  ▒▒▒▒▒▒▒▒░░░░░░░░░░   22.9%      4,117  (84 more — use -n 0 for all) │
╰───────────────────────────────────────────────────────────────────────────────────╯

The frame carries no title and it always closes: a line too wide for it wraps inside, at a path separator, rather than running past the border. Its colour is a gradient sweeping from the top-left corner, and it degrades — 24-bit if your terminal advertises COLORTERM=truecolor, a 12-step ramp at 256 colours, and two bright cyan-to-blue tones at 8. --ascii turns it into +--+; --no-box removes it, which is what you want when piping into grep or a diff.

The path leads, because it is what the report is about; the size is the first of the four numbers describing it. share labels the bar and the percentage beside it, which are one measurement in two forms -- the picture and the number. The last column is entry, not path, because it holds plain files as well as directories and what is printed is a name relative to the root.

The hatched last row is everything not listed, and it names how many that is -- so a truncated table always says it is truncated. The bar is share of the whole walk, so it always agrees with the number beside it. The column you sorted by is the one in colour — under -i, files takes the tone and size steps back. Sizes are cumulative, so any row agrees with du -s on that path.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

rapidu-0.2.2.tar.gz (141.0 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

rapidu-0.2.2-py3-none-any.whl (89.9 kB view details)

Uploaded Python 3

File details

Details for the file rapidu-0.2.2.tar.gz.

File metadata

  • Download URL: rapidu-0.2.2.tar.gz
  • Upload date:
  • Size: 141.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for rapidu-0.2.2.tar.gz
Algorithm Hash digest
SHA256 c9c260c6240aa83a9bfdde7a9f983fa7237587f3b3b6a4dc02d153bc7a8b468a
MD5 04ad7870801284dc77466b9ae0a1daa8
BLAKE2b-256 f8536a6fee1165f9f47729664bc0e28b37d56dad073af5098f66a94be424dd4f

See more details on using hashes here.

Provenance

The following attestation bundles were made for rapidu-0.2.2.tar.gz:

Publisher: release.yml on PursuitOfDataScience/rapidu

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rapidu-0.2.2-py3-none-any.whl.

File metadata

  • Download URL: rapidu-0.2.2-py3-none-any.whl
  • Upload date:
  • Size: 89.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for rapidu-0.2.2-py3-none-any.whl
Algorithm Hash digest
SHA256 59fb521db002f29a9bef5e5ecc39fab06a5ba0487ada7e11152d87d7f170b6f3
MD5 ca14edec15c87db4d3f9909b967ba0a1
BLAKE2b-256 d22b7a87c830e4f9840660d04e95552ac78b71a386074c93eb3da02b45500b53

See more details on using hashes here.

Provenance

The following attestation bundles were made for rapidu-0.2.2-py3-none-any.whl:

Publisher: release.yml on PursuitOfDataScience/rapidu

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page