fdu (Python)

Fast du replacement, with .gitignore-aware sizes and code and document counts, for the command line, Python, and Rust

Python bindings for fdu, which reports, for every directory in a tree at once, its size, file count, recency, and file kinds, and on request lines of code by language and words by document type. It walks a tree on several threads through each platform’s native directory interface; performance measurements compare it with other tools by build, platform, and tree.

Set Up with Any Coding Agent

Give the agent this instruction:

Run uvx --no-build fdu@latest --install-skill from the project root to install fdu’s self-contained skill for current and future agent sessions.

The skill needs no prior session context or installed command; fdu --skill prints it.

Install the Command Line

uvx --no-build fdu@latest .
uv tool install --no-build fdu

The first command runs fdu once; the second keeps it on PATH. uv tool upgrade --no-build fdu updates that install; uvx --no-build fdu@<version> --help runs one exact release. The repository README has the details. Prebuilt abi3 wheels cover GIL-enabled CPython 3.12 and newer on Linux glibc (x86-64 and arm64), macOS (x86-64 and arm64), and Windows x86-64, so installing needs no Rust toolchain. Free-threaded CPython, such as 3.14t, cannot install them and is not supported; an installer there may fall back to building the source distribution. --no-build makes uv fail instead of compiling when no compatible wheel exists. If uv selects a free-threaded interpreter, pass --python 3.14. An exclude-newer policy in uv can filter a newly published fdu release. Review and allow the first-party fdu package in that policy, or wait for its cool-off to expire.

Use as a Python Library

uv add fdu
pip install fdu

Choose among three entry-point groups over the same Rust engine:

For a standalone final metadata summary, use an explicit summary request:

from pathlib import Path

import fdu

report = fdu.report(Path("."), fdu.Query(views=(fdu.View.SUMMARY,)))

Under the default cache policy, a sole unfiltered metadata summary over the full population without analyzers can use a compact native reduction without a per-file index. Filters, analysis, or snapshot requirements can need retained index state. Query an appropriate existing handle instead of rescanning for its summary.

Index and OpenedIndex both retain inventory in Rust. Index returns after requested preparation and leaves refresh to the caller; status records incomplete coverage. An Index returned by fdu.open(..., stale_ok=True) cannot refresh or start a watch; reopen with stale_ok=False for filesystem verification and subsequent refresh. OpenedIndex allows reads during discovery and can own background observation when requested. Offline jobs can leave observe=False (the default), wait for required coverage, inspect partial/limit outcomes, and consume and release pages. An early partial summary is not a final whole-tree answer; discovery can invalidate continuations. Query-bound Index.watch() is a separate convenience: it copies the index into an independent fixed-query session whose iterator applies updates and drives persistence.

Keep per-file discovery, analysis, reconciliation, observation, queries, and shutdown in Rust. Python should send coarse requests and receive selected answers. Choose detail separately: an Index total can be small, and an opened report can be large and eager. Complete exports support custom Python processing at allocation and copy cost; native queries avoid exporting input rows but still require engine work. Paging adds calls and cursor handling while reducing each response and potentially peak retention if pages are released; total conversion and version invalidation still apply. The surface guide explains lifecycle choices and costs; performance depends on the measured workload.

Lines of code by language and words by document type, from one scan:

from pathlib import Path

import fdu

report = fdu.report(
    Path("/path/to/tree"),
    fdu.Query(views=(fdu.View.CODE, fdu.View.DOCUMENTS)),
)
print(report.render())
print(report.notes)

The two content views run the code and words analyzers they show; print gives the same tables as fdu . --view=code,documents, and report.sections holds them as typed values.

fdu.report also takes a sequence of disjoint paths, reported as one, as fdu docs src does: every size, row, share, and bound is over the union. report.root is then None and report.roots names each root (ReportRoot(label, path)); file rows, status errors, and refused .gitignore files carry root, the position of their root in roots, so report.roots[row.root].path / row.path names the entry; and a tree section has a total and one RootTree per root. A str or Path is always one root. fdu.open and fdu.scan take one root.

The public package is fdu; fdu._native is private build machinery. The supported API includes typed query and scan options, immutable report sections, roll-ups, per-path provenance, cache management, refresh results, and change feeds. .gitignore is read by default: tree, summary, extension, and file rows carry their ignored share, Selection(ignored=...) selects one side, and Status.ignore_rules names any file the limits refused. ScanOptions carries the switch (read_controls) and both limits (control_budget, control_line_limit). Cache status is typed as CacheState, with StaleReason and LeftoverKind, and clear_all_caches returns a ClearSummary. Filesystem failures that prevent an operation from starting remain exceptions. Errors that make a scan partial remain structured data on Status, so callers can use the covered result without losing the reason it is incomplete. The roll-up adapter example shows one scan serving several application-owned summaries without parsing terminal output or adopting fdu’s machine schema as the application’s internal model.

from pathlib import Path

import fdu

index = fdu.open(
    Path("/path/to/tree"),
    cache=fdu.CachePolicy.AUTO,
    scan=fdu.ScanOptions(one_filesystem=True),
    analysis=fdu.AnalysisOptions(analyze=fdu.Analysis.ALL),
)
print(index.status.complete)
print(index.total().by_extension)

report = index.report(
    fdu.Query(
        views=(fdu.View.TYPES, fdu.View.FAMILIES, fdu.View.DOCUMENTS),
        selection=fdu.Selection(limit=20, size=fdu.SizeMetric.APPARENT),
    )
)
print(report.provenance.freshness)
print(report.sections)
print(report.as_dict())

mark = index.clock
result = index.refresh()
print(result.status, index.since(mark).changes)

The index API (Index) returns whole selected results in bulk while retaining the inventory in Rust. The opened API below also supports continuation-based paging. Open, scan, and the native reconciliation phase of refresh run with the GIL released, so unrelated Python threads and independent indexes can progress. So does a report’s capacity observation, only when it asks for View.VOLUMES; every other Index.report holds the GIL. Content analysis streams every eligible file through EOF. Binary data, invalid UTF-8, and unsupported SLOC languages remain visible as coverage without making the operation partial; I/O failures and files changed during a read remain operational errors. One Index object still has PyO3 runtime borrow exclusion: an overlapping call on that same object is rejected rather than becoming an unsynchronized shared-index access. The wheel enables the optional watch dependency and exposes Index.watch() as a closable, query-bound change feed. It copies the index into an independent session; iteration verifies and applies changes there and drives persistence. Watch.report() returns its latest applied answer without draining events or updating the original Index. Native watch startup currently holds the GIL, while iteration waits release it. Content analysis itself remains one-shot: refresh reanalyzes after metadata reconciliation, while a watch feed reports metadata changes.

Analysis names the same analyzers as the command line’s --analyze: lines, code, and words, with none and all as totals. AnalysisOptions takes one of them or a comma-separated set such as "code,words", plus a worker count. A content view requests the analysis it shows, as --view code does on the command line: fdu.report(root, fdu.Query(views=(fdu.View.CODE, fdu.View.DOCUMENTS))) runs the code and words analyzers without naming them. An index holds exactly the analyzers it was opened with, so index.report never adds one: the same view on an index opened without words raises InvalidArgumentError naming the analyzer. AnalysisOptions is the control for the rest: analysis shown in a metadata view, such as code lines in View.LANGUAGES, and analysis retained for an index’s reports and rerun by refresh. Typed report sections expose stable type/family groups, exact share fractions, line and word slots, page denominators, coverage outcomes, analyzer provenance, detection source and confidence, and generated/vendor/documentation flags. The original extension grouping remains available as the extensions view. The package supports Python 3.12 and newer and builds one abi3-py312 extension rather than separate native payloads for every Python minor release.

Population, Code Overview, and Cache Destination

fdu.report() derives discovery and content analysis from Selection.ignored:

import fdu

report = fdu.report(
    ".",
    fdu.Query(
        views=(fdu.View.CODE,),
        selection=fdu.Selection(ignored=fdu.IgnoredEntries.EXCLUDE),
    ),
    cache_dir="/path/to/cache",
)

include is the default. exclude prunes safely ignored subtrees; only discovers ignored matches through ordinary ancestors and analyzes only ignored bodies. For retained work, fdu.open(..., ignored=...) and fdu.scan(..., ignored=...) choose the initial population. A default read inherits it. An Include index can answer a narrower selection; an index that never retained a population cannot widen its answer.

View.CODE runs code analysis and reports a CodeSection, the view code analysis also defaults to, whose overview includes selected source lines, language shares, ignored/non-ignored contributions, and coverage. SortKey.CODE_LINES ranks files or directories when code analysis was requested. File rows carry nullable sort_value and classification evidence; unavailable counts remain distinct from zero.

cache_dir on open/report and cache lifecycle calls names the exact destination and wins over FDU_CACHE_DIR. Otherwise XDG_CACHE_HOME/fdu wins over the platform default: ~/.cache/fdu on macOS and Linux, %LOCALAPPDATA%/fdu on Windows. list_caches and clear_all_caches take a keyword cache directory, independently of a scan root. cache_directory(cache_dir=...) resolves that destination without a root. cache_path(root, cache_dir=...) identifies the root’s .metadata.bin file; a matching .analysis.bin stores derived metrics without source bodies.

Directory Inventories and Formats

A default Query() keeps the existing directory tree. Select a flat presentation when requesting a report, so an ordinary tree need not materialize a complete inventory:

from pathlib import Path

import fdu

index = fdu.open(Path("/path/to/tree"))
for name in (".venv", "venv", "node_modules", "target"):
    report = index.report(
        fdu.Query(
            format=fdu.Format.LONG,
            selection=fdu.Selection(
                kinds=(fdu.EntryKind.DIR,),
                include=(name,),
                before="30d",
                sort=fdu.SortKey.TIME,
                reverse=True,
            ),
        )
    )
    print(report.render())
    print(report.render(fdu.Format.JSON))

All four reads use one retained index without rescanning. View.LIST is the metadata default. Format.TREE explicitly requests the current tree; PATHS lists matching paths, LONG adds size and signed age, and JSON/JSONL/YAML carry exact metrics. TEXT selects automatic human presentation. The View.FILES preset uses flat name ordering, and View.TREE presents structured directory output. Largest/recent remain regular-file presets, with optional Paths/Long output.

Every time a report reads is on one clock, Selection.time: TimeBasis.CHANGED by default, the newer of modification and status change, so a copy or an extracted archive that kept an older modification time is still new, or TimeBasis.MODIFIED, content age as ls -l shows. The since/before window, SortKey.TIME, every age, recent, and ages read it, and Report.request.time names it.

Recent Writes, Ages, and Volumes

One walk says where the data written in a window now lives, how the selection divides by age, and how full each filesystem is:

from pathlib import Path

import fdu

index = fdu.open(Path("/path/to/tree"))
recent = index.report(
    fdu.Query(
        views=(fdu.View.TREE, fdu.View.AGES),
        selection=fdu.Selection(kinds=(fdu.EntryKind.FILE,), since="24h"),
    )
)
print(f"{recent.selected.bytes} of {recent.scanned.bytes} bytes written in a day")
for section in recent.sections:
    if isinstance(section, fdu.AgesSection):
        for band in section.bands:
            print(band.band, band.tally.files, band.cumulative.bytes)

disk = index.report(fdu.Query(views=(fdu.View.VOLUMES,)))
for section in disk.sections:
    if isinstance(section, fdu.VolumesSection):
        for row in section.volumes:
            print(row.path, row.capacity_bytes, row.available_bytes, row.counted.bytes)

Keep EntryKind.FILE with a window: without it a directory with any activity in the window is selected whole. Every report carries Report.selected and Report.scanned, each an EntryTotals of files, directories, and both byte metrics, so a filtered root’s 100% can be read against the whole scan without parsing a note. View.AGES returns an AgesSection: nine AgeBand values, newest first, each with an AgeTally of files, bytes, and the ignored part, a cumulative tally of it and every newer band, and a total equal to a summary of the same selection. Where the selection admits files alone, each band is also the files kinds=(EntryKind.FILE,) with the window it names would select. View.VOLUMES returns a VolumesSection of VolumeRow values: each filesystem the report touches, its capacity, available space, type, and whether it shares space in a pool, beside the VolumeCounted files and bytes this report counted there. A filesystem the report entered at several paths is one row, named by the first, with each other path in also_entered as a VolumeEntered. Capacity is read when the report is, and below the root only where the walk listed and the operating system’s mount table shows a mounted filesystem, not an automount trigger (autofs), at exactly that path, so reading it mounts nothing, except an automount that expires between the report’s read of the mount table and its call there, a window as long as the calls before it. Any other row’s capacity is None, with an unknown_reason that says why, such as not a mount point now, so it was not asked; View.FULL leaves volumes out.

Directory FileRow values carry subtree bytes, allocated, files, dirs, complete, time_ns, age_ns, and time_at; non-directory counts are None. A directory whose subtree was not listed in full has complete=False, lower-bound sizes, age_ns=None, and time_at=None. Every FileRow also carries both raw clocks, mtime_ns and changed_ns, so an arrival carrying old content is a row with a recent changed_ns and an old mtime_ns. Each TreeNode carries the same time_ns, complete (None for a file), age_ns, and time_at: the newest time among the entries the row counts, of any kind, never the report root’s own time, and None when it counts nothing. Its newest_file_time_ns keeps the newest regular file alone. Report.age_reference_ns is the instant every age is measured from, including negative future ages and pre-epoch mtime, and Report.age_reference_at is that instant as a datetime. A Watch.report() measures from the instant it is called; the windows its query selects by stay as the query resolved them. time_at and age_reference_at are timezone-aware UTC datetime values derived from the exact nanoseconds, floored to the microsecond. An unrepresentable reference yields None age. Exclusions win throughout the subtree; nested roots may overlap, while grouped totals count their union once. Age describes change, not access or last use. Native entry/lookup projections retain inode attributes; these report rows deliberately carry subtree metrics instead.

A Report owns its requested projection. Re-render it to another serialization or between Paths and Long without querying again; request another report to change between a bounded tree and complete flat inventory. An incompatible conversion raises InvalidArgumentError rather than silently listing only visible tree rows. Selection(depth=5, min_share="1%", breadth=Bound.ALL, limit=Bound.ALL) describes the ordinary tree defaults. Breadth bounds children per directory; limit bounds data rows per section. TreeSection.limits records the effective controls. Each TreeOmission.entries counts directly hidden children: an omitted directory is one entry, while its nullable files count covers regular files throughout that hidden subtree. The optional TreeSection.remainder combines hidden contents across one tree, with nullable files, bytes, and allocated totals and bound reasons in stable order. It is None when nothing is hidden. A displayed root-level directory represents all its descendants, including children hidden by display bounds. The remainder counts only usage outside those displayed root-level rows. Parent totals already include the remainder, so do not add it to them. The terminal gives that remainder one root-level line, such as … and 12,345 more files, with its combined size and root share in the usual columns. Use report.render(bar_size=20) for wider bars, or bar_size=0 (also negative values) to hide them; the default is 10. Machine formats are unchanged. Machine reports use fdu.report/12. Machine List output is complete unless explicitly limited. Details and exact fields are in the usage guide and machine-output reference.

Long-Lived Roots

fdu.opened exposes the progressive API (OpenedIndex) over the shared native engine. It starts progressive discovery, returns several projections from one coherent version, exposes exact resumable changes, verifies explicit path sets, accepts discovery priorities, and joins all native work on close:

from pathlib import Path

from fdu import opened

with opened.OpenedIndex.open(Path("."), opened.OpenedOptions(observe=True)) as index:
    answer = index.read(
        opened.Tree(page=opened.Page(limit=200, max_work=100_000)),
        opened.Diagnostics(),
    )
    cursor = answer.change_cursor
    changed = index.changes(cursor, timeout=1.0)

The methods are synchronous. Opened native open/read/state, change polling, refresh, prioritization, and close release the GIL. Report serialization and Python result construction currently hold it. An async application adapts them with its own executor policy, keeping task and shutdown ownership visible. The package does not run a private event loop or shell out to the command line. Cancelling an executor future does not itself cancel native work. Closing an OpenedIndex shuts down its shared root for all aliases rather than one request.

Retain the inventory in Rust and request summaries, selected answers, or bounded pages for ordinary Python use. Complete reports remain available when needed, with allocation cost proportional to the selected result. Public reports eagerly decode JSON into dictionaries and typed Python models and retain both the wire representation and a native rendering handle; as_dict() copies the wire containers again. Neither reports nor pages are zero-copy exports. Complete paged enumeration can bound Python retention when previous pages are released, subject to continuation invalidation as the engine version changes.

The wheel also exposes the native Rust CLI as the fdu console script. Argument parsing, help, streams, color, errors, broken-pipe handling, and exit status all use the same Rust process boundary as the Cargo-installed binary; there is no Python CLI reimplementation.

Status: 0.x. A new minor release may change the Python API; the release process states the compatibility rules. Building and testing the package from a checkout is covered in the repository README.

License: MIT.