anscom

High-performance native C recursive file scanner for Python. v1.5.3

MIT Licensed

Multi-threaded · Terabyte-scale · Zero dependencies · Cross-platform

pip install anscom

---

What it is

Anscom is a Python C extension that scans directories at raw OS speed. It uses direct kernel syscalls (getdents64 on Linux, FindFirstFileW on Windows, readdir/lstat on macOS), a multi-threaded work queue, and per-thread statistics accumulation. It never loads file contents into memory. It never follows symlinks. It never slows down as the filesystem grows.

The result is always a plain Python dict — six keys minimum, more when you ask for them.

import anscom

result = anscom.scan("/mnt/storage")
# → {'total_files': 2841903, 'scan_errors': 0, 'duration_seconds': 1.87,
#    'scanned_path': '/mnt/storage', 'categories': {...}, 'extensions': {...}}

2.8 million files. 1.87 seconds. 16 threads. No configuration.

---

What's New in v1.5.3

A fully backward-compatible release that answers the question counts alone can't: where did the disk space actually go? Counting 8,900 code files tells you nothing if 1,240 videos are eating 340 GB. largest_n shows you the top individual files; this shows you the totals.

Feature	Parameter	Description
Byte accounting	with_size=True	Adds total_size, category_sizes and extension_sizes to the result dict — bytes alongside the existing counts
JSON export	export_json (existing)	The three new keys are written to the JSON report automatically when with_size=True — no new parameter needed

result = anscom.scan("/mnt/storage", with_size=True, silent=True)

print(result["total_size"])          # 366_917_402_112
print(result["category_sizes"])      # {'Videos': 340_..., 'Code/Source': 2_..., ...}
print(result["extension_sizes"])     # {'mkv': 210_..., 'mp4': 130_..., 'py': 8_..., ...}

Why it's opt-in. On Linux the fast path uses d_type from getdents64 and never calls stat() unless a feature needs it. Byte totals need st_size, so with_size=True costs one extra stat() per file. Leave it off and the count-only hot path stays exactly as fast as v1.5.2 — zero extra syscalls. On Windows the size arrives free in WIN32_FIND_DATAW, and the macOS/BSD backend already calls lstat, so neither pays anything extra either way.

Nothing else changed. Without the flag the result dict has the identical shape it had in v1.5.2 — the three new keys are simply absent. Every existing script runs unchanged.

---

What's New in v1.5.2

A tiny, fully backward-compatible release that adds a mascot. No scan behavior, call signature, or output format changes — every existing script runs exactly as before.

Feature	How	Description
Mascot	anscom.about()	Prints Chuhi, the anscom mascot (the house mouse of a Kolkata apartment), plus a short credit line. Returns None.

import anscom
anscom.about()      # meet Chuhi

---

What's New in v1.5.1

A small, fully backward-compatible release focused on ergonomics and correctness. No call signature or output format from v1.5.0 changes — every existing script runs unchanged, and the three optional keys behave exactly as before.

Feature	How	Description
Current directory	anscom.pwd()	Returns the current working directory as an absolute path string — the Unix pwd, as a value you can assign or pass.
No-argument scan	anscom.scan()	path is now optional; called with no path it scans the current working directory.
Resolved root	result["scanned_path"]	Every result dict now reports the absolute path that was actually walked — always present.

Under the hood this release also fixes Python reference leaks in result-dict construction and drops a per-thread buffer that was pre-allocated even on count-only scans, so a plain anscom.scan() now does zero heap work on the hot path. See the Changelog.

---

What's New in v1.5.0

v1.5.0 is a major feature release — the largest single update since the initial release. Every existing parameter, behavior, and output format from v1.3.0 is fully preserved.

Feature	Parameter	Description
File list return	return_files=True	Returns every scanned file as a list of dicts with path,size,ext,category,mtime
CSV export	export_csv="out.csv"	Writes per-file data to a UTF-8 CSV — zero dependencies
Largest-N report	largest_n=20	Top N files by size via per-thread min-heap — O(log N) per file, no extra pass
Duplicate detection	find_duplicates=True	Groups files by size then CRC32 of first 4KB — returns grouped path lists
Regex filter	regex_filter="pattern"	Only counts files whose full path matches the pattern. Uses POSIX regexecon Linux/macOS (no GIL); Python refallback on Windows

Performance note: All five features are strictly opt-in. A plain anscom.scan(".") with no new parameters runs the identical hot path as v1.3.0 — no extra syscalls, no allocations per file, no behavioral change.

Migration from v1.3.0

No breaking changes. All v1.3.0 code runs unchanged on v1.5.0. The new parameters all default to off.

# v1.3.0 code — works identically on v1.5.0
result = anscom.scan("/data", silent=True, ignore_junk=True)

# v1.5.0 — opt into new features as needed
result = anscom.scan(
    "/data",
    silent          = True,
    ignore_junk     = True,
    return_files    = True,   # new
    largest_n       = 20,     # new
    find_duplicates = True,   # new
    export_csv      = "inventory.csv",  # new
)

---

Table of Contents

• Installation
• Quick Start
• Full API Reference
• Return Value
• All Parameters in Depth
  • path
  • max_depth
  • workers
  • min_size
  • extensions
  • ignore_junk
  • silent
  • show_tree
  • callback
  • export_json
  • export_tree
  • export_csv
  • return_files
  • largest_n
  • find_duplicates
  • regex_filter
  • with_size
• Export Features
• Current Directory
• Tree Mode
• Exclusion Filter
• Report Format
• File Categories and Extensions
• Architecture
• Security and Compliance
• Recipes
• Changelog
• License

---

Installation

pip install anscom

Requires Python 3.6+. Works on Linux, macOS, and Windows.

Windows source builds require the "Desktop development with C++" workload from Visual Studio Build Tools.

No runtime dependencies. Every feature in v1.5.0 works with nothing else installed.

Verify

import anscom
r = anscom.scan(".", silent=True)
print(r["total_files"], "files —", round(r["duration_seconds"], 3), "s")

---

Quick Start

import anscom

# Default scan — prints live counter + full report
anscom.scan(".")

# Silent scan — just get the dict
result = anscom.scan(".", silent=True)

# Scan a specific path with more depth
result = anscom.scan("/home/user/projects", max_depth=20, silent=True)

# Print the category breakdown
for cat, count in result["categories"].items():
    if count > 0:
        print(f"{cat:20s} {count:>10,}")

---

Full API Reference

anscom.scan(
    path = ".",              # str      — optional (defaults to CWD, like pwd)
    max_depth    = 6,        # int
    show_tree    = False,    # bool
    workers      = 0,        # int
    min_size     = 0,        # int
    extensions   = None,     # list[str] | None
    callback     = None,     # callable | None
    silent       = False,    # bool
    ignore_junk  = False,    # bool
    export_json  = None,     # str | None
    export_tree  = None,     # str | None
    return_files = False,    # bool       ← new in v1.5.0
    export_csv   = None,     # str | None ← new in v1.5.0
    largest_n    = 0,        # int        ← new in v1.5.0
    find_duplicates = False, # bool       ← new in v1.5.0
    regex_filter = None,     # str | None ← new in v1.5.0
    with_size    = False,    # bool       ← new in v1.5.3
) -> dict

---

Return Value

The return value is always a dict. Six keys are always present. Six more are added on demand.

Key	Type	Always?	Description
total_files	int	✓	Files that passed all filters and were categorized
scan_errors	int	✓	Paths that failed to open (permissions, broken links)
duration_seconds	float	✓	Wall-clock time from first thread spawn to last join
scanned_path	str	✓	Absolute path that was actually walked (resolved from path)
categories	dict[str, int]	✓	All 9 categories, always present even if zero
extensions	dict[str, int]	✓	Only non-zero extension counts
files	list[dict]	return_files=True	Per-file records:path,size,ext,category,mtime
largest_files	list[dict]	largest_n > 0	Top-N files by size:path,size
duplicates	list[list[str]]	find_duplicates=True	Groups of paths sharing identical content (size + CRC32)
total_size	int	with_size=True	Total bytes across every file that was counted
category_sizes	dict[str, int]	with_size=True	All 9 categories, in bytes, always present even if zero
extension_sizes	dict[str, int]	with_size=True	Only non-zero extension byte totals

The nine category keys inside result["categories"]:

"Code/Source"    "Documents"      "Images"         "Videos"
"Audio"          "Archives"       "Executables"    "System/Config"
"Other/Unknown"

---

All Parameters in Depth

path

Type: str — Optional — Default: current working directory

The root directory to scan. Accepts relative paths (., ../data), absolute paths (/mnt/storage, C:\Users), an empty string, or None. Omit it entirely — or pass "" / None — to scan the current working directory (the Unix pwd). Whatever you pass, the absolute path that was actually walked is returned as result["scanned_path"].

anscom.scan()      # scan the current working directory
anscom.scan(".")
anscom.scan("/mnt/nas")
anscom.scan("C:\\Users\\Aditya\\Documents")
anscom.scan("")    # same as "." / no argument

---

max_depth

Type: int — Default: 6 — Range: [0, 64]

Maximum directory recursion depth. Depth 0 means only the immediate children of path are examined — no subdirectories are entered. Depth 64 is the hard ceiling enforced in C.

# Only the top level — no recursion
anscom.scan("/data", max_depth=0, silent=True)

# Standard project scan
anscom.scan("/project", max_depth=6, silent=True)

# Deep NAS or archive scan
anscom.scan("/mnt/archive", max_depth=30, silent=True)

# Maximum depth — unlimited for practical purposes
anscom.scan("/", max_depth=64, silent=True)

Values below 0 are clamped to 0. Values above 64 are clamped to 64.

---

workers

Type: int — Default: 0

Number of worker threads. 0 auto-detects the hardware CPU count via sysconf(_SC_NPROCESSORS_ONLN) on Linux/macOS and GetSystemInfo() on Windows. If auto-detection fails, falls back to 4.

When show_tree=True, workers is forced to 1 regardless of what is passed — multiple threads writing to stdout would produce interleaved output.

# Auto (recommended for most cases)
anscom.scan("/data", workers=0)

# Pin to a specific count
anscom.scan("/data", workers=8)

# Maximum parallelism on a 64-core machine
anscom.scan("/data", workers=64)

At shallow depths the work queue feeds all threads efficiently. At depth >= 3 each thread recurses inline, so thread count has diminishing returns past ~16 for typical filesystems unless the tree is extremely wide.

---

min_size

Type: int — Default: 0 (no filter)

Skip all files smaller than this many bytes. Files below the threshold are not counted, not categorized, and not included in return_files or export_csv output.

# Only files larger than 1 MB
anscom.scan("/data", min_size=1024 * 1024, silent=True)

# Only files larger than 100 MB
anscom.scan("/mnt/video", min_size=100 * 1024 * 1024, silent=True)

# Only files larger than 1 GB
anscom.scan("/mnt/backup", min_size=1024 ** 3, silent=True)

On Linux, fstatat() is called to retrieve file size only when this filter is active. On Windows, the size is available directly in WIN32_FIND_DATAW at no extra syscall cost.

---

extensions

Type: list[str] | None — Default: None

Extension whitelist. When set, only files whose extension matches one of the listed strings are counted. All other files are silently skipped — they do not appear in counts, categories, files, export_csv, or any other output.

Pass extensions without the leading dot, lowercase.

# Count only Python files
result = anscom.scan("/repo", extensions=["py"], silent=True)

# Count only web code
result = anscom.scan("/project", extensions=["js", "ts", "jsx", "tsx", "css", "html"])

# Count only media
result = anscom.scan("/media", extensions=["mp4", "mkv", "mov", "avi", "mp3", "flac"])

# Count only documents
result = anscom.scan("/docs", extensions=["pdf", "docx", "xlsx", "pptx", "md", "txt"])

Unknown extensions (not in the built-in table) are also excluded when a whitelist is active.

---

ignore_junk

Type: bool — Default: False

When True, the following directories are skipped entirely — no opendir, no syscall, no recursion. The check is a case-insensitive match on the directory basename, at any depth, under any parent.

Skipped directories:

Category	Directories
Version control	.git .svn .hg
IDE metadata	.idea .vscode
Dependency trees	node_modules bower_components site-packages .venv venv env
Build output	build dist target __pycache__
Cache / temp	temp tmp .cache .pytest_cache .mypy_cache

# Measure dependency bloat
raw   = anscom.scan("/project", ignore_junk=False, silent=True)
clean = anscom.scan("/project", ignore_junk=True,  silent=True)
bloat = raw["total_files"] - clean["total_files"]
print(f"Dependency files: {bloat:,}")

# Fast production audit — skip all junk
result = anscom.scan("/codebase", ignore_junk=True, workers=32, silent=True)

The default is False — Anscom counts everything unless you opt in to exclusions.

---

silent

Type: bool — Default: False

When False (default), Anscom prints:

• A live "Scanned files: N ..." counter that updates every 250ms
• The full summary report and extension breakdown on completion

When True, all of that is suppressed. The returned dict is always identical regardless of this flag.

silent=True does not suppress tree output from show_tree=True — those are separate.

# For scripting — no output, just the data
result = anscom.scan("/data", silent=True)

# For interactive use — full live output
anscom.scan("/data")

---

show_tree

Type: bool — Default: False

When True, prints a DFS-ordered directory tree to sys.stdout as each entry is discovered. Forces workers=1 to guarantee correct ordering.

  |-- [src]
  |   |   |-- main.py
  |   |   |-- utils.py
  |   |   |-- [tests]
  |   |   |   |   |-- test_main.py
  |-- [docs]
  |   |   |-- readme.md
  |-- config.json

• Square brackets [name] indicate a directory
• No brackets indicates a regular file
• Each depth level adds " | " (6 characters) of indentation

Output is produced one line at a time via PySys_WriteStdout. Any sys.stdout redirect in Python will capture every line. There is no internal buffer — a 50 million file filesystem produces 50+ million lines without accumulating memory.

# Print tree to terminal
anscom.scan(".", show_tree=True, max_depth=4)

# Capture tree in Python
import io, sys
buf = io.StringIO()
sys.stdout = buf
anscom.scan(".", show_tree=True, max_depth=3, silent=True)
sys.stdout = sys.__stdout__
tree_text = buf.getvalue()

# Save tree to file (see also export_tree)
anscom.scan("/data", show_tree=True, silent=True, export_tree="tree.txt")

---

callback

Type: callable | None — Default: None

A Python callable invoked approximately every 1 second with the current scanned file count as a single int argument. Fired by the progress thread every 4th tick (250ms × 4 = 1000ms).

The GIL is acquired before each call and released immediately after. Scan worker threads are never blocked by callback invocation.

def on_progress(n):
    print(f"\rScanned: {n:,}", end="", flush=True)

result = anscom.scan("/data", callback=on_progress, silent=True)
print()

# Push to Prometheus
from prometheus_client import Gauge
g = Gauge("files_scanned", "Current file scan count")
anscom.scan("/data", callback=lambda n: g.set(n), silent=True)

---

export_json

Type: str | None — Default: None

Path to write the full result dict as a formatted JSON file. Uses Python's built-in json module — no external dependencies. Written with 4-space indentation after the scan completes.

The JSON file contains all keys that are in the returned dict, including optional keys (files, largest_files, duplicates) when those features are enabled in the same call.

anscom.scan("/data", export_json="report.json", silent=True)

# With optional features — JSON gets those keys too
anscom.scan(
    "/data",
    export_json     = "report.json",
    return_files    = True,
    largest_n       = 10,
    find_duplicates = True,
    silent          = True
)

Example output:

{
    "total_files": 21008,
    "scan_errors": 0,
    "duration_seconds": 1.5186,
    "categories": {
        "Code/Source": 5955,
        "Documents": 203,
        "Images": 151,
        "Videos": 0,
        "Audio": 730,
        "Archives": 0,
        "Executables": 0,
        "System/Config": 5707,
        "Other/Unknown": 8992
    },
    "extensions": {
        "py": 5955,
        "pyc": 5707,
        "mp3": 730,
        "txt": 160,
        "png": 151
    }
}

---

export_tree

Type: str | None — Default: None

Path to write the tree output to a text file. Only active when show_tree=True.

The file is written incrementally — each line is written and flushed as it is produced. For a filesystem with 50 million entries this produces a multi-gigabyte file without accumulating any output in memory. stdout and the file both receive every line simultaneously.

anscom.scan(
    "/mnt/storage",
    show_tree   = True,
    export_tree = "filesystem_tree.txt",
    silent      = True,
    max_depth   = 64
)

---

export_csv

Type: str | None — Default: None — New in v1.5.0

Path to write a per-file inventory as a UTF-8 CSV. Columns: path, size, ext, category, mtime.

• path: full absolute path, RFC 4180-quoted (double-quoted, inner quotes doubled)
• size: file size in bytes as an integer
• ext: lowercase extension without the dot (empty string for unrecognized extensions)
• category: one of the 9 category names
• mtime: Unix timestamp (seconds since epoch) of last modification

anscom.scan("/data", export_csv="inventory.csv", silent=True)

Loading the CSV downstream:

# With pandas
import pandas as pd
df = pd.read_csv("inventory.csv")
print(df.groupby("category")["size"].sum().sort_values(ascending=False))

# Convert to Excel
df.to_excel("report.xlsx", index=False)

# Standard library only
import csv
with open("inventory.csv", newline="", encoding="utf-8") as f:
    for row in csv.DictReader(f):
        print(row["path"], row["size"])

# With openpyxl directly
import csv, openpyxl
wb = openpyxl.Workbook()
ws = wb.active
with open("inventory.csv", newline="", encoding="utf-8") as f:
    for row in csv.reader(f):
        ws.append(row)
wb.save("report.xlsx")

---

return_files

Type: bool — Default: False — New in v1.5.0

When True, the result dict gains a "files" key containing a Python list of dicts — one entry per scanned file.

Each dict has five fields:

Field	Type	Description
path	str	Full absolute path to the file
size	int	File size in bytes
ext	str	Lowercase extension (no dot), empty string if unrecognized
category	str	One of the 9 category names
mtime	int	Unix timestamp of last modification

result = anscom.scan("/project", return_files=True, silent=True)

# Iterate
for f in result["files"]:
    print(f["path"], f["size"], f["category"])

# Filter in Python
large_code = [
    f for f in result["files"]
    if f["category"] == "Code/Source" and f["size"] > 50_000
]

# Sort by size descending
by_size = sorted(result["files"], key=lambda f: f["size"], reverse=True)
print("Largest file:", by_size[0]["path"])

# Group by extension
from collections import defaultdict
by_ext = defaultdict(list)
for f in result["files"]:
    by_ext[f["ext"]].append(f)

len(result["files"]) == result["total_files"] is always true.

---

largest_n

Type: int — Default: 0 (disabled) — New in v1.5.0

When > 0, finds the top N files by size across the entire scanned filesystem. Uses a per-thread min-heap of capacity N — O(log N) per file, no extra pass, no sorting of the full file list. After all threads join, per-thread heaps are merged and sorted descending.

The result dict gains a "largest_files" key where each entry is a dict with path (str) and size (int).

result = anscom.scan("/mnt/storage", largest_n=20, silent=True)

for f in result["largest_files"]:
    gb = f["size"] / (1024 ** 3)
    print(f"{gb:8.2f} GB  {f['path']}")

The printed report also gains a section:

=== TOP 20 LARGEST FILES ===========================
  1073741824 bytes : /data/backup/archive.tar.gz
   536870912 bytes : /data/media/4k_reel.mkv
...
===================================================

# Find the single largest file
result = anscom.scan("/mnt/nas", largest_n=1, silent=True)
top = result["largest_files"][0]
print(f"Largest: {top['path']} ({top['size']:,} bytes)")

# Top 100 across a petabyte volume
result = anscom.scan("/mnt/petabyte", largest_n=100, workers=64, silent=True)

---

find_duplicates

Type: bool — Default: False — New in v1.5.0

When True, detects duplicate files using a two-phase algorithm:

1. Size bucketing — all files sorted by size. Files with a unique size are skipped entirely — zero I/O.
2. CRC32 fingerprinting — for each same-size group (≥2 files, non-zero size), the first 4096 bytes of each file are read and CRC32 is computed. Files with matching CRC32 are reported as duplicates.

The result dict gains a "duplicates" key: a list of groups, each group being a list of path strings. Every group has at least 2 members.

result = anscom.scan("/media-library", find_duplicates=True, silent=True)

print(f"Duplicate groups: {len(result['duplicates'])}")

for group in result["duplicates"]:
    print(f"\nDuplicate set ({len(group)} files):")
    for path in group:
        print(f"  {path}")

Calculating reclaimable space (combine with return_files=True):

result = anscom.scan(
    "/mnt/archive",
    find_duplicates = True,
    return_files    = True,
    silent          = True
)

size_map = {f["path"]: f["size"] for f in result["files"]}

wasted = sum(
    sum(size_map.get(p, 0) for p in group[1:])   # keep 1, discard rest
    for group in result["duplicates"]
)

print(f"Reclaimable: {wasted / (1024**3):.2f} GB across {len(result['duplicates'])} groups")

The printed report adds:

=== DUPLICATES SUMMARY ============================
Groups found : 142
===================================================

---

regex_filter

Type: str | None — Default: None — New in v1.5.0

A regular expression pattern. When set, only files whose full absolute path matches the pattern are counted, categorized, and included in any file-tracking output (return_files, export_csv, find_duplicates, largest_n).

Platform behavior:

• Linux / macOS: Compiled with POSIX regcomp(REG_EXTENDED | REG_NOSUB), matched with regexec — no GIL acquisition , runs fully in C inside the worker threads.
• Windows: Falls back to Python's re module (GIL acquired per file). For large scans on Windows, prefer extensions whitelist filtering which has zero GIL cost.

The pattern is also compiled with Python's re.compile before the scan starts. An invalid pattern raises ValueError immediately.

# Only .py files anywhere under a tests/ directory
result = anscom.scan("/codebase", regex_filter=r"/tests/.*\.py$", silent=True)

# Only files in directories named 'src'
result = anscom.scan("/project", regex_filter=r"/src/", silent=True)

# Only Python test files
result = anscom.scan("/repo", regex_filter=r"test_.*\.py$", silent=True)
print(f"Test files: {result['total_files']}")

# Invalid patterns raise ValueError immediately — no scan is started
try:
    anscom.scan("/data", regex_filter=r"[invalid(")
except ValueError as e:
    print(e)  # Failed to compile regex_filter.

---

with_size

Type: bool — Default: False — New in v1.5.3

When True, Anscom accumulates byte totals alongside the counts it already collects. The result dict gains three keys:

Key	Type	Description
total_size	int	Sum of every counted file's size, in bytes
category_sizes	dict[str, int]	Bytes per category — all 9 keys present, mirroring categories
extension_sizes	dict[str, int]	Bytes per extension — only non-zero entries, mirroring extensions

result = anscom.scan("/mnt/storage", with_size=True, silent=True)

def gb(n):
    return f"{n / 1024**3:.2f} GB"

print("Total:", gb(result["total_size"]))

for cat, size in sorted(result["category_sizes"].items(),
                        key=lambda kv: kv[1], reverse=True):
    if size:
        print(f"  {cat:16s} {gb(size):>12s}")

Counts and bytes always agree. The byte totals cover exactly the files that were counted — never a superset. Every filter (extensions, regex_filter, min_size) is applied before a file's size is added, so total_size is always the true sum of the total_files files.

# Whatever the whitelist admits, total_size is the sum of exactly those files
r = anscom.scan("/repo", extensions=["py"], with_size=True, silent=True)
print(r["total_files"], "files,", r["total_size"], "bytes")

Where counts mislead and bytes don't:

r = anscom.scan("/mnt/nas", with_size=True, silent=True)

for cat in r["categories"]:
    n, b = r["categories"][cat], r["category_sizes"][cat]
    if n:
        print(f"{cat:16s} {n:>9,} files  {b/1024**3:>9.2f} GB  "
              f"{b/n/1024:>8.1f} KB avg")

# Code/Source        8,912 files       2.06 GB     242.3 KB avg
# Videos             1,240 files     340.11 GB  287654.2 KB avg
#   ← 7x fewer files, 165x more disk

Cost. Opt-in for a reason. On Linux the scanner reads directory entries with getdents64 and uses d_type to identify regular files without ever calling stat(). Byte totals need st_size, so with_size=True adds one fstatat() per file. On a multi-million-file tree that is measurable. Leave the flag off and the count-only path is byte-for-byte the v1.5.2 hot path.

The extra call is also skipped when it isn't needed: if min_size, largest_n, return_files, export_csv or find_duplicates is already active, the size has been fetched anyway and with_size reuses it for free.

Platform	Extra cost with with_size=True
Linux	One fstatat() per file — unless another feature already needs it
Windows	None — size is already in WIN32_FIND_DATAW
macOS / BSD	None — the backend already calls lstat

With export_json — the three keys are picked up automatically:

anscom.scan("/data", with_size=True, export_json="report.json", silent=True)

{
    "total_files": 21008,
    "total_size": 4823910442,
    "category_sizes": {
        "Code/Source": 210554901,
        "Documents": 88120043,
        "Images": 512004112,
        "Videos": 3912004112,
        "Audio": 101227274,
        "Archives": 0,
        "Executables": 0,
        "System/Config": 0,
        "Other/Unknown": 0
    },
    "extension_sizes": {
        "mp4": 3912004112,
        "png": 512004112,
        "py": 210554901,
        "mp3": 101227274,
        "pdf": 88120043
    }
}

Note on export_csv — the CSV already carries a per-file size column and is unchanged by this flag. Aggregate totals are deliberately kept out of it so the file stays a clean rectangular table that pandas.read_csv can load without special handling. Roll your own totals downstream:

import pandas as pd
df = pd.read_csv("inventory.csv")
print(df.groupby("category")["size"].sum().sort_values(ascending=False))

The printed report is unchanged — byte totals are returned as data, not printed.

---

Export Features

All export parameters are independent and combinable. A single scan pass can write to all simultaneously — one traversal, multiple outputs, no re-scanning.

result = anscom.scan(
    "/mnt/storage",
    max_depth       = 20,
    workers         = 32,
    ignore_junk     = True,
    silent          = True,
    largest_n       = 50,
    find_duplicates = True,
    return_files    = True,
    export_json     = "audit.json",
    export_csv      = "inventory.csv",
    show_tree       = True,
    export_tree     = "tree.txt",
)
# One scan. Four output files. Full in-memory results.

Parameter	Format	Dependencies	Notes
export_json	JSON	None (built-in)	Full result dict including optional keys
export_csv	CSV	None (built-in)	Per-file: path, size, ext, category, mtime
export_tree	Plain text	Requires show_tree=True	Written line-by-line, safe at any scale

---

Current Directory

Two conveniences for working with the current working directory, both returning values — nothing is printed, so the results stay composable.

anscom.pwd()

Returns the current working directory as an absolute path string — the Unix pwd, as a value.

import anscom

root = anscom.pwd()          # e.g. '/home/user/project'
result = anscom.scan(root)   # feed it straight back in

Signature: anscom.pwd() -> str. Takes no arguments. Raises OSError if the working directory cannot be read (for example, if it has been deleted).

No-argument scan() and scanned_path

anscom.scan() with no path scans the current working directory. Regardless of what path you pass, the returned dict always includes scanned_path — the absolute path that was actually walked, resolved via realpath on Linux/macOS and _fullpath on Windows.

r = anscom.scan(silent=True)          # scans the current directory
print(r["scanned_path"])              # → '/home/user/project'

r = anscom.scan("../logs", silent=True)
print(r["scanned_path"])              # → resolved absolute path, not '../logs'

If a path cannot be resolved (missing or unreadable), scanned_path falls back to the raw input string, so the key is always present.

---

Mascot (new in v1.5.2)

Meet Chuhi — the anscom mascot, named after the house mouse of a Kolkata apartment.

anscom.about()

Prints the mascot as ASCII art followed by a short credit line, then returns None. It takes no arguments.

import anscom
anscom.about()

Signature: anscom.about() -> None. The art is about 82 columns wide, so a reasonably wide terminal shows it best.

---

Tree Mode

# Basic tree to terminal
anscom.scan(".", show_tree=True)

# Tree saved to file
anscom.scan("/project", show_tree=True, export_tree="tree.txt", silent=True)

# Deep tree, no terminal output
import sys, io
sys.stdout = io.StringIO()
anscom.scan("/mnt/volume", show_tree=True, export_tree="tree.txt", max_depth=64)
sys.stdout = sys.__stdout__

Output format:

  |-- [src]            ← [brackets] = directory
  |   |   |-- main.py  ← no brackets = regular file
  |   |   |-- [lib]
  |   |   |   |   |-- utils.py
  |-- config.json
  |-- [tests]
  |   |   |-- test_core.py

• One " | " block per depth level (6 chars each)
• At depth 64: 384 characters of indentation — all structurally valid
• DFS order is strict: every file inside a directory appears before that directory's sibling
• workers is forced to 1 — required for correct ordering
• No internal buffer — safe at 50+ million entries

---

Exclusion Filter

ignore_junk=True skips these directory names at any depth, at any nesting level:

.git         .svn         .hg          .idea        .vscode
node_modules bower_components site-packages .venv   venv
env          build         dist          target      __pycache__
temp         tmp           .cache        .pytest_cache .mypy_cache

The check is case-insensitive basename comparison — not a path substring match. A node_modules at /project/frontend/node_modules/ is caught regardless of nesting depth.

---

Report Format

Printed to sys.stdout when silent=False (the default).

anscom v1.5.3 (Threads: 16)
Target: /data

Scanned files: 21008 ...

=== SUMMARY REPORT ================================
+-----------------+--------------+----------+
| Category        | Count        | Percent  |
+-----------------+--------------+----------+
| Code/Source     |         5955 |   28.34% |
| System/Config   |         5707 |   27.16% |
| Other/Unknown   |         8992 |   42.81% |
| Documents       |          203 |    0.97% |
| Images          |          151 |    0.72% |
+-----------------+--------------+----------+
| TOTAL FILES     |        21008 |  100.00% |
+-----------------+--------------+----------+

=== DETAILED EXTENSION BREAKDOWN ==================
+-----------------+--------------+
| Extension       | Count        |
+-----------------+--------------+
| .py             |         5955 |
| .pyc            |         5707 |
| .mp3            |          730 |
| .txt            |          160 |
| .png            |          151 |
+-----------------+--------------+

Time     : 1.5186 seconds
Errors   : 0 (permission denied / inaccessible)
===================================================

=== TOP 20 LARGEST FILES ===========================   ← only with largest_n > 0
  1073741824 bytes : /data/backup/full.tar.gz
...

=== DUPLICATES SUMMARY ============================   ← only with find_duplicates=True
Groups found : 142
===================================================

Capture programmatically:

import io, sys
buf = io.StringIO()
sys.stdout = buf
anscom.scan("/data")
sys.stdout = sys.__stdout__
report_text = buf.getvalue()

---

File Categories and Extensions

170+ extensions across 9 categories. The table is sorted lexicographically and validated at module init — if the sort invariant is violated, import anscom raises RuntimeError.

Category	Sample Extensions
Code/Source	c cpp cs go h html java js json jsx kt lua php py r rb rs sh sql swift ts vue xml yaml yml
Documents	csv doc docx epub md mobi odp ods odt pdf ppt pptx rst rtf txt xls xlsx
Images	ai avif bmp gif heic ico jpeg jpg png psd raw svg tiff webp
Videos	avi flv mkv mov mp4 mpeg ogv webm wmv
Audio	aac flac m4a mid mp3 ogg wav wma
Archives	7z bz2 deb dmg gz iso jar rar tar tgz zip
Executables	app bin class dll elf exe msi pyd so
System/Config	bak cfg conf db env gitignore ini log pyc reg sys tmp ttf woff
Other/Unknown	Any extension not in the above table

---

Architecture

OS backends

Three separate scanning implementations compiled and selected at build time:

Platform	Backend	Mechanism
Linux	getdents64	Direct syscall(SYS_getdents64, dirfd, buf, 131072)— raw kernel ABI, 128KB read buffer,d_typefor zero-stat type detection
Windows	FindFirstFileW	Wide-char wchar_tpaths, UTF-16→UTF-8 conversion, size+mtime from WIN32_FIND_DATAWat no extra syscall cost
macOS / BSD	POSIX readdir	opendir/readdirwith lstatfor type resolution

Thread model

main thread
  ├── spawn N worker threads (all waiting on cond var)
  ├── spawn 1 progress thread
  ├── push root path to queue
  ├── wait until queue.count == 0 && active_workers == 0
  └── join all threads → merge stats

worker thread (×N)
  └── loop: queue_pop → process_dir_recursive → queue_task_done

process_dir_recursive
  ├── depth < 3: push subdirs to queue (parallel pickup by idle threads)
  └── depth ≥ 3: recurse inline (avoids queue overhead for deep narrow trees)

Per-thread stats — zero locks during counting

Each thread has its own ScanStats struct with ext_counts[170+] and cat_counts[9]. No lock is acquired during file categorization. The only shared atomic write per file is a single __sync_fetch_and_add for the progress counter. Stats are merged in one serial pass after all threads join.

Slab path allocator

Each thread allocates (max_depth + 2) * PATH_MAX bytes once before scanning. Path strings during traversal are written into slab[depth * PATH_MAX] via snprintf. Zero heap allocation during traversal.

Extension hash table

512-slot open-addressing hash table with FNV-1a hash and linear probing. Built once at module init from the sorted extension table. O(1) average lookup, no heap allocation, never modified after init.

FileArray pre-allocation

When return_files, export_csv, or find_duplicates is enabled, each thread pre-allocates a FileInfo array of 65,536 entries before scanning begins. Growth beyond that doubles via realloc. For typical filesystems: zero reallocations during the scan.

Byte accounting

Each thread carries total_size, cat_sizes[9] and ext_sizes[170+] in the same ScanStats struct as the counts, accumulated with no lock. They are merged in their own serial pass after the threads join. The accumulation sits after every filter's early return, so bytes and counts always describe the same set of files. With with_size=False the fields stay zero and the accumulation branch is never entered.

Min-heap for largest_n

Each thread maintains a min-heap of capacity N. Per-file cost: O(log N) comparison, no lock. Thread heaps merged globally after join using the same push logic.

Two-phase duplicate detection

1. qsort all files by size — O(M log M), no I/O
2. For each same-size group ≥2 members: read first 4KB of each, compute CRC32, sort by CRC32, group consecutive matches

Zero I/O for unique-size files. One bounded read per candidate. CRC32 computed using a fully inlined lookup table — no external library.

---

Security and Compliance

Property	Guarantee
No file contents read	Only directory entries and metadata. Exception:find_duplicates=Truereads up to 4KB per candidate — bounded, opt-in, read-only
Symlinks never followed	Linux:fstatat(AT_SYMLINK_NOFOLLOW). POSIX:lstat. Windows:FILE_ATTRIBUTE_REPARSE_POINTskipped unconditionally
Depth hard-capped at 64	Enforced in C at the top of every process_dir_recursivecall — cannot be bypassed by filesystem topology
All path assembly bounded	snprintf(slab, PATH_MAX, ...)— always null-terminated, always within PATH_MAXbytes
Errors counted, not silenced	Every failed opendir/open/FindFirstFileWincrements scan_errorsand continues — the final count is exact
Work queue bounded	131,072 fixed slots. Overflow falls back to inline recursion — no unbounded allocation
Hash table immutable after init	Built once at module load. No runtime modification
Zero external dependencies	No mandatory third-party packages — no supply chain surface

---

Recipes

Storage cost allocation

import anscom

result = anscom.scan("/mnt/nas", workers=16, ignore_junk=True, silent=True)
total = result["total_files"]
cats  = result["categories"]

media = cats["Videos"] + cats["Images"] + cats["Audio"]
code  = cats["Code/Source"]
docs  = cats["Documents"]

print(f"Media   : {media:>10,}  ({media/total*100:5.1f}%)")
print(f"Code    : {code:>10,}  ({code/total*100:5.1f}%)")
print(f"Docs    : {docs:>10,}  ({docs/total*100:5.1f}%)")
print(f"Total   : {total:>10,}  in {result['duration_seconds']:.2f}s")

Pre-migration audit

import anscom

result = anscom.scan(
    "/legacy-server/data",
    max_depth    = 30,
    silent       = True,
    return_files = True,
    export_json  = "audit.json",
    export_csv   = "inventory.csv"
)

print(f"Recorded {result['total_files']:,} files")
print(f"Errors  : {result['scan_errors']}")

CI/CD policy gate

import anscom, sys

result = anscom.scan("./repo", silent=True, ignore_junk=True)

violations = []
if result["categories"]["Executables"] > 0:
    violations.append(f"{result['categories']['Executables']} executable files")
if result["categories"]["Videos"] > 0:
    violations.append(f"{result['categories']['Videos']} video files")

if violations:
    for v in violations:
        print(f"POLICY VIOLATION: {v}")
    sys.exit(1)

print("File composition check passed.")

Storage reclamation

import anscom

result = anscom.scan(
    "/mnt/media-archive",
    find_duplicates = True,
    return_files    = True,
    workers         = 16,
    silent          = True
)

size_map = {f["path"]: f["size"] for f in result["files"]}
wasted   = sum(
    sum(size_map.get(p, 0) for p in group[1:])
    for group in result["duplicates"]
)

print(f"Duplicate groups : {len(result['duplicates'])}")
print(f"Reclaimable      : {wasted / (1024**3):.2f} GB")

groups_by_waste = sorted(
    result["duplicates"],
    key=lambda g: sum(size_map.get(p, 0) for p in g[1:]),
    reverse=True
)
for group in groups_by_waste[:5]:
    waste = sum(size_map.get(p, 0) for p in group[1:])
    print(f"\n  {waste / (1024**2):.1f} MB wasted:")
    for path in group:
        print(f"    {path}")

Top-100 largest files

import anscom

result = anscom.scan("/mnt/storage", largest_n=100, workers=32, silent=True)

total_gb = sum(f["size"] for f in result["largest_files"]) / (1024**3)
print(f"Top 100 total: {total_gb:.1f} GB\n")

for i, f in enumerate(result["largest_files"][:10], 1):
    print(f"{i:3}. {f['size']/1024**3:8.2f} GB  {f['path']}")

Regex scan — test files only

import anscom
from collections import Counter
import os

result = anscom.scan(
    "/codebase",
    regex_filter = r"/tests?/.*\.py$",
    return_files = True,
    silent       = True
)

print(f"Test files: {result['total_files']}")

dirs = Counter(os.path.dirname(f["path"]) for f in result["files"])
for d, count in dirs.most_common(10):
    print(f"  {count:4d}  {d}")

Live Prometheus push

import anscom
from prometheus_client import Gauge, start_http_server

start_http_server(9090)
g_progress = Gauge("anscom_files_scanned",   "Files scanned so far")
g_total    = Gauge("anscom_total_files",      "Total files found")
g_duration = Gauge("anscom_duration_seconds", "Scan duration")

result = anscom.scan(
    "/data-lake",
    callback = lambda n: g_progress.set(n),
    silent   = True,
    workers  = 32
)

g_total.set(result["total_files"])
g_duration.set(result["duration_seconds"])

Where the disk actually went

import anscom

r = anscom.scan("/mnt/nas", with_size=True, ignore_junk=True,
                workers=16, silent=True)

def human(n):
    for unit in ("B", "KB", "MB", "GB", "TB"):
        if n < 1024 or unit == "TB":
            return f"{n:.1f} {unit}"
        n /= 1024

print(f"{r['total_files']:,} files, {human(r['total_size'])} total\n")

rows = sorted(r["category_sizes"].items(), key=lambda kv: kv[1], reverse=True)
for cat, size in rows:
    if not size:
        continue
    count = r["categories"][cat]
    share = size / r["total_size"] * 100
    print(f"{cat:16s} {count:>9,} files  {human(size):>10s}  {share:5.1f}%")

print("\nTop extensions by size:")
top = sorted(r["extension_sizes"].items(), key=lambda kv: kv[1], reverse=True)
for ext, size in top[:10]:
    print(f"  .{ext:<10s} {human(size):>10s}")

Full audit — everything at once

import anscom

result = anscom.scan(
    "/mnt/storage",
    max_depth       = 20,
    workers         = 32,
    ignore_junk     = True,
    silent          = True,
    largest_n       = 50,
    find_duplicates = True,
    return_files    = True,
    with_size       = True,
    export_json     = "audit.json",
    export_csv      = "inventory.csv",
    show_tree       = True,
    export_tree     = "tree.txt",
)

print(f"Files        : {result['total_files']:,}")
print(f"Total size   : {result['total_size'] / 1024**3:.2f} GB")
print(f"Duration     : {result['duration_seconds']:.3f}s")
print(f"Dup groups   : {len(result['duplicates'])}")
print(f"Largest file : {result['largest_files'][0]['path']}")
print("Written      : audit.json  inventory.csv  tree.txt")

---

Changelog

v1.5.3 (current)

• Added with_size — opt-in byte accounting. Result dict gains total_size, category_sizes and extension_sizes
• Added the three size keys flow into export_json automatically — no new export parameter
• Improved on Linux the extra stat() is issued only when with_size=True and no other feature has already fetched the size; the count-only hot path is unchanged
• Unchanged export_csv still carries the per-file size column and no aggregate rows, so the output stays a clean rectangular table
• Unchanged the printed report — byte totals are returned as data, not printed

v1.5.2

• Added about() — prints the anscom mascot Chuhi (the house mouse of a Kolkata apartment) as ASCII art plus a short credit line, then returns None. No scan behavior changed.

v1.5.1

• Added pwd() — returns the current working directory as an absolute string (Unix pwd), as a value, never printed
• Added no-argument scan() — path is now optional and defaults to the current working directory
• Added scanned_path — every result dict reports the absolute path that was actually walked (resolved via realpath / _fullpath)
• Fixed Python reference leaks in result-dict construction — PyDict_SetItemString and PyList_Append do not steal references, so inline temporaries are now released (notably one set per file under return_files)
• Improved the per-file array is no longer pre-allocated on count-only scans — a plain scan() now performs zero heap work on the hot path

v1.5.0

• Added return_files — per-file list in result dict with path, size, ext, category, mtime
• Added export_csv — per-file inventory as UTF-8 CSV, zero dependencies, RFC 4180-compliant quoting
• Added largest_n — top-N files by size using per-thread min-heap, O(log N) per file
• Added find_duplicates — size-bucket + CRC32 duplicate detection, zero I/O for unique-size files
• Added regex_filter — path pattern filter; POSIX regexec on Linux/macOS (no GIL), Python re fallback on Windows
• Added FILEARRAY_INIT_CAP (65536) pre-allocation per thread — zero reallocations for typical scans
• Fixed fstatat on Linux called only when needed — two separate guards for type resolution vs. size/mtime collection
• Fixed sorted_top paths are strdup'd independently from global_heap — no lifetime overlap, no double-free
• Removed export_excel — was crashing on Windows due to openpyxl Workbook.read_only exception; use export_csv + pandas.to_excel() instead
• Improved Full docstring on anscom.scan accessible via help(anscom.scan)

v1.3.0

• Added export_json, export_excel, export_tree
• Fixed DFS tree output ordering
• Added file tracking in tree mode

v1.2.0 and earlier

• Multi-threaded worker pool with condition-variable termination detection
• getdents64 direct syscall on Linux
• Per-thread statistics, zero shared state during scan
• ignore_junk, min_size, extensions, callback, silent

---

License

MIT License. Free for personal and commercial use.