Python getdents

Iterate large directories efficiently with python.

About

python-getdents is a simple wrapper around Linux system call getdents64 (see man getdents for details).

Implementation is based on solution descibed in You can list a directory containing 8 million files! But not with ls. article by Ben Congleton.

Install

pip install getdents

For development

python3 -m venv env
. env/bin/activate
pip install -e .[test]

Building Wheels

pip install cibuildwheel
cibuildwheel --platform linux --output-dir wheelhouse

Run tests

ulimit -v 33554432 && py.test tests/

Usage

from getdents import getdents

for inode, type_, name in getdents("/tmp"):
    print(name)

Advanced

While getdents provides a convenient wrapper with ls-like filtering, you can use getdents_raw for more control:

import os
from getdents import DT_LNK, O_GETDENTS, getdents_raw

fd = os.open("/tmp", O_GETDENTS)

for inode, type_, name in getdents_raw(fd, 2**20):
    if type_ == DT_LNK and inode != 0:
        print("found symlink:", name, "->", os.readlink(name, dir_fd=fd))

os.close(fd)

Batching

In case you need more control over syscalls, you may call instance of getdents_raw instead. Each call corresponds to single getdents64 syscall, returning list of hovever many entries fits in buffer size. Call returns None when there are no more entries to read.

it = getdents_raw(fd, 2**20)

for batch in iter(it, None):
     for inode, type, name in batch:
        ...

Free-threading

While it is not so wise idea to do an I/O from multiple threads on a single file descriptor, you can do it if you need to. This package supports free-threading (nogil) in Python.

CLI

Usage

python-getdents [-h] [-b N] [-o NAME] PATH

Options

Option	Description
-b N	Buffer size (in bytes) to allocate when iterating over directory. Default is 32768, the same value used by glibc, you probably want to increase this value. Try starting with 16777216 (16 MiB). Best performance is achieved when buffer size rounds to size of the file system block.
--buffer-size N
-o NAME	Output format: plain (default) Print only names. csv Print as comma-separated values in order: inode, type, name. csv-headers Same as csv, but print headers on the first line also. json output as JSON array. json-stream output each directory entry as single json object separated by newline.
--output-format NAME

Exit codes

3 - Requested buffer is too large
4 - PATH not found.
5 - PATH is not a directory.
6 - Not enough permissions to read contents of the PATH.

Examples

python-getdents /path/to/large/dir
python -m getdents /path/to/large/dir
python-getdents /path/to/large/dir -o csv -b 16777216 > dir.csv

Release files for getdents 1.0.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for getdents 1.0.0
File	Size	Uploaded
getdents-1.0.0.tar.gz	13.8 kB	Sep 2, 2025	Details

Built distributions (wheels)

Table of built distributions (wheels) for getdents 1.0.0
File	Interpreter	ABI	Platform	Reset
getdents-1.0.0-cp314-cp314t-musllinux_1_2_x86_64.whl 16.2 kB Sep 2, 2025	CPython 3.14	CPython 3.14 free-threading	Linux musl 1.2+ x86-64	Details
getdents-1.0.0-cp314-cp314t-musllinux_1_2_aarch64.whl 17.0 kB Sep 2, 2025	CPython 3.14	CPython 3.14 free-threading	Linux musl 1.2+ ARM64	Details
getdents-1.0.0-cp314-cp314t-manylinux_2_28_x86_64.whl 15.9 kB Sep 2, 2025	CPython 3.14	CPython 3.14 free-threading	Linux glibc 2.28+ x86-64	Details
getdents-1.0.0-cp314-cp314t-manylinux_2_28_aarch64.whl 17.2 kB Sep 2, 2025	CPython 3.14	CPython 3.14 free-threading	Linux glibc 2.28+ ARM64	Details
getdents-1.0.0-cp310-abi3-musllinux_1_2_x86_64.whl 15.5 kB Sep 2, 2025	CPython 3.10	abi3	Linux musl 1.2+ x86-64	Details
getdents-1.0.0-cp310-abi3-musllinux_1_2_aarch64.whl 16.2 kB Sep 2, 2025	CPython 3.10	abi3	Linux musl 1.2+ ARM64	Details
getdents-1.0.0-cp310-abi3-manylinux_2_28_x86_64.whl 15.2 kB Sep 2, 2025	CPython 3.10	abi3	Linux glibc 2.28+ x86-64	Details
getdents-1.0.0-cp310-abi3-manylinux_2_28_aarch64.whl 16.3 kB Sep 2, 2025	CPython 3.10	abi3	Linux glibc 2.28+ ARM64	Details

Total release size: 143.4 kB

Release files / getdents-1.0.0.tar.gz

Download URL	getdents-1.0.0.tar.gz
Size	13.8 kB
Tags	Source
SHA-256 checksum How to use checksums	`80ab2825a09e5b1107fe3d166458d01d4a7cedfe255ee9762d12c68c9f890d24`
BLAKE2b-256 checksum How to use checksums	`10aacbdc87f71e8659f579557beb5d719e82459f70cdac6c089f948bce6cd76a`
Upload date	Sep 2, 2025
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/6.1.0 CPython/3.12.9`

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/ZipFile/python-getdents
Publishing commit: github.com/ZipFile/python-getdents/tree/d10a294cdf90a384a9f9661d40cf29d11e7c09c3

Workflow

Publishing configuration: github.com/ZipFile/python-getdents/blob/d10a294cdf90a384a9f9661d40cf29d11e7c09c3/.github/workflows/publish.yml
Publishing logs: github.com/ZipFile/python-getdents/actions/runs/17412564198/attempts/1

Artifact

Subject: getdents-1.0.0.tar.gz
SHA-256 checksum: 80ab2825a09e5b1107fe3d166458d01d4a7cedfe255ee9762d12c68c9f890d24 Verifying attestations

Release files / getdents-1.0.0-cp314-cp314t-musllinux_1_2_x86_64.whl

Download URL	getdents-1.0.0-cp314-cp314t-musllinux_1_2_x86_64.whl
Size	16.2 kB
Tags	CPython 3.14 CPython 3.14 free-threading Linux musl 1.2+ x86-64
SHA-256 checksum How to use checksums	`35238c0e4fa94b266099abd00391ce1716d439cd4b127427ac385bc49fa230cd`
BLAKE2b-256 checksum How to use checksums	`b8c26bf256ec5358ab95608f0f1a9671f7e441bfb9045ce046f8396d5be4d609`
Upload date	Sep 2, 2025
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/6.1.0 CPython/3.12.9`

Provenance

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/ZipFile/python-getdents
Publishing commit: github.com/ZipFile/python-getdents/tree/d10a294cdf90a384a9f9661d40cf29d11e7c09c3

Workflow

Publishing configuration: github.com/ZipFile/python-getdents/blob/d10a294cdf90a384a9f9661d40cf29d11e7c09c3/.github/workflows/publish.yml
Publishing logs: github.com/ZipFile/python-getdents/actions/runs/17412564198/attempts/1

Artifact

Subject: getdents-1.0.0-cp314-cp314t-musllinux_1_2_x86_64.whl
SHA-256 checksum: 35238c0e4fa94b266099abd00391ce1716d439cd4b127427ac385bc49fa230cd Verifying attestations

Release files / getdents-1.0.0-cp314-cp314t-musllinux_1_2_aarch64.whl

Download URL	getdents-1.0.0-cp314-cp314t-musllinux_1_2_aarch64.whl
Size	17.0 kB
Tags	CPython 3.14 CPython 3.14 free-threading Linux musl 1.2+ ARM64
SHA-256 checksum How to use checksums	`381f0081be3bdd249f121e51b13b477c615b516be31d08b8bf6b839ea968b48e`
BLAKE2b-256 checksum How to use checksums	`d361e7304e86899d2b8181ab23035e54b0322f70c019439ece5b83b0cf1888bf`
Upload date	Sep 2, 2025
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/6.1.0 CPython/3.12.9`

Provenance

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/ZipFile/python-getdents
Publishing commit: github.com/ZipFile/python-getdents/tree/d10a294cdf90a384a9f9661d40cf29d11e7c09c3

Workflow

Publishing configuration: github.com/ZipFile/python-getdents/blob/d10a294cdf90a384a9f9661d40cf29d11e7c09c3/.github/workflows/publish.yml
Publishing logs: github.com/ZipFile/python-getdents/actions/runs/17412564198/attempts/1

Artifact

Subject: getdents-1.0.0-cp314-cp314t-musllinux_1_2_aarch64.whl
SHA-256 checksum: 381f0081be3bdd249f121e51b13b477c615b516be31d08b8bf6b839ea968b48e Verifying attestations

Release files / getdents-1.0.0-cp314-cp314t-manylinux_2_28_x86_64.whl

Download URL	getdents-1.0.0-cp314-cp314t-manylinux_2_28_x86_64.whl
Size	15.9 kB
Tags	CPython 3.14 CPython 3.14 free-threading Linux glibc 2.28+ x86-64
SHA-256 checksum How to use checksums	`2ce612bdc9cc3690dc568b360bafee319afca151b9abb38fea376e7ddd344085`
BLAKE2b-256 checksum How to use checksums	`5a8a4055f0eeb93b7a9251ff720dc1c4d5352cac7b36a975ffaf072073c105a4`
Upload date	Sep 2, 2025
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/6.1.0 CPython/3.12.9`

Provenance

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/ZipFile/python-getdents
Publishing commit: github.com/ZipFile/python-getdents/tree/d10a294cdf90a384a9f9661d40cf29d11e7c09c3

Workflow

Publishing configuration: github.com/ZipFile/python-getdents/blob/d10a294cdf90a384a9f9661d40cf29d11e7c09c3/.github/workflows/publish.yml
Publishing logs: github.com/ZipFile/python-getdents/actions/runs/17412564198/attempts/1

Artifact

Subject: getdents-1.0.0-cp314-cp314t-manylinux_2_28_x86_64.whl
SHA-256 checksum: 2ce612bdc9cc3690dc568b360bafee319afca151b9abb38fea376e7ddd344085 Verifying attestations

Release files / getdents-1.0.0-cp314-cp314t-manylinux_2_28_aarch64.whl

Download URL	getdents-1.0.0-cp314-cp314t-manylinux_2_28_aarch64.whl
Size	17.2 kB
Tags	CPython 3.14 CPython 3.14 free-threading Linux glibc 2.28+ ARM64
SHA-256 checksum How to use checksums	`7c6e461ec4d14e8ea668faab5e68467940dd0d8000f4c6d2f3f91832bddb0769`
BLAKE2b-256 checksum How to use checksums	`6ee4a4c3172a2e1d17621dd52884d48a5120673379ec478ff8e1c312124770fe`
Upload date	Sep 2, 2025
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/6.1.0 CPython/3.12.9`

Provenance

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/ZipFile/python-getdents
Publishing commit: github.com/ZipFile/python-getdents/tree/d10a294cdf90a384a9f9661d40cf29d11e7c09c3

Workflow

Publishing configuration: github.com/ZipFile/python-getdents/blob/d10a294cdf90a384a9f9661d40cf29d11e7c09c3/.github/workflows/publish.yml
Publishing logs: github.com/ZipFile/python-getdents/actions/runs/17412564198/attempts/1

Artifact

Subject: getdents-1.0.0-cp314-cp314t-manylinux_2_28_aarch64.whl
SHA-256 checksum: 7c6e461ec4d14e8ea668faab5e68467940dd0d8000f4c6d2f3f91832bddb0769 Verifying attestations

Release files / getdents-1.0.0-cp310-abi3-musllinux_1_2_x86_64.whl

Download URL	getdents-1.0.0-cp310-abi3-musllinux_1_2_x86_64.whl
Size	15.5 kB
Tags	CPython 3.10 Linux musl 1.2+ x86-64 abi3
SHA-256 checksum How to use checksums	`cdd21d302592fa4c2b4e983d30b07a0be8b41846ec1413d2ffd2034a287e25ce`
BLAKE2b-256 checksum How to use checksums	`a897c31cb9dafdba8edba3983e14fc063cd885c99c0a5d4d0da3c692d43b06c0`
Upload date	Sep 2, 2025
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/6.1.0 CPython/3.12.9`

Provenance

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/ZipFile/python-getdents
Publishing commit: github.com/ZipFile/python-getdents/tree/d10a294cdf90a384a9f9661d40cf29d11e7c09c3

Workflow

Publishing configuration: github.com/ZipFile/python-getdents/blob/d10a294cdf90a384a9f9661d40cf29d11e7c09c3/.github/workflows/publish.yml
Publishing logs: github.com/ZipFile/python-getdents/actions/runs/17412564198/attempts/1

Artifact

Subject: getdents-1.0.0-cp310-abi3-musllinux_1_2_x86_64.whl
SHA-256 checksum: cdd21d302592fa4c2b4e983d30b07a0be8b41846ec1413d2ffd2034a287e25ce Verifying attestations

Release files / getdents-1.0.0-cp310-abi3-musllinux_1_2_aarch64.whl

Download URL	getdents-1.0.0-cp310-abi3-musllinux_1_2_aarch64.whl
Size	16.2 kB
Tags	CPython 3.10 Linux musl 1.2+ ARM64 abi3
SHA-256 checksum How to use checksums	`8f992fa25380d76f88cb89cd582b1cb5b64e9bb1142cb26776ccf3b40044f7f4`
BLAKE2b-256 checksum How to use checksums	`3a05a9d854115b73022e681cfa916e24cf70d38eaf19bc57a592097fa99b601f`
Upload date	Sep 2, 2025
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/6.1.0 CPython/3.12.9`

Provenance

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/ZipFile/python-getdents
Publishing commit: github.com/ZipFile/python-getdents/tree/d10a294cdf90a384a9f9661d40cf29d11e7c09c3

Workflow

Publishing configuration: github.com/ZipFile/python-getdents/blob/d10a294cdf90a384a9f9661d40cf29d11e7c09c3/.github/workflows/publish.yml
Publishing logs: github.com/ZipFile/python-getdents/actions/runs/17412564198/attempts/1

Artifact

Subject: getdents-1.0.0-cp310-abi3-musllinux_1_2_aarch64.whl
SHA-256 checksum: 8f992fa25380d76f88cb89cd582b1cb5b64e9bb1142cb26776ccf3b40044f7f4 Verifying attestations

Release files / getdents-1.0.0-cp310-abi3-manylinux_2_28_x86_64.whl

Download URL	getdents-1.0.0-cp310-abi3-manylinux_2_28_x86_64.whl
Size	15.2 kB
Tags	CPython 3.10 Linux glibc 2.28+ x86-64 abi3
SHA-256 checksum How to use checksums	`49cd092b360b52a40802ef6fa08e50346ad36dd67f63f05781501f957ca21ab0`
BLAKE2b-256 checksum How to use checksums	`8a2c4765c59e3349d60856c77215b32f5fe6b6cb2cc1675639d359fd12424a30`
Upload date	Sep 2, 2025
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/6.1.0 CPython/3.12.9`

Provenance

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/ZipFile/python-getdents
Publishing commit: github.com/ZipFile/python-getdents/tree/d10a294cdf90a384a9f9661d40cf29d11e7c09c3

Workflow

Publishing configuration: github.com/ZipFile/python-getdents/blob/d10a294cdf90a384a9f9661d40cf29d11e7c09c3/.github/workflows/publish.yml
Publishing logs: github.com/ZipFile/python-getdents/actions/runs/17412564198/attempts/1

Artifact

Subject: getdents-1.0.0-cp310-abi3-manylinux_2_28_x86_64.whl
SHA-256 checksum: 49cd092b360b52a40802ef6fa08e50346ad36dd67f63f05781501f957ca21ab0 Verifying attestations

Release files / getdents-1.0.0-cp310-abi3-manylinux_2_28_aarch64.whl

Download URL	getdents-1.0.0-cp310-abi3-manylinux_2_28_aarch64.whl
Size	16.3 kB
Tags	CPython 3.10 Linux glibc 2.28+ ARM64 abi3
SHA-256 checksum How to use checksums	`f62d1edd1522fd044439589c4e8c200b94a677d81ae3b86320eff8e3cd8ccb10`
BLAKE2b-256 checksum How to use checksums	`a192a28176f225841e06fd8c27c37951b045df648f89b8f2f04c65be430aef73`
Upload date	Sep 2, 2025
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/6.1.0 CPython/3.12.9`

Provenance

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/ZipFile/python-getdents
Publishing commit: github.com/ZipFile/python-getdents/tree/d10a294cdf90a384a9f9661d40cf29d11e7c09c3

Workflow

Publishing configuration: github.com/ZipFile/python-getdents/blob/d10a294cdf90a384a9f9661d40cf29d11e7c09c3/.github/workflows/publish.yml
Publishing logs: github.com/ZipFile/python-getdents/actions/runs/17412564198/attempts/1

Artifact

Subject: getdents-1.0.0-cp310-abi3-manylinux_2_28_aarch64.whl
SHA-256 checksum: f62d1edd1522fd044439589c4e8c200b94a677d81ae3b86320eff8e3cd8ccb10 Verifying attestations