A fast file tree scanner written in Rust
Project description
scandir-rs
The Python module is called scandir_rs
and installable via pip
. It is an alternative to os.walk()
and os.scandir()
with more features and higher speed. On Linux it is 3 - 11 times faster and on Windows 6 - 70 time faster (see benchmarks).
It releases the GIL and the scanning is done in a background thread. With different methods intermediate results can be read.
If you are just interested in directory statistics you can use the Count
.
scandir_rs
contains following classes:
Count
for determining statistics of a directory.Walk
for getting names of directory entries.Scandir
for getting detailed stats of directory entries.
For the API see:
Installation
For building this wheel from source you need the tool maturin
.
Install maturin
:
cargo install maturin
IMPORTANT: In order to build this project at least Rust version 1.61 is needed!
Build wheel:
Change to directory pyscandir
.
Build wheel (on Linux):
maturin build --release --strip
Build wheel on Windows:
maturin build --release --strip --no-sdist
maturin
will build the wheels for all Python versions installed on your system.
Alternatively you can use the build script build_wheels.py
. The precondition to run this script is to have pyenv
installed.
The script can build the wheel for specific Python versions or for all Python versions installed by pyenv
.
In addition it runs pytest
after successfull creation of each wheel.
python build_wheels.py
By default the script will build the wheel for the current Python interpreter.
If you want to build the wheel for specific Python version(s) by providing the argument --versions
.
python build_wheels.py --versions 3.11.8,3.12.2
To build the wheel for all installed Python versions:
python build_wheels.py --versions *
Instruction how to install pyenv
can be found here.
Examples
Get statistics of a directory:
from scandir_rs import Count, ReturnType
print(Count("/usr", return_type=ReturnType.Ext).collect())
The collect
method releases the GIL. So other Python threads can run in parallel.
The same, but asynchronously in background using a class instance:
from scandir_rs import Count, ReturnType
instance = Count("/usr", return_type=ReturnType.Ext)
instance.start() # Start scanning the directory in background
...
values = instance.results() # Returns the current statistics. Can be read at any time
...
if instance.busy(): # Check if the task is still running.
...
instance.stop() # If you want to cancel the task
...
instance.join() # Wait for the instance to finish.
and with a context manager:
import time
from scandir_rs import Count, ReturnType
with Count("/usr", return_type=ReturnType.Ext) as instance:
while instance.busy():
statistics = instance.results()
# Do something
time.sleep(0.01)
print(instance.results())
os.walk()
example:
from scandir_rs import Walk
for root, dirs, files in Walk("/usr"):
# Do something
with extended data:
from scandir_rs import Walk, ReturnType
for root, dirs, files, symlinks, other, errors in Walk("/usr", return_type=ReturnType.Ext):
# Do something
os.scandir()
example:
from scandir_rs import Scandir, ReturnType
for path, entry in Scandir("~/workspace", return_type=ReturnType.Ext):
# entry is a custom DirEntry object
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for scandir_rs-2.7.1-cp312-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 15ac0e27759d77031c3eb9ea4812314d983b95647881b8b8612921fe5f55a8e3 |
|
MD5 | 47ae8f7b0e3bc37e041570b083eb86b3 |
|
BLAKE2b-256 | 2a2b6d2542fe72f1705b40b8bb633c2e230823204bafc35e43fbaba1de8464d7 |
Hashes for scandir_rs-2.7.1-cp312-cp312-manylinux_2_34_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5fd55ef0f1de9454e8304abe4999294417dee92b53cef3c83bf2e0beb78aaf6c |
|
MD5 | f3de7e4bd81f327041a3a2a35d71d342 |
|
BLAKE2b-256 | 787bc93f05e4e76ebb7f265e352748a38261923e018548f46b0a08b1f0888753 |
Hashes for scandir_rs-2.7.1-cp311-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f5b290c6a01bfb1b32f75d7a8ee4e1ce94a4eb8a3a527e9df1bac4fd461d0227 |
|
MD5 | 18f6599b11df2a033a58b54f032cfb1b |
|
BLAKE2b-256 | b71176277c7537dba1753b5e376bf3f74707f6c7c3e277629afee78572cdcb0f |
Hashes for scandir_rs-2.7.1-cp311-cp311-manylinux_2_34_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c29606b82c17b7c7dec776e2db8bbfdbce5d0f21cec7d3e3cbd22b9e62fc5401 |
|
MD5 | c59944b659d39ac4b646031069109466 |
|
BLAKE2b-256 | 9876a42bd14a354d0bbce2ed30dde777dfcb2fd8c880507cb3914a9de3efe5fa |
Hashes for scandir_rs-2.7.1-cp310-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8b31ac380f4f941fe5b139b7577ac5c16eeb6cd9a8c0a8b40881cf827d4a6966 |
|
MD5 | acbedc7483e1fab046e75436957ddf1e |
|
BLAKE2b-256 | 53e3f5325fda4ccf035dc271f0076d665e419f7a544156f6d3c749ffd071aef1 |
Hashes for scandir_rs-2.7.1-cp310-cp310-manylinux_2_34_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 918e487d8004e36730d7cf78fc5d6b41aaf76bfd4a539ddf1f3b84a65b493090 |
|
MD5 | bc91efa63b48bd6fe44ce1cac4ee134a |
|
BLAKE2b-256 | b7bdab5ec6cdcf9540309d9edd100712d5130d6d0bb383312c7710c27e35641e |
Hashes for scandir_rs-2.7.1-cp39-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ee36a5c11150c8e3d05245c6ff38ac4a23122df77ba738a6ecce3740fa65c0fa |
|
MD5 | 9869f9d4bfac33f8af90b916a4c943f1 |
|
BLAKE2b-256 | 569a482927b024e50257d70ebf6dc525b55b0ec2d5415683795d29a3d418f727 |
Hashes for scandir_rs-2.7.1-cp39-cp39-manylinux_2_34_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2ea8ca7e20d7b10ff59b32042c4cbb9c0472ec47e344f99c437acdbab0e0886c |
|
MD5 | 718139c4bb5f73b5bff501114821b471 |
|
BLAKE2b-256 | 02d7c70b934394231592a67c198fde4650b7f03e992917c7b2044c347af090d8 |
Hashes for scandir_rs-2.7.1-cp38-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 830fcb682d804a30c186f622798996d4fc67f91d601899194a563a6d5fb3bbd8 |
|
MD5 | 5ed94e93e98f86a4e12a0ef5e6b67cfe |
|
BLAKE2b-256 | 97267fddb52a184eb5f3ccdf70bef997ef020d8b513a5fde56776d1e393f7b55 |
Hashes for scandir_rs-2.7.1-cp38-cp38-manylinux_2_34_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f9f6b8c2b51cc66479b1bc69fa54b2349535f056ccb3ee870a0a3e0e43388f96 |
|
MD5 | 2f2db89920288d22bd83cfecfdb73e4e |
|
BLAKE2b-256 | a0e2a601043272d1a5745273825638e96bcc62cf5817f93f0df822d7ce1ba953 |
Hashes for scandir_rs-2.7.1-cp37-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 683bbe911099b867ca495d5c793f602462725950bc211ab757dd97250931cabb |
|
MD5 | 34fd7cc1417ac5e040e16400267c9c75 |
|
BLAKE2b-256 | bea9b108c7da8c80f927d05b040b60ab6e9ddae147e60a41c068b8e85bbfe16e |
Hashes for scandir_rs-2.7.1-cp37-cp37m-manylinux_2_34_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9323d4672763108907b7d0d29ab02e7e0b615e8dc357edd2c438a43ac305509f |
|
MD5 | bbbc8be7d34b9886117474727ddd3cd1 |
|
BLAKE2b-256 | b64c54fa058ba7b14c135ea4aea264dae93e19741616de2388bba0e0c7660ebe |