Skip to main content

Threaded directory iteration via os.scandir() with progress indicator and resume function.

Project description

IterFilesystem

Threaded directory iteration via os.scandir() with progress indicator and resume function.

Requirement:

  • Python 3.6 or newer.

  • Pipenv. Packages and virtual environment manager.

Please: try, fork and contribute! ;)

Build Status on travis-ci.org

travis-ci.org/jedie/IterFilesystem

Build Status on appveyor.com

ci.appveyor.com/project/jedie/IterFilesystem

Coverage Status on coveralls.io

coveralls.io/r/jedie/IterFilesystem

Requirements Status on requires.io

requires.io/github/jedie/IterFilesystem/requirements/

Example

Use example CLI, e.g.:

~$ git clone https://github.com/jedie/IterFilesystem.git
~$ cd IterFilesystem
~/IterFilesystem$ pipenv install
~/IterFilesystem$ pipenv shell
(IterFilesystem) ~/IterFilesystem$ print_fs_stats --help
usage: print_fs_stats.py [-h] [-v] [--path PATH]
                         [--skip_dirs [SKIP_DIRS [SKIP_DIRS ...]]]
                         [--skip_filenames [SKIP_FILENAMES [SKIP_FILENAMES ...]]]
                         [--force_restart] [--complete_cleanup]

Scan filesystem and print some information

optional arguments:
  -h, --help            show this help message and exit
  -v, --version         show program's version number and exit
  --path PATH           The file path that should be scanned e.g.: "~/foobar/"
                        default is "~"
  --skip_dirs [SKIP_DIRS [SKIP_DIRS ...]]
                        Directory names to exclude from scan.
  --skip_filenames [SKIP_FILENAMES [SKIP_FILENAMES ...]]
                        File names to ignore.
  --force_restart       Don't use resume data: Delete persist queue data
                        before start.
  --complete_cleanup    Delete persist queue after complete scan.

example output looks like this:

(IterFilesystem) ~/IterFilesystem$ $ print_fs_stats --path ~/IterFilesystem --skip_dirs .tox .pytest_cache
Read/process: '/home/foobar/IterFilesystem'...
Skip directories:
    * .tox
    * .pytest_cache

[INFO:  61 2019-09-15 12:37:17,973] Save persist queue to: /home/foobar/IterFilesystem/persisted_queue_4Aubn
[INFO:  37 2019-09-15 12:37:18,252] Skip dir: '.idea'
[INFO:  37 2019-09-15 12:37:18,252] Skip dir: '.tox'
[INFO:  37 2019-09-15 12:37:18,270] Skip dir: '.pytest_cache'
Read filesystem with 83 items in 0.3sec.
*** all items processed ***: 100%|████████████████████████████████████| 83/83 [00:00<00:00, 85.25it/s]

total filesystem items: 83
Finish in.............: 1.0sec.
File count.....: 73 with 10991860 Bytes
Directory count: 10
Other count....: 0

History

Donating

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

iterfilesystem-0.0.1.tar.gz (11.2 kB view hashes)

Uploaded Source

Built Distributions

iterfilesystem-0.0.1-py3.6.egg (11.3 kB view hashes)

Uploaded Source

iterfilesystem-0.0.1-py2.py3-none-any.whl (13.3 kB view hashes)

Uploaded Python 2 Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page