python based easy file iterator, with recursive option, and file filtering. useful when you need to apply some function to a set of files.
- efipy stands for 'easy file iterator python'
python based easy file iterator, with recursive option, file filtering, and concurrent (multithreading) capabilities. useful when you need to apply some function to a set of files. also, could be described as a glob wrapper with UI, for example can inquire path with file completion capabilities, shows progress bar, and has increased flexibility.
I find this module is useful for automating day to day file iteration tasks, mostly on Windows where bash language is orders of magnitude weaker than python. so instead of writing some half cooked file iterator every time I need to get something done, I thought I would make this. also, after making several projects that needed a file iterator with a nice UI, and after deciding that copying and pasting this code is tedious and unhealthy, I decided to make this thing a project and upload it ot pypi.
pip install efipy
(pypi should download all of these automatically upon install)
- safer-prompt-toolkit - (optional)
- tqdm - (optional)
run(func, root_path=None, b_recursive=False, files_filter="*", b_yield_folders=False)
run func on all files that match.
- func - a callable, the function to be executed for each matching file in directories.
func receives a single parameter of type pathlib.Path and returns nothing.
- root_path - defaults to None. a directory, or a file in which to iterate. if file is given than runs only on that one file. if None is given, will prompt the user for a path, with path auto-completion and validation.
- b_recursive - defaults to False. if True, will search recursively in sub-folders. if False will limit search to current dir.
- files_filter - defaults to "*" (allows any path). a filter to limit search results for files, see "glob" for further details.
- b_yield_folders - defaults to False. weather to pass paths to folders (not files) to func as well (if you want to iterate on folders as well as files)
- number_of_threads - defaults to 1. the number of threads to be used in order to concurrently run on all files. select 1 in order to loop on files linearly.
- b_skip_errors - defaults to True. if True, then when error occurs while running func, prints it's traceback, and then proceeds to run func on the next path to be iterated.
- errors_log_file - defaults to None. if not None, prints error logs to the file at the path given. file is created & cleared when this function is called.
- b_progress_bar - if true uses tqdm to display progress of file iteration.
- a list of pathlib.Path instances that contains all the paths that matched the search (the exact same ones that were sent to func).
- if you don't like the fact that func receives a complicated pathlib.Path instance, you can just write
- take advantage of the root_path=None option. when no root path is supplied, the computer will prompt you for a path with some nice UI.
def func(path): path=str(path) # ... other code to do with path ... efipy.run(func)
prompts the user for an output path, and returns it. if path already exists, asks the user to confirm overwrite. also has path auto-completion and validation.
- default - the default path to prompt user with
- the validated path the user entered
lets say I have a folder in which all the files are numbered. let's say I want to rename every third file, so that it ends wit a q, for example:
before: -> after: file_one_000.txt some_other_file_001.md some_third_file_002.txt -> some_third_file_002q.txt some_fourth_file_003.txt yea_dogs_004.txt wow_005.log -> wow_005q.log
import os,efipy def rename(path): # rename files that end with divide by 3 if int(path.stem[-3:])%3 == 0: name_no_extention,extention = os.path.splitext(path) os.rename(path,name_no_extention + "q" + extention) efipy.run(rename)
import efipy def func(path): print(str(path)) efipy.run(func,root_path="some_root_folder",b_recursive=True,files_filter="*.q") # when run on the following file tree, will produce the following output: # file tree: # ─── some_root_folder # ├── f0.q # ├── dir1 # │ ├── f1.r # │ └── f2.q # └── dir2 # ├── f3.q # └── f4.q # output, the program will print out (maybe not in this order): # some_root_folder/f0.q # some_root_folder/dir1/f2.q # some_root_folder/dir2/f3.q # some_root_folder/dir2/f4.q
tip for easy usage
if you open ipython, you can easily get the docstring for any function by writing '?' after it, for example writing
efipy.run? will supply the docstring of efipy.run
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.