Assorted filesystem related utility functions, some of which have been bloating cs.fileutils for too long.

Latest release 20241122:

• FSPathBasedSingleton: add a .promote method to promote a filesystem path to an instance.
• FSPathBasedSingleton: new fspath_normalised(fspath) class method to produce a normalised form of the fspath for use as the key to the singleton registry.

atomic_directory(*da, **dkw)

Decorator for a function which fills in a directory which calls the function against a temporary directory then renames the temporary to the target name on completion.

Parameters:

• infill_func: the function to fill in the target directory
• make_placeholder: optional flag, default False: if true an empty directory will be make at the target name and after completion it will be removed and the completed directory renamed to the target name

findup(dirpath: str, criterion: Union[str, Callable[[str], Any]]) -> str

Walk up the filesystem tree looking for a directory where criterion(fspath) is not None, where fspath starts at dirpath. Return the result of criterion(fspath). Return None if no such path is found.

Parameters:

• dirpath: the starting directory
• criterion: a str or a callable accepting a str

If criterion is a str, use look for the existence of os.path.join(fspath,criterion)

Example:

# find a directory containing a `.envrc` file
envrc_path = findup('.', '.envrc')

# find a Tagger rules file for the Downloads directory
rules_path = findup(expanduser('~/Downloads', '.taggerrc')

fnmatchdir(dirpath, fnglob)

Return a list of the names in dirpath matching the glob fnglob.

Class FSPathBasedSingleton(cs.obj.SingletonMixin, HasFSPath, cs.deco.Promotable)

The basis for a SingletonMixin based on realpath(self.fspath).

FSPathBasedSingleton.__init__(self, fspath: Optional[str] = None, lock=None): Initialise the singleton:

On the first call:

• set .fspath to self._resolve_fspath(fspath)
• set ._lock to lock (or cs.threads.NRLock() if not specified)

FSPathBasedSingleton.fspath_normalised(fspath: str): Return the normalised form of the filesystem path fspath, used as the key for the singleton registry.

This default returns realpath(fspath).

As a contracting example, the cs.ebooks.kindle.classic.KindleTree class tries to locate the directory containing the book database, and returns its realpath, allowing some imprecision.

FSPathBasedSingleton.promote(obj): Promote None or str to a CalibreTree.

Class HasFSPath

A mixin for an object with a .fspath attribute representing a filesystem location.

The __init__ method just sets the .fspath attribute, and need not be called if the main class takes care of that itself.

HasFSPath.fnmatch(self, fnglob): Return a list of the names in self.fspath matching the glob fnglob.

HasFSPath.listdir(self): Return os.listdir(self.fspath).

HasFSPath.pathto(self, *subpaths): The full path to subpaths, comprising a relative path below self.fspath. This is a shim for os.path.join which requires that all the subpaths be relative paths.

HasFSPath.shortpath: The short version of self.fspath.

is_valid_rpath(rpath, log=None) -> bool

Test that rpath is a clean relative path with no funny business.

This is a Boolean wrapper for validate_rpath().

longpath(path, prefixes=None)

Return path with prefixes and environment variables substituted. The converse of shortpath().

needdir(dirpath, mode=511, *, use_makedirs=False, log=None) -> bool

Create the directory dirpath if missing. Return True if the directory was made, False otherwise.

Parameters:

• dirpath: the required directory path
• mode: the permissions mode, default 0o777
• log: log makedirs or mkdir call
• use_makedirs: optional creation mode, default False; if true, use os.makedirs, otherwise os.mkdir

rpaths(dirpath='.', **scan_kw)

A shim for scandirtree to yield relative file paths from a directory.

Parameters:

• dirpath: optional top directory, default '.'

Other keyword arguments are passed to scandirtree.

scandirpaths(dirpath='.', **scan_kw)

A shim for scandirtree to yield filesystem paths from a directory.

Parameters:

• dirpath: optional top directory, default '.'

Other keyword arguments are passed to scandirtree.

scandirtree(dirpath='.', *, include_dirs=False, name_selector=None, only_suffixes=None, skip_suffixes=None, sort_names=False, follow_symlinks=False, recurse=True)

Generator to recurse over dirpath, yielding (is_dir,subpath) for all selected subpaths.

Parameters:

• dirpath: the directory to scan, default '.'
• include_dirs: if true yield directories; default False
• name_selector: optional callable to select particular names; the default is to select names not starting with a dot ('.')
• only_suffixes: if supplied, skip entries whose extension is not in only_suffixes
• skip_suffixes: if supplied, skip entries whose extension is in skip_suffixes
• sort_names: option flag, default False; yield entires in lexical order if true
• follow_symlinks: optional flag, default False; passed to scandir
• recurse: optional flag, default True; if true, recurse into subdrectories

shortpath(fspath, prefixes=None, *, collapseuser=False, foldsymlinks=False)

Return fspath with the first matching leading prefix replaced.

Parameters:

• prefixes: optional list of (prefix,subst) pairs
• collapseuser: optional flag to enable detection of user home directory paths; default False
• foldsymlinks: optional flag to enable detection of convenience symlinks which point deeper into the path; default False

The prefixes is an optional iterable of (prefix,subst) to consider for replacement. Each prefix is subject to environment variable substitution before consideration. The default prefixes is from SHORTPATH_PREFIXES_DEFAULT: (('$HOME/', '~/'),).

validate_rpath(rpath: str)

Test that rpath is a clean relative path with no funny business; raise ValueError if the test fails.

Tests:

• not empty or '.' or '..'
• not an absolute path
• normalised
• does not walk up out of its parent directory

Examples:

>>> validate_rpath('')
False
>>> validate_rpath('.')

Release Log

Release 20241122:

• FSPathBasedSingleton: add a .promote method to promote a filesystem path to an instance.
• FSPathBasedSingleton: new fspath_normalised(fspath) class method to produce a normalised form of the fspath for use as the key to the singleton registry.

Release 20241007: FSPathBasedSingleton.init: use an NRLock for the default lock, using a late import with fallback to Lock.

Release 20241005: needdir: now returns True if the directory was made, False if it already existed.

Release 20240630: FSPathBasedSingleton: recent Pythons seem to check that init returns None, subclasses must test another way.

Release 20240623:

• shortpath(foldsymlinks=True): only examine symlinks which have clean subpaths in their link text - this avoids junk and also avoids stat()ing links which might be symlinks to mount points which might be offline.
• scandirtree: clean up the logic, possibly fix repeated mention of directories.

Release 20240522: shortpath: new collapseuser=False, foldsymlinks=False parameters, rename DEFAULT_SHORTEN_PREFIXES to SHORTPATH_PREFIXES_DEFAULT.

Release 20240422: New scandirtree scandir based version of os.walk, yielding (is_dir,fspath). New shim scandirpaths.

Release 20240412: HasFSPath: explain that the init is optional in the docstring.

Release 20240316: Fixed release upload artifacts.

Release 20240201:

• FSPathBasedSingleton: drop the default_factory parameter/attribute, let default_attr specify a callable.
• Singleton._resolve_fspath: fix reference to class name.

Release 20231129:

• HasFSPath: new listdir method.
• HasFSPath.pathto: accept multiple relative subpaths.
• FSPathBasedSingleton: accept cls.FSPATH_FACTORY as a factory function for the default fspath, makes it possible to defer the path lookup.
• Replace is_clean_subpath with validate_rpath/is_valid_rpath pair.

Release 20230806:

• Reimplement fnmatchdir using fnmatch.filter.
• No longer claim Python 2 compatibility.

Release 20230401: HasFSPath.shortpath: hand call before .fspath set.

Release 20221221: Replace use of cs.env.envsub with os.path.expandvars and drop unused environ parameter.

Release 20220918:

• FSPathBasedSingleton.init: return True on the first call, False on subsequent calls.
• FSPathBasedSingleton.init: probe dict for '_lock' instead of using hasattr (which plays poorly this early on with classes with their own getattr).
• needdir: accept optional log parameter to log mkdir or makedirs.
• HasFSPath: add a default str.

Release 20220805: Doc update.

Release 20220530: FSPathBasedSingleton._resolve_fspath: new envvar and default_attr parameters.

Release 20220429:

• New HasFSPath and FSPathBasedSingleton.
• Add longpath and shortpath from cs.fileutils.
• New is_clean_subpath(subpath).
• New needdir(path).
• New fnmatchdir(dirpath,fnglob) pulled out from HasFSPath.fnmatch(fnglob).

Release 20220327: New module cs.fs to contain more filesystem focussed functions than cs.fileutils, which is feeling a bit bloated.