More consistent file handling across POSIX and Windows systems
- (verb) to struggle, suffer, contend
Because everything about managing files on Windows is terrible.
$ python -m pip install --upgrade winnan
Use the winnan.open() function exactly like using Python’s built-in open() function. It aims to be a drop-in replacement.
import winnan with winnan.open("myfile") as fileobj: pass
The file descriptor underlying the returned file object has the following two properties on both POSIX and Windows systems:
The file descriptor is non-inheritable.
The file can be scheduled for deletion while the file descriptor is still open.
Saying “scheduled for deletion” rather than “deleted” is to be pedantic about how the file is put into a “delete pending” state on Windows until the last file handle is closed. See Mercurial’s developer documentation as a reference for the semantics of certain operations on the file while it is in a “delete pending” state.
Unsurprisingly, the complications around managing files involve dealing with other processes. Depending on what the Python application is doing, there may be a need to deal with not only processes spawned by the Python application but also other processes running on the machine.
As documented by PEP-446, opening a file concurrently with spawning a subprocess may lead to the file descriptor being inherited unintentionally.
The process cannot access the file because it is being used by another process.
is perhaps the most classic representation of this issue on Windows. In modern versions of Python, the O_NOINHERIT flag is set by default when opening file descriptors on Windows. Setting it on the opened file descriptor prevents it from being inherited by a child process. The equivalent O_CLOEXEC flag is also set by default in modern versions of Python when opening file descriptors on POSIX systems.
It is worth mentioning that due to limitations with being able to set close_fds=True when redirecting stdin, stdout, or stderr in older versions of Python, setting the O_NOINHERIT flag isn’t sufficient for preventing files descriptors from being leaked when spawning subprocesses concurrently. Consider guarding all calls to subprocess.Popen with a threading.Lock instance to avoid this as an additional issue in older versions of Python.
On POSIX systems, it is possible to unlink() a file while it is still open in another thread or process. On Windows, in all versions of Python, non-O_TEMPORARY files are opened with FILE_SHARE_READ | FILE_SHARE_WRITE as their sharing mode. Omitting FILE_SHARE_DELETE prevents another thread or process from attempting to delete a file while it is still open.
The purpose of this library is to mask these behavioral differences across different platforms and in older versions of Python.
There have been multiple attempts to address the FILE_SHARE_DELETE issue within CPython itself but they unfortunately never succeeded in being integrated.
- bpo-12939: tempfile.NamedTemporaryFile not particularly useful on Windows
- bpo-14243: Support for opening files with FILE_SHARE_DELETE on Windows
- bpo-15244: Add new io.FileIO using the native Windows API
Starting in Python 3.7, close_fds=True is able to be set even when redirecting stdin, stdout, or stderr.
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size & hash||File type||Python version||Upload date|
|winnan-0.1.0.tar.gz (50.4 kB) View hashes||Source||None|