python-atomicwrites
Atomic file writes.
from atomicwrites import atomic_write
with atomic_write('foo.txt', overwrite=True) as f:
f.write('Hello world.')
# "foo.txt" doesn't exist yet.
# Now it does.See API documentation for more low-level interfaces.
Features that distinguish it from other similar libraries (see Alternatives and Credit):
Race-free assertion that the target file doesn’t yet exist. This can be controlled with the overwrite parameter.
Windows support, although not well-tested. The MSDN resources are not very explicit about which operations are atomic. I’m basing my assumptions off a comment by Doug Cook, who appears to be a Microsoft employee:
Question: Is MoveFileEx atomic if the existing and new files are both on the same drive?
The simple answer is “usually, but in some cases it will silently fall-back to a non-atomic method, so don’t count on it”.
The implementation of MoveFileEx looks something like this: […]
The problem is if the rename fails, you might end up with a CopyFile, which is definitely not atomic.
If you really need atomic-or-nothing, you can try calling NtSetInformationFile, which is unsupported but is much more likely to be atomic.
Simple high-level API that wraps a very flexible class-based API.
Consistent error handling across platforms.
How it works
It uses a temporary file in the same directory as the given path. This ensures that the temporary file resides on the same filesystem.
The temporary file will then be atomically moved to the target location: On POSIX, it will use rename if files should be overwritten, otherwise a combination of link and unlink. On Windows, it uses MoveFileEx through stdlib’s ctypes with the appropriate flags.
Note that with link and unlink, there’s a timewindow where the file might be available under two entries in the filesystem: The name of the temporary file, and the name of the target file.
Also note that the permissions of the target file may change this way. In some situations a chmod can be issued without any concurrency problems, but since that is not always the case, this library doesn’t do it by itself.
fsync
On POSIX, fsync is invoked on the temporary file after it is written (to flush file content and metadata), and on the parent directory after the file is moved (to flush filename).
fsync does not take care of disks’ internal buffers, but there don’t seem to be any standard POSIX APIs for that. On OS X, fcntl is used with F_FULLFSYNC instead of fsync for that reason.
On Windows, _commit is used, but there are no guarantees about disk internal buffers.
Alternatives and Credit
Atomicwrites is directly inspired by the following libraries (and shares a minimal amount of code):
The Trac project’s utility functions, also used in Werkzeug and mitsuhiko/python-atomicfile. The idea to use ctypes instead of PyWin32 originated there.
abarnert/fatomic. Windows support (based on PyWin32) was originally taken from there.
Other alternatives to atomicwrites include:
sashka/atomicfile. Originally I considered using that, but at the time it was lacking a lot of features I needed (Windows support, overwrite-parameter, overriding behavior through subclassing).
The Boltons library collection features a class for atomic file writes, which seems to have a very similar overwrite parameter. It is lacking Windows support though.
License
Licensed under the MIT, see LICENSE.
1 maintainer
Metadata
1 maintainer
Release files for atomicwrites 1.4.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| atomicwrites-1.4.1.tar.gz | 14.2 kB | Without provenance | Details |
Release files / atomicwrites-1.4.1.tar.gz
| Download URL | atomicwrites-1.4.1.tar.gz |
|---|---|
| Size | 14.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
81b2c9071a49367a7f770170e5eec8cb66567cfbbc8c73d20ce5ca4a8d71cf11
|
|
BLAKE2b-256 checksum How to use checksums |
87c653da25344e3e3a9c01095a89f16dbcda021c609ddb42dd6d7c0528236fb2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
Python-urllib/2.7
|
Security for atomicwrites 1.4.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
