Skip to main content

Atomic file writes.

Project description

https://travis-ci.org/untitaker/python-atomicwrites.svg?branch=master https://ci.appveyor.com/api/projects/status/vadc4le3c27to59x/branch/master?svg=true

Atomic file writes.

from atomicwrites import atomic_write

with atomic_write('foo.txt', overwrite=True) as f:
    f.write('Hello world.')
    # "foo.txt" doesn't exist yet.

# Now it does.

Features that distinguish it from other similar libraries (see Alternatives and Credit):

  • Race-free assertion that the target file doesn’t yet exist. This can be controlled with the overwrite parameter.
  • Windows support, although not well-tested. The MSDN resources are not very explicit about which operations are atomic.
  • Simple high-level API that wraps a very flexible class-based API.
  • Consistent error handling across platforms.

How it works

It uses a temporary file in the same directory as the given path. This ensures that the temporary file resides on the same filesystem.

The temporary file will then be atomically moved to the target location: On POSIX, it will use rename if files should be overwritten, otherwise a combination of link and unlink. On Windows, it uses MoveFileEx through stdlib’s ctypes with the appropriate flags.

Note that with link and unlink, there’s a timewindow where the file might be available under two entries in the filesystem: The name of the temporary file, and the name of the target file.

Also note that the permissions of the target file may change this way. In some situations a chmod can be issued without any concurrency problems, but since that is not always the case, this library doesn’t do it by itself.

fsync

On POSIX, fsync is invoked on the temporary file after it is written (to flush file content and metadata), and on the parent directory after the file is moved (to flush filename).

fsync does not take care of disks’ internal buffers, but there don’t seem to be any standard POSIX APIs for that. On OS X, fcntl is used with F_FULLFSYNC instead of fsync for that reason.

On Windows, _commit is used, but there are no guarantees about disk internal buffers.

Alternatives and Credit

Atomicwrites is directly inspired by the following libraries (and shares a minimal amount of code):

Other alternatives to atomicwrites include:

  • sashka/atomicfile. Originally I considered using that, but at the time it was lacking a lot of features I needed (Windows support, overwrite-parameter, overriding behavior through subclassing).
  • The Boltons library collection features a class for atomic file writes, which seems to have a very similar overwrite parameter. It is lacking Windows support though.

License

Licensed under the MIT, see LICENSE.

Project details


Release history Release notifications

This version
History Node

1.1.5

History Node

1.1.0

History Node

1.0.0

History Node

0.1.9

History Node

0.1.8

History Node

0.1.7

History Node

0.1.6

History Node

0.1.5

History Node

0.1.4

History Node

0.1.3

History Node

0.1.2

History Node

0.1.1

History Node

0.1.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
atomicwrites-1.1.5-py2.py3-none-any.whl (7.5 kB) Copy SHA256 hash SHA256 Wheel 3.4 Aug 14, 2016
atomicwrites-1.1.5.tar.gz (19.0 kB) Copy SHA256 hash SHA256 Source None Aug 14, 2016

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging CloudAMQP CloudAMQP RabbitMQ AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page