atomic-json-io

Crash-safe, concurrency-safe atomic file writes for JSON, plain text, and JSONL on Linux. Pure standard library — zero runtime dependencies.

A reader of the target file always sees either the complete previous content or the complete new content, never a half-written (torn) file. Multiple threads or processes can write to the same target path at the same time without corrupting the file or racing on a shared temporary file.

Install

From PyPI:

pip install atomic-json-io

Or from source (GitHub):

pip install git+https://github.com/JohnLinotte/atomic-json-io.git

Requires Python 3.10+ and Linux.

Usage

Write JSON

from pathlib import Path
from atomic_json_io import write_json_atomic

write_json_atomic(Path("config.json"), {"name": "atomic", "version": 1})

Write text

from pathlib import Path
from atomic_json_io import write_text_atomic

write_text_atomic(Path("notes.md"), "# Title\n\nSome text")

Write JSONL

from pathlib import Path
from atomic_json_io import write_jsonl_atomic

write_jsonl_atomic(
    Path("events.jsonl"),
    [{"id": 1, "event": "open"}, {"id": 2, "event": "close"}],
)

All three helpers create parent directories automatically and add a trailing newline.

Why this is atomic on Linux

The write does not modify the target file in place. Instead it:

1. Serializes the payload to a brand-new temporary file in the same directory as the target.
2. Calls file.flush() then os.fsync(fd) so the bytes (and the file's data) are durably on disk before the swap.
3. Calls os.replace(tmp, target) to move the temporary file onto the final path.

On Linux, os.replace is implemented with the rename(2) syscall, which the POSIX standard guarantees to be atomic: at any instant the target name resolves to either the old inode or the new inode, never to a partially written file. A concurrent reader therefore opens one complete version or the other.

The fsync step matters for crash safety specifically: without it, a power loss right after os.replace could leave the directory entry pointing at a file whose data blocks were never written. Flushing and fsyncing before the rename closes that window.

The cross-filesystem pitfall

rename(2) — and therefore os.replace — is only atomic when the source and the target are on the same filesystem. That is exactly why the temporary file is created in the same directory as the target rather than in /tmp or some other location. A naive implementation that writes to /tmp and then "moves" the file onto a target on a different mount would fall back to a copy-then-delete under the hood, which is not atomic and can expose a torn file. Keeping the temporary file as a sibling of the target guarantees a same-filesystem rename.

How concurrency is handled

Every write generates its own temporary filename containing a UUID nonce, combined with the process id and the thread id:

<target>.<pid>.<tid>.<nonce>.tmp

Because each writer owns a distinct temporary file, two simultaneous writers never share or clobber each other's in-progress data. They each fsync their own temp file and then race only on the final os.replace; the last rename to complete wins, and the file is always one valid, complete payload. There is no shared-temp-file race that could make one writer's os.replace fail with FileNotFoundError because another writer already renamed the shared temp away.

This holds across threads (thread id + nonce) and across processes (pid + nonce).

Scope

Linux-only. There is no Windows portability claim — os.replace semantics over an existing target differ across platforms, and the fsync-before-rename durability argument relies on POSIX rename behavior.

License

MIT — see LICENSE.