Directory synchronization tool yd

Build and execute rsync commands from TOML configuration files.

(Un)Install

To install, run

uv tool install yd-cli

Then run

yd --install-completion

to install auto-completion in your shell.

In bash, the completion code is stored in ~/.bash_completions/yd.sh and that file is sourced from ~/.bashrc. Remove both and call uv tool uninstall yd-cli to remove this tool. The below mentioned configuration files need to be removed separately.

Create a config

Create a new configuration called photos (using a positive number of . as name is not allowed) with:

yd edit --new photos

This command

• creates ~/.config/yd/photos.toml (or under $XDG_CONFIG_HOME/yd if set to a nonempty value; relative paths are resolved from your home directory), and
• opens it with the executable selected via EDITOR, or nvim when EDITOR is unset. The selected editor must be available on PATH.

Reopen an existing configuration with:

yd edit photos

Configuration format

Example

# Phone backup
src_home = "/home/alice"
target_home = "/mnt/backup"
backup = "deleted/%Y-%m-%d"
exclude = [".venv/", "__pycache__/"]

[[commands]]
src = "Documents"
exclude = ["*.log"]

[[commands]]
src = "Pictures"
target = "pics-%Y-%m-%d"
delete_extra = "never"

Leading comment lines directly at the top of the file are treated as the configuration description and are shown by yd ls. The first description line of the form # # Personal backups in this header places the configuration in a panel titled Personal backups and is not included in the description. Leading and trailing whitespace is removed from the panel title. Configurations without a panel specification are shown in the Configurations panel.

Top-level options

mtp_target matters because rsync normally copies to a temporary file and renames it afterward, but that is not possible when syncing via MTP.

Command options

Notes

• src and target must stay within src_home and target_home after path resolution. A single . selects the corresponding home directory; otherwise, . and .. path components are not allowed.
• Literal targets are retained when they contain only empty directories. Dated targets serve as snapshots: if a dated target did not exist before synchronization and contains neither files nor symlinks afterward, yd removes its empty directory tree.
• src_home or target_home may be omitted from the configuration, but every missing value must then be supplied when running (via CLI parameters).
• Exactly one of --src-home - and --target-home - may read from standard input in a single run.
• If backup is omitted, yd creates a timestamped backup directory automatically unless --no-backup is specified.
• If a configured source directory exists but is empty, yd reports that no synchronization was performed for that command. E.g., forgetting to mount an external drive does not delete all corresponding copies on hard-drive. Emptiness is checked directly before synchronization starts. When delete_extra is "when_safe", yd suggests deleting the target contents manually only if the target does not overlap another command's target. If it overlaps, yd instead reports that deletion was disabled.
• If a valid date placeholder is included in the target, then the target_home is scanned for matching directories (possibly) created by this rule before the current target period. Thus, the current year is excluded from yearly targets, the current month from monthly targets, and today from daily targets. The 20 most recent matches are included as additional comparison destinations in the synchronization: if a file in src is included in one of those reference directories, then it will not be copied to target.

Run a config

Run a saved configuration by name:

yd run photos

Options

Synchronization semantics

• Synchronization does not cross mounted filesystems, and mount-point directories are omitted from the target.
• Symlinks are copied as symlinks; symlinks pointing outside the individual source directories are ignored.
• Permissions, modification times, access times, and hard links are preserved.
• File owner or group are not preserved.

Commands with overlapping targets are run serially in configuration order, and automatic deletion of extra target files is disabled for those commands. This permits multiple commands to contribute to the same target tree, but it does not make each individual command a no-op on an unchanged repeated run. If two sources map different contents to the same target file, the earlier command can overwrite that file before the later command restores its final contents on every run. Those writes can also create backups.

When commands are prepared, yd records the filesystem device and inode of every source directory. Immediately before starting a command, it verifies that the source path still names that same directory. A missing source, a non-directory, or a different device or inode aborts the run before rsync is started for that command; commands depending on it are not started.

This check is not a lock or a snapshot. From the beginning of the final check until rsync exits, a mounted source must remain mounted at the same path, and the source directory must not be replaced. yd does not continuously verify its identity after process startup. Files within the source should also not be modified during synchronization when a consistent result is required; concurrent content changes are subject to rsync behavior, particularly when deletion is enabled.

Environment variables

Logging is effectively disabled by default. Enable it by setting YD_LOGLEVEL when diagnosing a failed or interrupted run. When logging is enabled, yd run rejects any source, target, backup, or comparison tree that intersects with $XDG_STATE_HOME/yd. Until all synchronization paths have been validated, records are buffered in memory; if validation fails, those records are written as JSON Lines to standard error instead of the file.

List configs

List available configurations with:

yd ls

This shows the configuration name together with the optional leading-comment description.

Why yd?

• Has one character from synchronize and one from directory.
• Easy to type with both QWERTZ and QWERTY keyboards.
• Name was still available on PyPI.