Skip to main content

Invisible, auto-syncing personal storage for any git repo

Project description

Swarf

Invisible, auto-syncing personal storage for any git repo.

Swarf is the metal shavings left on the workshop floor after machining — the byproduct of making something. When you build software (especially with AI agents), you generate a lot of byproduct: research docs, design specs, agent skills, open questions, scratch notes. Swarf gives this material a durable home alongside any project, without touching the project itself.

The problem

You have files that should live near your code but don't belong in the repo:

  • Agent-generated research and design docs
  • Personal notes, roadmaps, and open questions
  • Agent skill files and config (.copilot/skills/, AGENTS.md)
  • Security audits, experiment logs, scratch work

Today these files are either untracked and local-only (one rm -rf from gone) or committed to the repo (polluting history, leaking on public repos). Neither is good.

How it works

Swarf creates a .swarf/ directory inside your project — a separate git repo, invisible to the host repo via your global gitignore. Files in .swarf/ are version-controlled and automatically synced to a remote backend.

my-project/                      ← your repo (public)
├── src/
├── tests/
├── .swarf/                      ← swarf repo (private, globally gitignored)
│   ├── docs/research/           ← durable notes
│   ├── docs/design/             ← specs and decisions
│   ├── links/                   ← files projected into the host tree
│   │   ├── AGENTS.md            ← symlinked to ./AGENTS.md
│   │   └── .copilot/skills/     ← symlinked to ./.copilot/skills/
│   └── open-questions.md
└── scratch/                     ← ephemeral, untracked by anything

Linking

Files in .swarf/links/ are automatically symlinked into the host repo tree. A file at .swarf/links/AGENTS.md appears as ./AGENTS.md. A file at .swarf/links/.copilot/skills/my-skill/SKILL.md appears at ./.copilot/skills/my-skill/SKILL.md. The directory structure is the manifest.

Linking runs automatically via a mise enter hook — every time you cd into the project, links are verified and fixed. You never run it manually.

Syncing

A background daemon watches all your .swarf/ directories for file changes. When files change, it waits for a quiet period (debounce), then commits and syncs to the configured backend. You never think about it.

Install

uv tool install swarf

Quick start

cd ~/projects/my-app

# Initialize with git backend
swarf init --backend git --remote git@github.com:you/my-app-internal.git

# Initialize with Google Drive backend (for work repos you don't control)
swarf init --backend rclone --remote "gdrive:swarf/my-app"

# That's it. The daemon handles everything from here.

After swarf init:

  • .swarf/ exists with standard directory structure
  • A mise enter hook is installed (auto-linking on cd)
  • The daemon is watching for changes (auto-syncing)
  • Your global gitignore hides .swarf/ from the host repo

Usage

# Check sync status across all projects
swarf status

# Validate setup is healthy
swarf doctor

# Start/stop the background daemon
swarf daemon start
swarf daemon stop
swarf daemon status

# Install daemon as a system service (auto-start on login)
swarf daemon install

Backends

Git (default)

Commits changes after a quiet period, pushes to a remote. Best for personal projects where you can create a private companion repo.

swarf init --backend git --remote git@github.com:you/project-internal.git

Rclone

Syncs the .swarf/ directory to any rclone-supported remote: Google Drive, Dropbox, S3, OneDrive, etc. Best for work environments where you can't easily create a private git repo.

# First: configure rclone with Google Drive (one-time)
# Use drive.file scope for minimal permissions
rclone config

# Then:
swarf init --backend rclone --remote "gdrive:swarf/work-monorepo"

The drive.file OAuth scope means the token can only access files swarf created — it can't read your other Google Drive documents.

How it works on a company monorepo

The same way. Your global gitignore hides .swarf/ from every repo on your machine, including ones you don't control. No changes to the monorepo's .gitignore, no submodules, no permission needed.

cd ~/work/big-monorepo
swarf init --backend rclone --remote "gdrive:swarf/work"

Your agent config, personal notes, and research are now durable and synced, completely invisible to your coworkers and CI.

How it works with AI agent containers

If you use containerized agents (like yolo-jail), the daemon runs on the host, not inside the container. The container mounts the workspace directory, so file changes from agents are visible to the host filesystem. The daemon picks them up and syncs automatically. Agents never need credentials.

Global gitignore

Swarf relies on your global gitignore (~/.config/git/ignore or equivalent) to hide .swarf/ and linked files from host repos. swarf init checks and prompts you to add entries if missing:

/.swarf/
/.mise.local.toml
AGENTS.md

This is the invisible layer that makes swarf work on any repo without modifying it.

Configuration

Each .swarf/ directory has a config.toml:

[sync]
backend = "git"          # or "rclone"
remote = "origin"        # git remote name, or rclone remote path
debounce = "5s"          # wait this long after last change before syncing

The daemon reads a central registry at ~/.config/swarf/drawers.toml that lists all known .swarf/ directories and their backends.

Contributing

See CONTRIBUTING.md for development setup and guidelines.

License

Apache 2.0

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

swarf-0.1.0.tar.gz (31.7 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

swarf-0.1.0-py3-none-any.whl (9.3 kB view details)

Uploaded Python 3

File details

Details for the file swarf-0.1.0.tar.gz.

File metadata

  • Download URL: swarf-0.1.0.tar.gz
  • Upload date:
  • Size: 31.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: uv/0.11.2 {"installer":{"name":"uv","version":"0.11.2","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for swarf-0.1.0.tar.gz
Algorithm Hash digest
SHA256 ec204f4ef2dd1ae3553405a62bf94539a93e678cc00df166a155d999cfea0d7d
MD5 c67f74003ad5c6ca9cc5e2a437a02e9b
BLAKE2b-256 c4609950a3d424a9ca1ebefaf4e4060de1c2c825a4d1e35d67cb1eb0f665ff59

See more details on using hashes here.

File details

Details for the file swarf-0.1.0-py3-none-any.whl.

File metadata

  • Download URL: swarf-0.1.0-py3-none-any.whl
  • Upload date:
  • Size: 9.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: uv/0.11.2 {"installer":{"name":"uv","version":"0.11.2","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for swarf-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 8f4d8acded3b3e8e4abb0088621507e0769bc62b117dfcace8b4af931301f3ba
MD5 838b02a63241cd2282165c018a7aefc6
BLAKE2b-256 a946fcebd95e2e24cf120bb8b71caea02c1261e6ada2570dca5c672703d6d3c7

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page