zfs-tenant

Give a friend a quota-capped corner of your ZFS pool for their encrypted backups, without giving them a shell or a look at your data.

Table of Contents

• Why
• How it works
• Security model
  • What it cannot hide
  • Residual risks
• Quick start on NixOS
• Manual setup (TrueNAS SCALE or any Linux)
• Pushing with syncoid by hand
• Restoring
• Removing datasets
• What the gate allows
• FAQ
• Development
• License

Why

A friend with a ZFS box is the cheapest off-site backup there is: you store their snapshots, they store yours. The hard part is trust. Replication tools assume root on the receiving side, and nobody wants to hand a friend root on the machine that holds their family photos.

A friend and I used to solve this with a TrueNAS VM on top of an iSCSI zvol, so each of us could be root inside a disposable VM instead of on the real NAS. It worked, but it was a lot of machinery for what is really a permissions problem.

OpenZFS already has the permission system: zfs allow can delegate receive, create, and destroy on one dataset to an unprivileged user, and a root-owned quota caps how much that user can store. What delegation alone does not do is stop that user from listing every dataset on your machine, or from running anything else once they can log in. zfs-tenant closes that gap twice: a forced command that only runs backup commands, and a user namespace that zfs zone restricts to the friend's own datasets. It also comes with the setup commands and NixOS modules that turn all of it into a few lines of config on both sides.

How it works

flowchart LR
    subgraph joe["Joe's machine"]
        sanoid["sanoid<br/>snapshots"] --> syncoid["syncoid<br/>(non-root, zfs send and hold only)"]
    end

    subgraph bas["Your machine"]
        sshd["sshd<br/>forced command"] --> gate["zfs-tenant gate<br/>joins the tenant's zone,<br/>allowlist scoped to the root"]
        gate --> zfs["zfs, as user zfs-tenant-joe<br/>delegated on tank/friends/joe only,<br/>sees only tank/friends/joe"]
        zfs --> root[("tank/friends/joe<br/>quota 2T, mountpoint=none")]
    end

    syncoid ==>|"raw encrypted stream<br/>over your tailnet"| sshd

The kernel is the jail; the gate removes the shell.

1. The tenant root. zfs-tenant setup creates tank/friends/joe, sets a quota, dataset and snapshot limits, and properties that make sure nothing under it is ever mounted, shared, or exposed as a device. Joe cannot change any of them.
2. Delegation. On the root itself, Joe's local user may only create and receive children. Below it, he may also destroy and send. OpenZFS checks these rights in the kernel on every operation, whatever program asks.
3. The gate. Joe's SSH key is pinned to zfs-tenant gate with restrict,from=...,command=... in authorized_keys. The gate parses the requested command, accepts only the forms syncoid and a restore need, checks that every dataset is inside Joe's root, and runs zfs with an argument list it builds itself. It never starts a shell.
4. The zone. A small service keeps a Linux user namespace alive for Joe, in which his uid maps to itself, and zfs zone attaches his root to it with zoned=on. The gate joins that namespace before it does anything, and the ZFS kernel module then answers dataset does not exist for every dataset that is not Joe's. Joe keeps his own uid in there, so he holds no capabilities and zfs allow still decides what he may change. If the service is down, the gate refuses to run.
5. Raw sends. Joe sends with zfs send -w, so his blocks arrive still encrypted with his key. The gate refuses and removes any newly received dataset that is not encrypted.

Security model

Every row is exercised by a two-node NixOS VM test with real OpenZFS and real syncoid (nix/integration-test.nix).

What it cannot hide

OpenZFS encryption protects file contents, not structure. From zfs-load-key(8): "ZFS will not encrypt metadata related to the pool structure, including dataset and snapshot names, dataset hierarchy, properties, file size, file holes, and deduplication tables." So you, as the host, can see the names, sizes, and snapshot times of Joe's datasets. Give datasets you send to a friend boring names, and never send with -p or -R, which would include properties.

You are also root on your own machine. You can always delete Joe's backup copy, even though you can never read it.

Residual risks

• The kernel parses the send streams Joe sends you. That is the same exposure as any ZFS replication.
• A leaked tenant key lets someone push data up to the quota and delete that tenant's backups.
• Inside a zone, pool-level information (zpool list, zpool status) and the parent datasets' sizes are still visible to zfs. The gate does not allow those commands; the zone only matters if the gate is bypassed.
• syncoid 2.3.0 pastes the resume token it gets from the receiving host into a shell on the sending machine without escaping it, so a malicious host could run commands on the sender as the user running syncoid. The sender module therefore runs syncoid as a dedicated user that holds only zfs send and hold rights on the datasets it pushes. If you push by hand, do the same.

Quick start on NixOS

Add the flake:

{
  inputs.zfs-tenant.url = "github:basnijholt/zfs-tenant";

  outputs = { nixpkgs, zfs-tenant, ... }: {
    nixosConfigurations.nas = nixpkgs.lib.nixosSystem {
      system = "x86_64-linux";
      modules = [
        zfs-tenant.nixosModules.host    # to give friends space
        zfs-tenant.nixosModules.sender  # to push your own backups
        ./configuration.nix
      ];
    };
  };
}

On the host, give Joe a tenant root:

services.zfs-tenant = {
  enable = true;
  tenants.joe = {
    dataset = "tank/friends/joe";
    quota = "2T";
    authorizedKeys = [ "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAA... joe-nas" ];
    allowedFrom = [ "100.64.0.12" ];  # Joe's tailnet address
  };
};

This creates the user zfs-tenant-joe, pins the key to the gate, applies the dataset, properties, and delegation on every boot, and runs zfs-tenant-zone-joe.service, which holds Joe's zone. The host needs OpenZFS 2.2 or newer for zfs zone. Set reservation = "2T"; as well if you want to guarantee Joe the space and hide how full your pool is.

On Joe's side, push with syncoid:

services.zfs-tenant-sender = {
  enable = true;
  targets.bas = {
    host = "bas-nas";
    user = "zfs-tenant-joe";
    sshKey = "/var/lib/zfs-tenant-sender/id_ed25519";
    knownHosts = "bas-nas ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAA...";
    datasets."tank/offsite" = "tank/friends/joe/offsite";
    onCalendar = "daily";
  };
};

Create the key once, readable only by the sender user, and send Joe's public half to the host:

sudo -u zfs-tenant-sender ssh-keygen -t ed25519 -N '' -f /var/lib/zfs-tenant-sender/id_ed25519

The source dataset (tank/offsite here) must be encrypted, and sanoid should snapshot it: the sender runs syncoid with --no-sync-snap, so it only sends the snapshots sanoid made.

On the tailnet, allow only Joe's node to reach port 22 on your host.

Manual setup (TrueNAS SCALE or any Linux)

The gate uses only the Python standard library, so a single file is enough. Download zfs-tenant.pyz from the latest release and keep it on a pool dataset, so it survives appliance updates:

curl -L -o /mnt/tank/admin/zfs-tenant.pyz \
  https://github.com/basnijholt/zfs-tenant/releases/latest/download/zfs-tenant.pyz

Or install it with uv tool install zfs-tenant or pip install zfs-tenant where that is possible.

1. Create a local user for Joe with a normal login shell such as bash, no password, and no extra groups. sshd runs forced commands through the login shell. On TrueNAS, give the user a home directory on a pool dataset so its authorized_keys persists.

2. Look at what setup will do, then run it as root:

   python3 zfs-tenant.pyz setup --root tank/friends/joe --user joe --quota 2T --dry-run
   sudo python3 zfs-tenant.pyz setup --root tank/friends/joe --user joe --quota 2T
   

   Delegation and properties live in the pool, so they survive reboots and appliance updates.

3. Produce the authorized_keys line and put it in that user's ~/.ssh/authorized_keys:

   python3 zfs-tenant.pyz authorized-key \
     --gate-command "/usr/bin/python3 /mnt/tank/admin/zfs-tenant.pyz gate --root tank/friends/joe --zfs /usr/sbin/zfs --zpool /usr/sbin/zpool --zone-pid-file /run/zfs-tenant-joe.pid" \
     --from 100.64.0.12 \
     ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAA... joe-nas
   

4. Start Joe's zone at boot, as root. On TrueNAS, add this as a post-init script:

   nohup python3 /mnt/tank/admin/zfs-tenant.pyz zone --root tank/friends/joe --user joe \
     --pid-file /run/zfs-tenant-joe.pid --zfs /usr/sbin/zfs >/var/log/zfs-tenant-joe.log 2>&1 &
   

   setup sets zoned=on, and the gate command above refuses to run until this holder is up. The kernel must allow unprivileged user namespaces (Debian and NixOS do by default).

Pushing with syncoid by hand

syncoid --no-privilege-elevation --no-sync-snap --sendoptions=w --compress=none \
  --recursive --delete-target-snapshots \
  tank/offsite joe@bas-nas:tank/friends/joe/offsite

• --no-privilege-elevation: the gate refuses sudo.
• --sendoptions=w: raw sends. The host never sees plaintext and the gate refuses anything else.
• --no-sync-snap: send sanoid's snapshots instead of creating syncoid's own, so the sending user needs only send and hold.
• --compress=none: raw encrypted data does not compress. The gate reports that lzop and mbuffer are missing on its side anyway, so syncoid skips them.
• --delete-target-snapshots: mirror your sanoid retention on the host.

The receive always runs with -u and never with -F; the gate drops -F because nothing under a tenant root can be mounted or modified between receives. Run syncoid as a non-root user with zfs allow -u <user> send,hold <dataset> on the sending side.

Restoring

Pull a snapshot back through the same key and unlock it at home:

ssh joe@bas-nas zfs send -w tank/friends/joe/offsite/photos@autosnap_2026-09-25_00:00:01_daily \
  | zfs receive -u tank/restored/photos
zfs load-key tank/restored/photos
zfs mount tank/restored/photos

If the transfer breaks, resume it with the token your side kept:

token=$(zfs get -H -o value receive_resume_token tank/restored/photos)
ssh joe@bas-nas zfs send -t "$token" | zfs receive -s -u tank/restored/photos

List what the host keeps for you with ssh joe@bas-nas zfs list -r -t all -o name,used,creation.

Removing datasets

Everything below your root is yours to remove:

ssh joe@bas-nas zfs destroy -r tank/friends/joe/old
ssh joe@bas-nas zfs destroy tank/friends/joe/offsite@autosnap_2026-01-01_00:00:01_daily

What the gate allows

D is a dataset inside the tenant root, D↓ a dataset strictly below it, S a snapshot name.

Anything else exits 126 with zfs-tenant: command not allowed: <reason>, and every decision is logged to the auth log with the tenant root. Every allowed command runs inside the tenant's zone.

FAQ

Why a user namespace but not a full container or VM? The part that needs isolating is ZFS, and zfs zone isolates exactly that: the kernel filters which datasets a namespace can see. A container would still need /dev/zfs, and the usual container setup makes the tenant root inside its namespace, which ZFS treats as the zone's administrator: that bypasses zfs allow, so the tenant could, for example, destroy the root dataset you created for them. zfs-tenant maps the tenant to its own uid inside the namespace instead, so delegation keeps deciding what it may change. The old VM setup existed only because TrueNAS replication needed root on the receiving end; delegation removes that need.

Why a holder service? In OpenZFS 2.4, zfs zone attaches a dataset to one running namespace, so something has to keep that namespace alive. OpenZFS master can attach datasets to a uid instead (zoned_uid); once that is released, the holder can go.

Why not zrepl? zrepl's sink mode does per-client subtrees, but it replaces sanoid and syncoid on both sides and runs as root on the receiver. Here the kernel enforces the boundary, and both sides keep the tools they already use.

Why push instead of pull? With pull, the host would need rights on the friend's machine, and the friend could not create or remove datasets on their own. With push, the friend owns their corner of your pool and you hold no keys to theirs.

Why Python and not a shell script? The gate's input is an untrusted string. Parsing it in shell invites word splitting and injection; Python gives a real tokenizer, strict allowlists, exec of an argument list, and unit tests. It stays dependency-free so it runs from a single file on appliances.

Development

just install  # uv sync --dev
just test     # unit tests
just lint     # ruff, mypy, ty
just vm-test  # two-node NixOS VM test with real OpenZFS and syncoid
just pyz      # build dist/zfs-tenant.pyz

License

MIT