pkgforge

Stage files into a build root and record their intended install metadata (mode, owner, group, type, and free-form key/value meta) in a file DB (JSON Lines, YAML, or SQLite), then dump that DB into packaging manifests — an RPM %files list or Debian install + permissions + dirs + fixperms files.

pkgforge is a small, dependency-light helper for unattended build pipelines on Linux: install a source into place, remember how it should be owned and permissioned, and emit that record for the packager.

Install

pip install pkgforge

Requires Python 3.9+. Runtime operations use POSIX facilities (chmod, chown, symlinks), so runtime targets Linux; the CLI and --help import cleanly on any platform. A tar-family archive given as a real path extracts via stdlib tarfile — no external archiver needed; a - (stdin) source and other formats (e.g. .iso) fall back to bsdtar. Neither path restores an archive's ownership or special bits as root, and a device node, FIFO or socket in the archive is refused.

Quick start

work="$(mktemp -d)"
export PKGFORGE_ROOT="$work/stage" PKGFORGE_DB="$work/files.jsonl"
mkdir -p "$PKGFORGE_ROOT"

pkgforge initdb
pkgforge install -p -m 755 -o root -g root ./build/tool /usr/bin
pkgforge install -p -m 640 -o root -g adm -O rpmprefix=%config ./tool.conf /etc
pkgforge install -D -d -m 755 -o root -g root --record-tree ./share /usr/share/tool
pkgforge dbdump -f rpmspecfiles rpm-files.txt
pkgforge dbdump -f debian debian/

See examples/stage_and_package.sh for a runnable end-to-end walkthrough.

Commands

install --method {copy,link,move} (env PKGFORGE_INSTALL_METHOD) stages a file or directory source by hardlinking or moving it instead of the default copy; install --record-tree (env PKGFORGE_INSTALL_RECORD_TREE) also records a directory or archive install's own contents, not just DESTINATION itself -- see the commands guide.

Global options (also read from the environment):

Global flags work before or after the subcommand. With no --buildroot or PKGFORGE_ROOT, the default is the current directory -- except that a cwd of / is refused (exit 2); pass --buildroot / to target the live filesystem on purpose.

Storage backends

The file DB has three interchangeable backends — every command behaves the same regardless of which is used:

The backend is picked from the --db extension (override with --db-format); reading an existing file auto-detects its actual format.

Dump formats

Output of the Quick start above:

$ pkgforge dbdump -f rpmspecfiles -
%config %attr(640,root,adm) "/etc/tool.conf"
%attr(755,root,root) "/usr/bin/tool"
%dir %attr(755,root,root) "/usr/share/tool"
%attr(644,root,root) "/usr/share/tool/data.txt"

$ pkgforge dbdump -f debian -
# === install ===
etc/tool.conf etc
usr/bin/tool usr/bin
usr/share/tool/data.txt usr/share/tool
# === permissions ===
/etc/tool.conf 640 root adm
/usr/bin/tool 755 root root
/usr/share/tool 755 root root
/usr/share/tool/data.txt 644 root root
# === dirs ===
usr/share/tool
# === fixperms ===
#!/bin/sh
# Generated by pkgforge dbdump -f debian. Usage: sh fixperms PACKAGE-DIR
set -e
d=${1:?usage: sh fixperms PACKAGE-DIR}
chown -- root:adm "$d"/etc/tool.conf
chmod -- 640 "$d"/etc/tool.conf
chown -- root:root "$d"/usr/bin/tool
chmod -- 755 "$d"/usr/bin/tool
chown -- root:root "$d"/usr/share/tool
chmod -- 755 "$d"/usr/share/tool
chown -- root:root "$d"/usr/share/tool/data.txt
chmod -- 644 "$d"/usr/share/tool/data.txt

File entries

Each entry records mode (octal string, e.g. 644), owner, group, type (file/directory/symlink), and a meta map. Two sentinels defer a field to the staged file: - ("leave at OS default") and -- ("resolve from disk", also spelled auto for -m). -m accepts only 1-4 octal digits or a sentinel and is validated before anything is staged.

Exclude / filter syntax

An --exclude statement is an optional leading ! (negate), zero or more inline tests, and a trailing glob:

(?type:file)**/*.pyc            # every .pyc file, at any depth
!(?meta:keep=1)**/tmp/**        # keep entries tagged keep=1 under tmp/ ...
**/tmp/**                       # ... paired with a broader exclude

Tests are (?type:file|directory|symlink) and (?meta:key=value); prefix a test name with ! ((?!type:file)) to invert just that test. ** recurses (a trailing ** means "the contents of this directory"); an absolute pattern anchors at the install path — the /-rooted path an entry has (dbdump) or will have (install/scan) in the file DB — the same coordinate on every command. install/scan prune an excluded directory's subtree, while dbdump decides entry by entry on its flat key list. See the exclude grammar guide for the full grammar and worked examples.

Documentation

Full docs at https://jose-pr.github.io/pkgforge/ — command reference, file-DB model, exclude grammar, dump formats, and the API reference.

API overview

pkgforge is primarily a CLI; the modules below are its importable surface. See the full contract in pkgforge/AGENTS.md, shipped inside the installed package, or the API reference.

Development

git clone https://github.com/jose-pr/pkgforge && cd pkgforge
python -m venv .venv/3.14-posix-$(uname -m) && . .venv/3.14-posix-$(uname -m)/bin/activate
pip install -e ".[dev,docs]"

black src tests benchmarks      # format (Python 3.10+)
pytest -q                       # tests
python benchmarks/run.py        # benchmarks (add --save to record; see `benchmarks/README.md`)
mkdocs serve                    # docs preview at http://127.0.0.1:8000

Name the venv <version>-<os>-<arch> (<os> is posix/nt/darwin) if you keep more than one interpreter around, e.g. to also test the >=3.9 floor.

Releasing

This project follows Semantic Versioning and keeps a CHANGELOG.md. Pushing a tag matching v* triggers the release workflow: test gate → build (checking the tag names the version built) → a strict docs build as a gate → GitHub release → publish. The release workflow never deploys the docs site itself: for a final release its last job dispatches the docs workflow at the tag, which owns every Pages deploy. Before 1.0, a MINOR version bump means the documented API broke; everything else (fixes, additions) is a PATCH.

tests/smoke_installed.py is not collected by pytest. Run it with a non-editable install's interpreter (e.g. from a built wheel in a fresh venv) to check the installed CLI surface (--version, --help, completion, the example) and the shipped files (AGENTS.md, README.md, py.typed):

python -m build --wheel --outdir dist
python -m venv /tmp/smoke && /tmp/smoke/bin/pip install dist/*.whl
/tmp/smoke/bin/python tests/smoke_installed.py

License

MIT — see LICENSE.