pl_vendor

pl_vendor copies Git repositories or selected upstream files and directories into another repository as ordinary files and records their exact commits in a deterministic YAML lockfile. It supports optional downstream patches and can verify that checked-in vendor trees still match their locked upstream commits.

Requirements

• Python 3.13 or newer
• Git 2.30 or newer

Rich provides terminal progress and styling, while TOMLKit preserves unrelated content and comments when pl-vendor edits pyproject.toml.

Every command checks the installed Git version before reading or changing vendored trees.

Configuration

Initialize a repository before adding packages:

pl-vendor init

This adds [tool.pl-vendor] to pyproject.toml with a required-version requirement for the running minor version (for example, ^0.4). It preserves unrelated content and comments, creates pyproject.toml when absent, and refuses to replace an existing [tool.pl-vendor] table. This is safe to run immediately after uv add --dev pl_vendor, while pyproject.toml still has uncommitted package changes.

Add a package interactively:

pl-vendor add

Or provide its settings as arguments (the path defaults to ./.plvendor/NAME and an omitted branch tracks the repository's default branch):

pl-vendor add example https://github.com/example/example.git

The default destination is relative to the project root that uv would discover from the current directory. In a workspace, this places it under the workspace root. This discovery reads local pyproject.toml files and does not require the uv executable.

Packages can instead be pinned to an exact commit. Use --subdirectory to export only one directory from an upstream repository:

pl-vendor add prairielearn-schemas \
  https://github.com/PrairieLearn/PrairieLearn.git \
  .prairielearn/schemas \
  --commit 2c60f70a0ef817a2dc33fe15cb8e56de5035d6ed \
  --subdirectory apps/prairielearn/src/schemas/schemas

This creates or updates pyproject.toml at the root of the consuming Git repository, fetches the package, and records its revision. The resulting manifest looks like:

[tool.pl-vendor]
required-version = "^0.4"

[tool.pl-vendor.packages.example]
path = "./.plvendor/example"
url = "https://github.com/example/example.git"

The required-version requirement uses caret semantics. ^0.4 accepts compatible 0.4.x versions of pl-vendor and rejects versions from another minor release. Version 0.4 reads vendoring configuration only from this pyproject.toml namespace; the former standalone plvendor.toml format is not loaded.

A commit-pinned subdirectory package looks like this:

[tool.pl-vendor.packages.prairielearn-schemas]
path = ".prairielearn/schemas"
url = "https://github.com/PrairieLearn/PrairieLearn.git"
commit = "2c60f70a0ef817a2dc33fe15cb8e56de5035d6ed"
subdirectory = "apps/prairielearn/src/schemas/schemas"

Use sources when one logical package consists of several files or directories from the same upstream revision. Repeat --source on the command line:

pl-vendor add prairielearn-htmlmustache \
  https://github.com/PrairieLearn/PrairieLearn.git \
  .prairielearn/htmlmustache \
  --branch master \
  --source .htmlmustache.jsonc \
  --source apps/prairielearn/src/lib/element-schemas/htmlmustache-plugin.ts

The equivalent manifest is:

[tool.pl-vendor]
required-version = "^0.4.6"

[tool.pl-vendor.packages.prairielearn-htmlmustache]
path = ".prairielearn/htmlmustache"
url = "https://github.com/PrairieLearn/PrairieLearn.git"
branch = "master"
sources = [
  ".htmlmustache.jsonc",
  "apps/prairielearn/src/lib/element-schemas/htmlmustache-plugin.ts",
]

Unlike subdirectory, which strips the selected directory itself, sources preserves every repository-relative path beneath the destination. The example installs .prairielearn/htmlmustache/.htmlmustache.jsonc and .prairielearn/htmlmustache/apps/prairielearn/src/lib/element-schemas/htmlmustache-plugin.ts. Each source may be a file, symbolic link, or directory. Source paths must be nonempty, relative POSIX paths that do not contain ..; duplicates and ancestor/descendant overlaps are rejected. A source must exist as a file or directory at the resolved revision. sources must be nonempty and cannot be combined with subdirectory.

Multi-source packages require pl_vendor 0.4.6 or newer. pl-vendor add --source automatically raises an existing required-version = "^0.4" floor to ^0.4.6. Existing packages using subdirectory remain compatible and do not need a manifest change. Prefer one multi-source package over several entries for one logical upstream artifact: all selected paths then resolve, lock, update, patch, and verify at one atomic commit.

Filtering selected files

Use package-level include and exclude lists to filter the assembled upstream tree. For example, vendor the element schemas without their TypeScript tests:

[tool.pl-vendor]
required-version = "^0.4.7"

[tool.pl-vendor.packages.htmlmustache]
url = "https://github.com/PrairieLearn/PrairieLearn.git"
path = ".htmlmustache"
subdirectory = "apps/prairielearn/src/lib/element-schemas"
exclude = ["**/*.test.ts"]

The equivalent command is:

pl-vendor add htmlmustache https://github.com/PrairieLearn/PrairieLearn.git \
  .htmlmustache --subdirectory apps/prairielearn/src/lib/element-schemas \
  --exclude '**/*.test.ts'

To install only .htmlmustache/htmlmustache-plugin.ts, set include = ["htmlmustache-plugin.ts"] in that package table, or use:

pl-vendor add htmlmustache https://github.com/PrairieLearn/PrairieLearn.git \
  .htmlmustache --subdirectory apps/prairielearn/src/lib/element-schemas \
  --include 'htmlmustache-plugin.ts'

This selects only that file; imported dependencies are not discovered automatically. Repeat --include and --exclude for multiple patterns. Quote shell globs so the shell does not expand them against local files.

Patterns match paths relative to the assembled vendor tree, before downstream patches. subdirectory strips its upstream prefix, so the examples use htmlmustache-plugin.ts. With sources = ["apps/schemas"], paths retain their repository prefix, so an include would be apps/schemas/plugin.ts. The destination prefix (such as .htmlmustache/) is never part of a pattern. sources and subdirectory remain literal upstream paths, not globs.

Matching is case-sensitive and follows Python 3.13 PurePosixPath.full_match: *, ?, and bracket classes match within path components; ** spans zero or more directories. In particular, **/*.test.ts matches both root.test.ts and elements/nested.test.ts. Hidden entries participate in matching.

• Omitted include selects everything; otherwise its patterns are combined as a union. Exclusions always win, regardless of ordering.
• A matched directory includes or excludes its entire subtree. A trailing / restricts a pattern to directories: tests/ excludes the root test directory, while **/tests/ excludes test directories at any depth. Necessary parent directories remain; empty directories are pruned when filters are configured.
• Individual unmatched patterns are allowed. If filtering leaves no upstream files or symlinks, the operation fails before replacing that package, even if a patch could add files. include = [] is invalid; exclude = [] is allowed.
• Patterns must be nonempty relative POSIX strings. Absolute paths, backslashes, NULs, empty path components, and . or .. components are rejected.
• Symlinks are matched as leaf entries, preserved verbatim, and never traversed. A directory-only pattern does not match a symlink pointing to a directory.

Filtering happens after source assembly and before patches. Patches that modify excluded files fail; patches may explicitly add files whose paths match exclusions. There is no second filtering pass after patches. Add, sync, update, check, overwrite checks, and content-based upstream checks all use this same selected, patched tree. Unexpected files in the installed tree still cause verification to fail.

Filters require pl_vendor 0.4.7 or newer. add --include or add --exclude automatically raises the manifest's minimum required-version to ^0.4.7 when needed; manual edits must set this floor explicitly. Self-update preserves the feature minimum. Existing manifests without filters and lockfile version 2 retain their behavior, and filtering does not change the upstream revision in the lockfile.

When changing an existing package's filters, commit the manifest change, then run pl-vendor sync htmlmustache. If the installed tree differs from the new selection, the normal overwrite protection requires pl-vendor sync htmlmustache --force. Review local edits before forcing replacement. Sync retains the locked commit and leaves unrelated packages alone.

Packages track the repository's default branch when both branch and commit are omitted. Set branch to track a specific branch, or commit to pin a complete lowercase 40-character Git SHA; branch and commit are mutually exclusive. subdirectory is relative to the upstream repository and defaults to its root.

The generated plvendor-lock.yaml records the resolved commit:

lockfileVersion: 2

packages:
  example: '0123456789abcdef0123456789abcdef01234567'

Optional patches live at .vendor-patches/<package-name>.patch and are applied only after the complete selected tree is assembled. Patch paths are relative to the installed vendored tree: stripped paths for a legacy subdirectory package and repository-relative paths for a sources package.

Commands

pl-vendor help
pl-vendor version
pl-vendor version example
pl-vendor version first second
pl-vendor init
pl-vendor self-update
pl-vendor update
pl-vendor update example
pl-vendor update example --force
pl-vendor add example https://github.com/example/example.git --branch release
pl-vendor remove example
pl-vendor remove first second
pl-vendor lock
pl-vendor lock --check
pl-vendor lock --upgrade
pl-vendor lock --upgrade-package example
pl-vendor sync
pl-vendor sync example
pl-vendor sync example --force
pl-vendor check installed
pl-vendor check installed example second
pl-vendor check branches
pl-vendor check upstream content
pl-vendor check upstream revision example
pl-vendor check latest content
pl-vendor check latest revision

pl-vendor version prints the installed tool version. When package names are provided, it instead prints their eight-character locked revisions, one per line in the requested order.

pl-vendor self-update looks up the latest release on PyPI, pins an existing dependency-groups.dev entry to that exact version, and updates [tool.pl-vendor].required-version to its minor release. The dev entry is optional. When the entry exists and uv is available on PATH, it asks whether to run uv sync. Answer never to add run-uv-sync = false to the base [tool.pl-vendor] table and skip this prompt on future self-updates.

Like uv add and uv remove, pl-vendor add and pl-vendor remove update the manifest, lockfile, and installed files together. Remove accepts one or more package names and deletes their complete managed trees, including local or generated content. Removing the final package removes the [tool.pl-vendor] configuration and lockfile while preserving the rest of pyproject.toml; downstream patches are retained.

pl-vendor lock fills missing lock entries while preferring existing revisions. Pass --upgrade or --upgrade-package NAME to explicitly advance branch packages; commit-pinned packages never advance. A commit pin is authoritative, so lock reconciles a stale lock entry to the manifest without contacting the remote. This does not modify vendored files.

pl-vendor update remains a convenience command that advances branch packages and syncs one or all packages in one step. For a pinned package, it installs the manifest commit and updates the lockfile to that exact value without rewriting the manifest. If the manifest exists but the lockfile does not, update bootstraps a complete lockfile from the manifest instead of requiring a separate lock step. Because the lockfile is generated state, update also replaces it when it has uncommitted changes; uncommitted pyproject.toml changes remain protected.

When a lockfile exists, update and sync compare each selected vendored tree with its locked, patched upstream contents and refuse to overwrite any differences. This protects edits whether or not they are committed to the consuming repository. Pass --force to replace those edits explicitly. Destinations that already exactly match the requested upstream trees are left untouched; when neither the selected trees nor the lockfile changes, the command reports No changes were made.

pl-vendor sync does not update the lockfile: it replaces the selected vendored trees with their exact locked revisions and reapplies downstream patches. As with uv's default exact sync, generated files inside those managed trees are removed. Replacing other local differences requires --force.

During update, sync, and check, entries from the same source share a temporary blobless Git object store. Each revision is checked out once, using a sparse worktree containing only the configured upstream source paths unless an entry requests the repository root. This avoids fetching unrelated file contents and reuses unchanged objects when entries select different commits from the same source. Branch lookups are also batched per source URL.

Fetches from different sources run concurrently, with a progress bar for each source revision in an interactive terminal. Revisions from one source run in sequence so they can safely share their Git object store. The concurrency limit defaults to 50, matching uv's download limit. Set concurrent-downloads under [tool.pl-vendor] in the consuming project's pyproject.toml to change it.

Like uv, pl-vendor writes human-readable operation status to standard error and keeps standard output empty. In an interactive terminal, add, update, sync, and check show transient fetch and install progress. Redirected output retains only deterministic phase summaries and installed package records:

Fetched 2 source revisions in 1.24s
Installed 3 packages in 84ms
 + first abcdef12 -> vendor/first
 ~ second 12345678 -> vendor/second (patched)
 ~ third 87654321 -> vendor/third

Fetch counts represent unique source URL and revision pairs, while install counts represent destinations that were created or replaced. When a check needs both locked and branch-head contents, it includes them in one fetch phase and summary. Unchanged destinations are not included in the install summary or package records. + marks a newly created destination and ~ marks an existing destination that was replaced. Revisions are shortened to eight characters for display; the lockfile remains authoritative for complete revisions.

Every pl-vendor check mode verifies that each selected installed tree exactly matches its locked, selected, and patched upstream contents. Package names follow the complete check mode; with no package names, it checks all packages. check installed performs only this reproducibility check, while check branches also rejects commit pins.

check upstream content additionally compares each branch package's locked contents with its branch head while allowing pins. An upstream commit that changes only files outside the configured selection does not fail this check. check upstream revision instead requires the locked commit SHA to equal the branch head. The corresponding check latest content and check latest revision modes perform the same comparisons but reject pinned packages. Checks never modify installed files or the lockfile.

In 0.5, bare check becomes a usage error. Replace its former exact branch-head check with check upstream revision, which additionally verifies installed contents. verify [PACKAGE ...] and require-latest [PACKAGE ...] remain as deprecated, warning aliases through 0.5 and will be removed in 0.6. Use check installed instead of verify; replace require-latest with check latest content, or use check latest revision when the former --exact-revision option was used. The deprecated aliases retain their original check scope and exit behavior during the migration release.

Repository commands discover the consuming repository from the current directory. Pass --root PATH after the command to operate on a different repository. For check, common options belong after the complete leaf mode, such as check latest content --root PATH. Pass --no-progress to suppress transient animation without hiding the durable summaries. --color auto|always|never controls styling; auto is the default and also honors standard terminal color environment variables such as NO_COLOR.

Development

This repository uses uv 0.9 or newer to manage its development environment:

uv sync --dev
make test
make format
make build

The test suite is kept in the top-level tests directory so it is separate from the installable pl_vendor package.

To run a disposable walkthrough of the main workflow backed by local Git repositories:

./scripts/demo.sh

Pass --keep to retain the generated upstream and consumer repositories for inspection.

To publish a release, start from a clean working tree and run:

make publish-version VERSION=0.2.0

This ensures the Dockerized self-hosted publishing runner is online, runs the test suite and static checks, updates pyproject.toml and uv.lock, builds the distributions, creates a release commit and annotated v0.2.0 tag, and atomically pushes both to origin. The tag starts the PyPI publishing workflow. Use REMOTE=name to publish through a different Git remote. The requested version must be semantically newer than the current package version and all local or remote release tags.

If pyproject.toml and uv.lock already contain the target version in an unpushed commit, publish that commit without creating a redundant release commit:

make publish-version VERSION=0.4.0 ALLOW_PREBUMPED_VERSION=1

This bypass requires the requested version to equal the current project version, the release tag to be absent locally and remotely, and the local branch to be ahead of its remote branch.

Self-hosted publishing runner

Publishing uses a repository-scoped Linux ARM64 GitHub Actions runner in Docker so the workflow does not consume GitHub-hosted runner minutes. GitHub still orchestrates the workflow and provides its artifact storage, pypi environment, and short-lived OIDC credentials for PyPI trusted publishing. The container has no host filesystem mounts and does not receive the Docker socket.

Authenticate the GitHub CLI with an account that can administer this repository and start the runner with:

gh auth status
make runner-start

make publish-version runs runner-ensure automatically. On macOS, that command starts Docker Desktop when necessary, starts an existing runner container when possible, and rebuilds and registers it only when needed.

The runner can also be managed directly:

make runner-status
make runner-stop
make runner-start
make runner-remove
make clean-runner

Use runner-stop to stop the container while retaining its GitHub registration. Use runner-remove before discarding the container so the registration is removed too. clean-runner removes the registration along with the container, image, volume, and network.

To publish an existing tag whose earlier workflow did not upload to PyPI, first make sure this workflow is present on main, then run:

make runner-ensure
gh workflow run publish.yml --ref main -f release_tag=v0.4.6

The manually supplied tag must match both the release-tag syntax and the version in that tag's pyproject.toml.