pl_vendor

pl_vendor copies Git repositories or selected upstream files and directories into another repository as ordinary files and records their exact commits in a deterministic YAML lockfile. It supports optional downstream patches and can verify that checked-in vendor trees still match their locked upstream commits.

Requirements

• Python 3.13 or newer
• Git 2.30 or newer

Rich provides terminal progress and styling, while TOMLKit preserves unrelated content and comments when pl-vendor edits pyproject.toml. Versioned Draft 2020-12 schemas for manifests and lockfiles ship in every distribution.

Every command checks the installed Git version before reading or changing vendored trees.

Configuration

Initialize a repository before adding packages:

pl-vendor init

This idempotently adds [tool.pl-vendor] schema 1 to pyproject.toml. It preserves unrelated content and comments, creates pyproject.toml when absent, and reports a no-op when the current schema is already present. For a 0.4 manifest, it atomically replaces required-version with schema-version = 1 and removes the obsolete run-uv-sync setting. This is safe to run immediately after uv add --dev pl_vendor, while pyproject.toml still has uncommitted package changes.

Check without changing files:

pl-vendor init --check

It exits 0 for schema 1, 1 for missing or migratable 0.4 configuration, and 2 for malformed, pre-0.4, or future configuration. Other commands do not migrate implicitly; they direct 0.4 users to init.

Add a package interactively:

pl-vendor add

Or provide its settings as arguments (the path defaults to ./.plvendor/NAME and an omitted branch tracks the repository's default branch):

pl-vendor add example https://github.com/example/example.git

The default destination is relative to the project root that uv would discover from the current directory. In a workspace, this places it under the workspace root. This discovery reads local pyproject.toml files and does not require the uv executable.

Packages can instead be pinned to an exact commit. Use --subdirectory to export only one directory from an upstream repository:

pl-vendor add prairielearn-schemas \
  https://github.com/PrairieLearn/PrairieLearn.git \
  .prairielearn/schemas \
  --commit 2c60f70a0ef817a2dc33fe15cb8e56de5035d6ed \
  --subdirectory apps/prairielearn/src/schemas/schemas

This creates or updates pyproject.toml at the root of the consuming Git repository, fetches the package, and records its revision. The resulting manifest looks like:

[tool.pl-vendor]
schema-version = 1

[tool.pl-vendor.packages.example]
path = "./.plvendor/example"
url = "https://github.com/example/example.git"

PrairieLearn compatibility checks use the latest commit on the official PrairieLearn/PrairieLearn master branch by default. A fork or deployment branch can be configured independently of vendored packages:

[tool.pl-vendor.prairielearn]
url = "https://github.com/example/PrairieLearn.git"
branch = "production"

Both fields are optional and fall back independently to the official URL and master. pl-vendor init leaves these defaults implicit.

The manifest schema and installed package version are independent. pl-vendor 0.5 reads schema 1 and writes lockfile schema 2; it recognizes an implicit 0.4 manifest only for init migration. See the schema changelog and migration guide, including exact standalone 0.3 instructions.

A commit-pinned subdirectory package looks like this:

[tool.pl-vendor.packages.prairielearn-schemas]
path = ".prairielearn/schemas"
url = "https://github.com/PrairieLearn/PrairieLearn.git"
commit = "2c60f70a0ef817a2dc33fe15cb8e56de5035d6ed"
subdirectory = "apps/prairielearn/src/schemas/schemas"

Use sources when one logical package consists of several files or directories from the same upstream revision. Repeat --source on the command line:

pl-vendor add prairielearn-htmlmustache \
  https://github.com/PrairieLearn/PrairieLearn.git \
  .prairielearn/htmlmustache \
  --branch master \
  --source .htmlmustache.jsonc \
  --source apps/prairielearn/src/lib/element-schemas/htmlmustache-plugin.ts

The equivalent manifest is:

[tool.pl-vendor]
schema-version = 1

[tool.pl-vendor.packages.prairielearn-htmlmustache]
path = ".prairielearn/htmlmustache"
url = "https://github.com/PrairieLearn/PrairieLearn.git"
branch = "master"
sources = [
  ".htmlmustache.jsonc",
  "apps/prairielearn/src/lib/element-schemas/htmlmustache-plugin.ts",
]

Unlike subdirectory, which strips the selected directory itself, sources preserves every repository-relative path beneath the destination. The example installs .prairielearn/htmlmustache/.htmlmustache.jsonc and .prairielearn/htmlmustache/apps/prairielearn/src/lib/element-schemas/htmlmustache-plugin.ts. Each source may be a file, symbolic link, or directory. Source paths must be nonempty, relative POSIX paths that do not contain ..; duplicates and ancestor/descendant overlaps are rejected. A source must exist as a file or directory at the resolved revision. sources must be nonempty and cannot be combined with subdirectory.

Prefer one multi-source package over several entries for one logical upstream artifact: all selected paths then resolve, lock, update, patch, and verify at one atomic commit.

Filtering selected files

Use package-level include and exclude lists to filter the assembled upstream tree. For example, vendor the element schemas without their TypeScript tests:

[tool.pl-vendor]
schema-version = 1

[tool.pl-vendor.packages.htmlmustache]
url = "https://github.com/PrairieLearn/PrairieLearn.git"
path = ".htmlmustache"
subdirectory = "apps/prairielearn/src/lib/element-schemas"
exclude = ["**/*.test.ts"]

The equivalent command is:

pl-vendor add htmlmustache https://github.com/PrairieLearn/PrairieLearn.git \
  .htmlmustache --subdirectory apps/prairielearn/src/lib/element-schemas \
  --exclude '**/*.test.ts'

To install only .htmlmustache/htmlmustache-plugin.ts, set include = ["htmlmustache-plugin.ts"] in that package table, or use:

pl-vendor add htmlmustache https://github.com/PrairieLearn/PrairieLearn.git \
  .htmlmustache --subdirectory apps/prairielearn/src/lib/element-schemas \
  --include 'htmlmustache-plugin.ts'

This selects only that file; imported dependencies are not discovered automatically. Repeat --include and --exclude for multiple patterns. Quote shell globs so the shell does not expand them against local files.

Patterns match paths relative to the assembled vendor tree, before downstream patches. subdirectory strips its upstream prefix, so the examples use htmlmustache-plugin.ts. With sources = ["apps/schemas"], paths retain their repository prefix, so an include would be apps/schemas/plugin.ts. The destination prefix (such as .htmlmustache/) is never part of a pattern. sources and subdirectory remain literal upstream paths, not globs.

Matching is case-sensitive and follows Python 3.13 PurePosixPath.full_match: *, ?, and bracket classes match within path components; ** spans zero or more directories. In particular, **/*.test.ts matches both root.test.ts and elements/nested.test.ts. Hidden entries participate in matching.

• Omitted include selects everything; otherwise its patterns are combined as a union. Exclusions always win, regardless of ordering.
• A matched directory includes or excludes its entire subtree. A trailing / restricts a pattern to directories: tests/ excludes the root test directory, while **/tests/ excludes test directories at any depth. Necessary parent directories remain; empty directories are pruned when filters are configured.
• Individual unmatched patterns are allowed. If filtering leaves no upstream files or symlinks, the operation fails before replacing that package, even if a patch could add files. include = [] is invalid; exclude = [] is allowed.
• Patterns must be nonempty relative POSIX strings. Absolute paths, backslashes, NULs, empty path components, and . or .. components are rejected.
• Symlinks are matched as leaf entries, preserved verbatim, and never traversed. A directory-only pattern does not match a symlink pointing to a directory.

Filtering happens after source assembly and before patches. Patches that modify excluded files fail; patches may explicitly add files whose paths match exclusions. There is no second filtering pass after patches. Add, sync, update, check, overwrite checks, and content-based upstream checks all use this same selected, patched tree. Unexpected files in the installed tree still cause verification to fail.

Filtering does not change the upstream revision in the lockfile.

When changing an existing package's filters, commit the manifest change, then run pl-vendor sync htmlmustache. If the installed tree differs from the new selection, the normal overwrite protection requires pl-vendor sync htmlmustache --force. Review local edits before forcing replacement. Sync retains the locked commit and leaves unrelated packages alone.

Packages track the repository's default branch when both branch and commit are omitted. Set branch to track a specific branch, or commit to pin a complete lowercase 40-character Git SHA; branch and commit are mutually exclusive. subdirectory is relative to the upstream repository and defaults to its root.

Destination paths are canonicalized before comparison. Exact, normalized, case-equivalent, and ancestor/descendant collisions are rejected, as are .git, .vendor-patches, pyproject.toml, plvendor-lock.yaml, their case variants, and paths beneath reserved directories. Resolved paths must remain inside the repository; symlink escapes and non-directory destinations are rejected.

The generated plvendor-lock.yaml records the resolved commit:

lockfileVersion: 2

packages:
  example: '0123456789abcdef0123456789abcdef01234567'

Optional patches live at .vendor-patches/<package-name>.patch and are applied only after the complete selected tree is assembled. Patch paths are relative to the installed vendored tree: stripped paths for a legacy subdirectory package and repository-relative paths for a sources package.

Commands

pl-vendor help
pl-vendor --version
pl-vendor list
pl-vendor list example first
pl-vendor init
pl-vendor init --check
pl-vendor self-update
pl-vendor update
pl-vendor update example second
pl-vendor update example second --force
pl-vendor add example https://github.com/example/example.git --branch release
pl-vendor remove example
pl-vendor remove first second
pl-vendor lock
pl-vendor lock --check
pl-vendor lock --upgrade
pl-vendor lock --upgrade-package example --upgrade-package second
pl-vendor sync
pl-vendor sync example second
pl-vendor sync example second --force
pl-vendor check installed
pl-vendor check installed example second
pl-vendor check branches
pl-vendor check upstream content
pl-vendor check upstream revision example
pl-vendor check latest content
pl-vendor check latest revision
pl-vendor check prairielearn python

pl-vendor --version prints the installed tool version. pl-vendor list [PACKAGE ...] prints deterministic package names, full locked revisions, and paths in manifest or requested order. version is a deprecated alias for list through 0.5 and is removed in 0.6.

pl-vendor self-update is deprecated through 0.5 and removed in 0.6. It no longer queries PyPI, prompts, or writes exact version constraints. It preserves an existing dev requirement's extras and marker, removes its version specifier, and delegates a targeted upgrade to uv. Use the native command directly:

uv add --dev --raw pl-vendor --upgrade-package pl-vendor

Direct-URL requirements are not rewritten.

add, remove, update, and sync resolve, fetch, patch, and validate every selected package before changing the repository. They then replace all trees and metadata as one rollback-protected transaction, including interruption and final-validation rollback. Remove accepts one or more package names and deletes their complete managed trees, including local or generated content. Removing the final package removes the packages table and lockfile while preserving the base [tool.pl-vendor] configuration and the rest of pyproject.toml; downstream patches are retained.

pl-vendor lock fills missing lock entries while preferring existing revisions. Pass --upgrade or repeat --upgrade-package NAME to explicitly advance branch packages; commit-pinned packages never advance. A commit pin is authoritative, so lock reconciles a stale lock entry to the manifest without contacting the remote. This does not modify vendored files.

pl-vendor update advances and syncs zero or more named packages in one step; no names means all. sync accepts the same multi-package selection. A package literally named all is selectable; the old all spelling means every package only when no such package exists. For a pinned package, it installs the manifest commit and updates the lockfile to that exact value without rewriting the manifest. If the manifest exists but the lockfile does not, update bootstraps a complete lockfile from the manifest instead of requiring a separate lock step. Because the lockfile is generated state, update also replaces it when it has uncommitted changes; uncommitted pyproject.toml changes remain protected.

When a lockfile exists, update and sync compare each selected vendored tree with its locked, patched upstream contents and refuse to overwrite any differences. This protects edits whether or not they are committed to the consuming repository. Pass --force to replace those edits explicitly. Destinations that already exactly match the requested upstream trees are left untouched; when neither the selected trees nor the lockfile changes, the command reports No changes were made.

pl-vendor sync does not update the lockfile: it replaces the selected vendored trees with their exact locked revisions and reapplies downstream patches. Untracked extras are retained for comparison only when Git marks them ignored and they are absent upstream. Tracked extras and upstream-required or tracked files are always compared, even when an ignore rule matches. The same Git-aware comparison drives checks, overwrite protection, and no-op detection. Replacing other local differences requires --force.

During update, sync, and check, entries from the same source share a temporary blobless Git object store. Each revision is checked out once, using a sparse worktree containing only the configured upstream source paths unless an entry requests the repository root. This avoids fetching unrelated file contents and reuses unchanged objects when entries select different commits from the same source. Branch lookups are also batched per source URL.

Fetches from different sources run concurrently, with a progress bar for each source revision in an interactive terminal. Revisions from one source run in sequence so they can safely share their Git object store. The concurrency limit defaults to 50, matching uv's download limit. Set concurrent-downloads under [tool.pl-vendor] in the consuming project's pyproject.toml to change it.

Like uv, pl-vendor writes human-readable operation status to standard error and keeps standard output empty. In an interactive terminal, add, update, sync, and check show transient fetch and install progress. Redirected output retains only deterministic phase summaries and package outcomes:

Fetched 2 source revisions in 1.24s
Installed 1 package in 84ms
 + first abcdef12 -> vendor/first
 ~ second 12345678 -> vendor/second (patched)
 ~ third 87654321 -> vendor/third

Fetch counts represent unique source URL and revision pairs, while install counts represent destinations that were created or updated. When a check needs both locked and branch-head contents, it includes them in one fetch phase and summary. Every selected package gets an outcome record: + marks an update, ~ marks an unchanged destination, and - marks an error. Revisions are shortened to eight characters for display; the lockfile remains authoritative for complete revisions.

The vendored-package pl-vendor check modes verify that each selected installed tree exactly matches its locked, selected, and patched upstream contents. Package names follow the complete check mode; with no package names, they check all packages. check installed performs only this reproducibility check, while check branches also rejects commit pins.

check upstream content additionally compares each branch package's locked contents with its branch head while allowing pins. An upstream commit that changes only files outside the configured selection does not fail this check. check upstream revision instead requires the locked commit SHA to equal the branch head. The corresponding check latest content and check latest revision modes perform the same comparisons but reject pinned packages. Checks never modify installed files or the lockfile.

check prairielearn python is a separate compatibility-only check. It resolves the latest commit on the configured PrairieLearn branch, sparsely fetches its root pyproject.toml, and checks the consuming project's root [project].dependencies against PrairieLearn's exact direct dependency pins. A local requirement passes when PrairieLearn declares the same normalized package name and its pinned version satisfies the local version constraint. A prairielearn requirement, such as a course library's dependency on PrairieLearn's Python package, is accepted as PrairieLearn itself. Missing packages and incompatible versions are reported together and produce exit status 1.

This check ignores local optional dependencies and dependency groups. Environment markers, extras, and direct URL requirements are rejected because their runtime support cannot be established from PrairieLearn's direct pins alone. The command does not read plvendor-lock.yaml or inspect installed vendored trees; use check installed for that purpose. It requires an initialized [tool.pl-vendor] configuration, but it does not require any configured vendored packages.

In 0.5, bare check becomes a usage error. Replace its former exact branch-head check with check upstream revision, which additionally verifies installed contents. verify [PACKAGE ...] and require-latest [PACKAGE ...] remain as deprecated, warning aliases through 0.5 and will be removed in 0.6. Use check installed instead of verify; replace require-latest with check latest content, or use check latest revision when the former --exact-revision option was used. The deprecated aliases retain their original check scope and exit behavior during the migration release.

Repository commands discover the consuming repository from the current directory. Common --root, --no-progress, and --color options are accepted before the command or after its complete leaf command, such as --root PATH sync or check latest content --root PATH. Pass --no-progress to suppress transient animation without hiding the durable summaries. --color auto|always|never controls styling; auto is the default and also honors standard terminal color environment variables such as NO_COLOR.

The canonical add form is add NAME URL [DESTINATION] --branch BRANCH. The redundant --name, --url, and --path options and positional branch form warn through 0.5 for compatibility.

Python API

The supported facade exports VendorProject, immutable VendorPackage, OperationResult, PackageResult, CheckResult, VendorError, and __version__. Pass a VendorPackage to VendorProject.add(). Library operations return structured results and default to silent in-memory output; the CLI owns terminal rendering and progress. Failed checks return their primary diagnostics in CheckResult.issues in reporting order; repair guidance and Git diff bodies remain presentation output rather than structured issues. Raw TOML/lock mutation, Git backends, reporting helpers, and PrairieLearn constants are internal implementation details.

Development

This repository uses uv 0.9 or newer to manage its development environment:

uv sync --dev
make test
make format
make build

The test suite is kept in the top-level tests directory so it is separate from the installable pl_vendor package.

To run a disposable walkthrough of the main workflow backed by local Git repositories:

./scripts/demo.sh

Pass --keep to retain the generated upstream and consumer repositories for inspection.

To publish a release, start from a clean working tree and run:

make publish-version VERSION=0.5.1

This runs the test suite and static checks, updates pyproject.toml and uv.lock, builds the distributions, creates a release commit and annotated v0.5.1 tag, and atomically pushes both to origin. The tag starts the PyPI publishing workflow on GitHub-hosted runners. Use REMOTE=name to publish through a different Git remote. The requested version must be semantically newer than the current package version and all local or remote release tags.

If pyproject.toml and uv.lock already contain the target version in an unpushed commit, publish that commit without creating a redundant release commit:

make publish-version VERSION=0.5.1 ALLOW_PREBUMPED_VERSION=1

This bypass requires the requested version to equal the current project version, the release tag to be absent locally and remotely, and the local branch to be ahead of its remote branch.

To publish an existing tag whose earlier workflow did not upload to PyPI, first make sure this workflow is present on main, then run:

gh workflow run publish.yml --ref main -f release_tag=v0.5.1

The manually supplied tag must match both the release-tag syntax and the version in that tag's pyproject.toml.