lockstep

Does what's actually installed match the lockfile? Runtime dependency drift, not a commit-time CVE scan.

carabiner's deps engine answers "does the lockfile committed to this repo have a known CVE" -- a question about the source tree, checked in CI. lockstep answers a different one: does the environment actually running right now -- the container, the server, the venv someone SSHed into and patched by hand -- still match what that lockfile says should be there. A CVE scan at commit time can't see drift that happens after deploy; this is built specifically to.

$ lockstep check requirements.lock --quiet-matched
[!=] 'click' is pinned to 8.1.8 but 8.1.7 is installed
[--] 'requests' is pinned to 2.32.3 but is not installed
[++] 'six' 1.16.0 is installed but not declared anywhere in the lockfile

3/6 matched, 3 drifted

And the drift a version number can't show -- installed code that was edited in place after install:

$ lockstep verify
[!=] click 8.1.7: click/core.py was modified after install

5/6 package(s) intact, 529 file(s) checked

(Both real output, from a real venv. The lockfile also pinned uvloop ; sys_platform != "win32" -- correctly not expected on the Windows machine this ran on, so not reported missing.)

Install

pip install lockstep-evidence   # the command it installs is `lockstep`

(lockstep was already taken on PyPI, so the package is lockstep-evidence; the command is still lockstep.)

Use

Run it inside the environment you want to verify -- the container at runtime, the deployed server, the venv itself -- against a lockfile:

lockstep check requirements.lock

It reads the running interpreter's own installed-package metadata (importlib.metadata, stdlib, no subprocess) and compares it against the lockfile. --json prints the full machine-readable report; --quiet-matched hides packages that already match, showing only drift.

Or check a different environment without installing anything into it:

lockstep check uv.lock --python .venv/bin/python

--python runs a stdlib-only probe in that interpreter and judges every environment marker against its platform and Python, not lockstep's.

A container image works the same way, with nothing installed into it:

$ lockstep check requirements.lock --image myapp:latest
[!=] 'starlette' is pinned to 1.6.0 but 0.40.0 is installed
[++] 'six' 1.17.0 is installed but not declared anywhere in the lockfile

8/10 matched, 2 drifted

(Real output: an image built from fastapi's own locked requirements, then one built on top of it with a downgrade and a stray package. The clean image reads 9/9.) --image runs the probe with docker run --rm --network none, trying python3 then python.

The probe ignores the working directory: python -c puts it on sys.path, and a source checkout's stray *.egg-info there used to read as an installed package -- caught when lockstep reported itself "installed" in a venv it had never been installed into.

Checked on real projects: after uv sync --frozen --no-default-groups, fastapi's and flask's own uv.lock read 9/9 and 8/8 matched, and a downgraded starlette or a stray six exits 1. With extras, the uv.lock walk names the same 42 packages at the same versions as uv export --format pylock.toml.

lockstep verify: was installed code edited in place?

Every wheel install writes a RECORD -- each file it put down, with its sha256. pip writes it and never reads it again. So a file in site-packages hand-patched on a live server, "fixed" during a debugging session and left that way, or tampered with after install still reports the same version to pip, to pip freeze, and to lockstep check. The version string didn't change; the code did.

lockstep verify re-hashes every installed file against its own RECORD entry and names each one that's modified or missing. lockstep verify requests urllib3 checks just those. It's fast -- hashing is the only work (about 500 files in well under a second here) -- so it fits in a container's startup or a deploy's health check. A package installed without a RECORD (a legacy egg-style editable install) is listed as uncheckable, never counted as intact.

What it doesn't prove: that the original install was the real package. A RECORD is written by whoever installed it; if that install came from a compromised index, it'll describe the compromised files faithfully. Pair it with hash-pinned installs (pip install --require-hashes) for that.

Three kinds of drift, not one

extra is the headline case: the hotfix someone pip installed directly on a live server that never made it back into the lockfile, silently outliving the deploy that was supposed to make it official. A commit-time scan of the lockfile alone can never see this -- by the time it exists, the running environment and the committed lockfile have already diverged.

Versions are compared the way pip compares them (PEP 440, via packaging): 1.0 and 1.0.0 match, and so do 2.0RC1 and 2.0rc1. A === pin is the exception PEP 440 defines: exact string equality.

pip, setuptools, wheel and distribute are never reported as extra, because pip freeze leaves them out of the lockfile it writes. If your lockfile does pin one of them, it's checked like anything else.

Exit code is 1 if anything drifted, 0 if every declared package matches.

Lockfile formats

• requirements-style (pip freeze, pip-compile, uv pip compile, poetry export): ==/=== pins, extras, hashes, and environment markers -- a pin whose marker doesn't hold here (colorama ; sys_platform == "win32" on Linux) isn't expected, so it's never a false "missing".
• pylock.toml (PEP 751, the standard lockfile format), with per-package markers.
• Pipfile.lock: the default packages, i.e. what pipenv install --deploy puts in production.
• uv.lock, read directly: walked from the workspace members along the edges whose markers hold here, the way uv sync --no-dev installs it. --group dev and --extra NAME add groups and extras (repeatable). Note that [tool.uv] default-groups lives in pyproject.toml, so name those groups if the environment was synced with them.
• An SBOM (CycloneDX or SPDX JSON, any *.json): the pkg:pypi/... packages it lists, so the question becomes "is this environment what our SBOM says?". OS and npm packages in the same SBOM are not this check's business. A Python-only CycloneDX SBOM's local installs (no package URL) are expected present, version unchecked -- that is how cyclonedx-py environment records an editable project. Checked against a real cyclonedx-py SBOM of fastapi's environment: 9/9.
• poetry.lock isn't read directly; lockstep prints the export command instead: poetry export -f requirements.txt.

The lockfile must be fully pinned: every direct and transitive dependency, the way all of the above produce it. A hand-written requirements.txt listing only top-level packages makes every transitive dependency read as extra -- correct, technically, but noise.

Not the same question as pip check

pip check already ships with pip, and answers the obvious first question: are the packages currently installed internally consistent with each other's own declared requirements (nothing needs foo>=2.0 while foo 1.0 is what's actually there)? It never reads a lockfile at all -- it can't tell you whether reality matches what you intended to have installed, only whether what's installed is self-consistent right now. An environment can pass pip check cleanly while still being three versions behind its own lockfile, or running a hotfixed package the lockfile has never heard of -- exactly lockstep's extra case, above.

The two are complementary, not competing: pip check catches a broken dependency graph; lockstep check catches drift from your own declared source of truth. Worth running both -- neither substitutes for the other, and pip check costs nothing extra since it's already there.

What this does NOT do

• Python only, via pip/PyPI metadata. One ecosystem done correctly beats three done shallowly. npm/Cargo/etc. runtime drift is the same idea, applied to a different package manager; not built here.
• No CVE data, no severity, no vulnerability database. That's carabiner's deps engine's job already, at commit time. This only asks "does reality match the declaration," not "is the declared version itself safe" -- genuinely different questions, and conflating them would duplicate an engine that already exists and does that one well.
• Doesn't distinguish a top-level extra from a transitive one. Real dependency-tree resolution (which package pulled in which) is pip-compile-level complexity; out of scope for what's meant to be a small, fast check.

Tests

pip install -e .
python tests/test_names.py       # PEP 503 name normalization
python tests/test_parse.py       # lockfile parsing -- extras, ===, hashes, markers
python tests/test_installed.py   # reads THIS interpreter's real installed packages
python tests/test_check.py       # matched/version_mismatch/missing/extra classification
python tests/test_cli.py         # the real CLI, against this real environment
python tests/test_verify.py      # RECORD verification, pylock.toml, Pipfile.lock, markers

35 tests. Several exist because testing against real pip-compile output shapes (not synthetic "package==1.2.3" lines) caught real parsing bugs during development: PEP 440's === arbitrary-equality operator being mangled by a regex that matched == first and left a stray = on the version, and package[extra]==1.2.3 syntax needing to be handled explicitly rather than assumed away.

MIT licensed.