Skip to main content

graphpatch 0.2.3

Documentation is hosted on Read the Docs.

Overview

graphpatch is a library for activation patching (often also referred to as “ablation”) on PyTorch neural network models. You use it by first wrapping your model in a PatchableGraph and then running operations in a context created by PatchableGraph.patch():

pg = PatchableGraph(model, **inputs, use_cache=False)
# Applies patches to the multiplication result within the activation function of the
# MLP in the 18th transformer layer. ProbePatch records the last observed value at the
# given node, while ZeroPatch zeroes out the value seen by downstream computations.
with pg.patch("transformer.h_17.mlp.act.mul_3": [probe := ProbePatch(), ZeroPatch()]):
   output = pg(**inputs)
# Patches are applied in order. probe.activation holds the value prior
# to ZeroPatch zeroing it out.
print(probe.activation)

In contrast to other approaches, graphpatch can patch (or record) any intermediate tensor value without manual modification of the underlying model’s code. See Working with graphpatch for some tips on how to use the generated graphs.

Note that graphpatch activation patches are compatible with AutoGrad! This means that, for example, you can perform optimizations over the value parameter to AddPatch:

delta = torch.zeros(size, requires_grad=True, device="cuda")
optimizer = torch.optim.Adam([delta], lr=0.5)
for _ in range(num_steps):
   with graph.patch({node_name: AddPatch(value=delta)):
      logits = graph(**prompt_inputs)
   loss = my_loss_function(logits)
   loss.backward()
   optimizer.step()

For a practical usage example, see the demo of using graphpatch to replicate ROME.

Prerequisites

The only mandatory requirements are torch>=2 and numpy>=1.17. Version 2+ of torch is required because graphpatch leverages torch.compile(), which was introduced in 2.0.0, to extract computational graphs from models. CUDA support is not required. numpy is required for full compile() support.

Python 3.8–3.12 are supported. Note that torch versions prior to 2.1.0 do not support compilation on Python 3.11, and versions prior to 2.4.0 do not support compilation on Python 3.12; you will get an exception when trying to use graphpatch with such a configuration. No version of torch yet supports compilation on Python 3.13.

Installation

graphpatch is available on PyPI, and can be installed via pip:

pip install graphpatch

Note that you will likely want to do this in an environment that already has torch, since pip may not resolve torch to a CUDA-enabled version by default. You don’t need to do anything special to make graphpatch compatible with transformers, accelerate, and bitsandbytes; their presence is detected at run-time. However, for convenience, you can install graphpatch with the “transformers” extra, which will install known compatible versions of these libraries along with some of their optional dependencies that are otherwise mildly inconvenient to set up:

pip install graphpatch[transformers]

Model compatibility

For full functionality, graphpatch depends on being able to call torch.compile() on your model. This currently supports a subset of possible Python operations–for example, it doesn’t support context managers. graphpatch implements some workarounds for situations that a native compile() can’t handle, but this coverage isn’t complete. To deal with this, graphpatch has a graceful fallback that should be no worse of a user experience than using module hooks. In that case, you will only be able to patch an uncompilable submodule’s inputs, outputs, parameters, and buffers. See Notes on compilation for more discussion.

transformers integration

graphpatch is theoretically compatible with any model in Huggingface’s transformers library, but note that there may be edge cases in specific model code that it can’t yet handle. For example, it is not (yet!) compatible with the key-value caching implementation, so if you want full compilation of such models you should pass use_cache=False as part of the example inputs.

graphpatch is compatible with models loaded via accelerate and with 8-bit parameters quantized by bitsandbytes. This means that you can run graphpatch on multiple GPU’s and/or with quantized inference very easily on models provided by transformers:

model = LlamaForCausalLM.from_pretrained(
   model_path,
   device_map="auto",
   quantization_config=BitsAndBytesConfig(load_in_8bit=True),
   torch_dtype=torch.float16,
)
pg = PatchableGraph(model, **example_inputs, use_cache=False)

For transformers models supporting the GenerationMixin protocol, you will also be able to use convenience functions like generate() in combination with activation patching:

# Prevent Llama from outputting "Paris"
with pg.patch({"lm_head.output": ZeroPatch(slice=(slice(None), slice(None), 3681))}):
   output_tokens = pg.generate(**inputs, max_length=20, use_cache=False)

Version compatibility

graphpatch should be compatible with all versions of optional libraries matching the minimum version requirements, but this is a highly ambitious claim to make for a Python library. If you end up with errors that seem related to graphpatch’s integration with these libraries, you might try changing their versions to those listed below. This list was automatically generated as part of the graphpatch release process. It reflects the versions used while testing graphpatch 0.2.3:

accelerate==1.0.0
bitsandbytes==0.44.1
numpy==1.24.4 (Python 3.8)
numpy==2.0.2 (Python 3.9)
numpy==2.1.1 (later Python versions)
sentencepiece==0.2.0
transformer-lens==2.4.1
transformers==4.45.2

Alternatives

Module hooks are built in to torch and can be used for activation patching. You can even add them to existing models without modifying their code. However, this will only give you access to module inputs and outputs; accessing or patching intermediate values still requires a manual rewrite.

TransformerLens provides the HookPoint class, which can record and patch intermediate activations. However, this requires manually rewriting your model’s code to wrap the values you want to make patchable.

TorchLens records and outputs visualizations for every intermediate activation. However, it is currently unable to perform any activation patching.

nnsight offers a nice activation patching API, but is limited to module inputs and outputs.

pyvene offers fine-grained control over activation patches (for example, down to a specific attention head), and a description language/serialization format to allow specification of reproducible experiments.

Documentation index

Metadata

Release files for graphpatch 0.2.3

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for graphpatch 0.2.3
File Size Uploaded Provenance Actions
graphpatch-0.2.3.tar.gz 59.1 kB Without provenance Details for graphpatch-0.2.3.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for graphpatch 0.2.3
File Interpreter ABI Platform Actions
graphpatch-0.2.3-py3-none-any.whl Python 3 none any Details for graphpatch-0.2.3-py3-none-any.whl

Total release size: 123.9 kB

Release files / graphpatch-0.2.3.tar.gz

Download URL graphpatch-0.2.3.tar.gz
Size 59.1 kB
Tags Source
SHA-256 checksum
How to use checksums
cc966043ed32ae0bd7d321438e8d19642667be14b4975f8d8842b3bbd7d4063b
BLAKE2b-256 checksum
How to use checksums
e569a1e3e44ca9043441ed0c928b2a7e531daf245a1bf521dc855c7b36d5ecb8
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via poetry/1.8.3 CPython/3.10.11 Darwin/23.6.0

Release files / graphpatch-0.2.3-py3-none-any.whl

Download URL graphpatch-0.2.3-py3-none-any.whl
Size 64.8 kB
Tags Python 3
SHA-256 checksum
How to use checksums
a736f607121bc4b9348bc95126eda60d5583fcf097935bb9da95979d88c137b5
BLAKE2b-256 checksum
How to use checksums
8956044cdd2b8d18872119e27fbc2d702926421818d1940a36c9bba4a711ead6
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via poetry/1.8.3 CPython/3.10.11 Darwin/23.6.0

Security for graphpatch 0.2.3

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page