packdiff

Pack a repository into a single, LLM-friendly Markdown file.

English · 简体中文

You are about to paste a codebase into an LLM. Which files? How big is the prompt? Did you just include .env?

packdiff answers all three. It walks a repository, skips the noise and the credentials, and writes one clean Markdown document — directory tree, language overview, and every text file in a fenced code block — with a real token estimate in the header.

packdiff . -o context.md                # pack the current repo
packdiff --max-tokens 32k -o ctx.md     # ...and fit it in a 32k window
packdiff --diff -o review.md            # ...or just what you have not committed
packdiff src/ docs/ --include '*.py'    # ...or just two trees, Python only

Install

pip install packdiff

Or with pipx if you would rather not put it inside a virtualenv:

pipx install packdiff

Python 3.9 or newer. No dependencies — standard library only.

Quick start

cd ~/projects/my-service
packdiff

# packdiff · repository context pack

> Generated by packdiff v0.3.0 on 2026-10-13 10:24 UTC.
> Roots: `my-service`
> 5 files · 103 B · ~215 tokens (estimated)

## Overview

| Language | Files | Lines |
| --- | ---: | ---: |
| markdown | 1 | 3 |
| python | 2 | 3 |
| gitignore | 1 | 2 |
| text | 1 | 1 |

## Directory tree

```text
my-service/
├── src/
│   ├── __init__.py
│   └── core.py
├── .gitignore
├── README.md
└── requirements.txt
```

## Files

### `.gitignore`
```gitignore
.venv/
*.pyc
```

### `README.md`
```markdown
# Demo

A tiny service.
```

### `requirements.txt`
```text
requests>=2.31
```

### `src/__init__.py`
```python
from .core import main
```

### `src/core.py`
```python
def main():
    print("hi")
```

## Skipped as secret-like

Review these before sharing the pack; use `--include-secrets` to override.

- `.env`

That is the whole point: a document you can paste, diff, cache, and re-generate when the code changes.

What ends up in the pack

Included: every text file, including dotfiles such as .github/workflows/ci.yml and .gitignore.

Excluded by default:

Every exclusion is reported on stderr, so nothing disappears silently:

$ packdiff
packdiff: 5 files, 103 B from my-service
packdiff: skipped .env: looks like a secret (pass --include-secrets)

Reviewing only what changed

The most common way to use this tool is not "here is my repository" but "here is what I just changed". --diff packs only the files that differ from a revision, and it includes uncommitted and untracked files, because a new file is part of the change:

packdiff --diff                      # what have I not committed yet?
packdiff --diff main                 # what does this branch add on top of main?
packdiff --diff HEAD~3 -o review.md  # the last three commits
packdiff src/ --diff                 # ...restricted to one subtree

It composes with everything else, so the change set still passes through .gitignore, the binary check and the secret filter:

packdiff --diff --max-tokens 16k | pbcopy

Deletion is handled the way you would hope: a deleted file appears in the change set from git, then fails to match anything on disk, so it simply is not packed. Nothing changed, or everything in the change set was filtered out, is a clear error rather than an empty pack:

$ packdiff --diff
packdiff: error: no files changed against 'HEAD'

Change detection shells out to git diff, so the answer is the same one git status just gave you — and if git is missing, or the revision is wrong, or you point --diff at a directory outside the repository, you get an error that says so instead of a silently empty pack.

A worked example

examples/demo-project.pack.md is a committed pack of a small demo project, so you can read the real output without installing anything. The demo directory is arranged to show the filters doing their job: cache/ and debug.log are gone via .gitignore, assets/logo.png is gone as binary, and .env.example survives because a template is not a secret.

python3 examples/regenerate.py    # rebuild it after changing the format

tests/test_examples.py regenerates the pack and compares it byte for byte against the committed file, so the example cannot quietly drift out of date.

Token budgets

--max-tokens keeps the pack inside a context window instead of blowing past it mid-paste.

packdiff --max-tokens 32k -o ctx.md

Files are admitted in a deliberate order: first the ones whose name suggests architecture — README, pyproject.toml / package.json, entry points such as main.py or index.ts — and then the rest largest-first, on the theory that a 400-line module tells you more about a repo than a 3-line __init__.py. Whatever does not fit is listed at the bottom of the document rather than dropped silently:

## Deferred by token budget

These files matched but did not fit; raise `--max-tokens` to include them.

- `src/packdiff/walk.py` (11.9 KiB, ~3,049 tokens)

A first pass holds back 3 % of the budget (at least 120 tokens) for the header and the tree. Per-file estimates cannot know how large those will turn out to be, though, so the finished document is then measured, and the weakest admitted file is deferred and the document re-rendered until the estimate actually fits. --max-tokens is a guarantee on the rendered output, not an aspiration.

How honest is the estimate?

There is no tokenizer in this repo, by design — a 200 MB wheel for a dependency-free CLI would be absurd. Instead the estimator charges ASCII characters at 1 / 4 tokens each (the familiar "~4 characters per token" rule) and non-ASCII characters at one token each, which covers CJK text properly instead of underestimating it fourfold. Expect to land within roughly ±10 % of a BPE tokenizer on ordinary source code. Tune it with --token-ratio if you know your model better than the rule of thumb does.

Ignore rules

Three layers, evaluated in this order:

1. Built-in denylist — build artefacts and lockfiles (--no-default-ignores to disable).
2. .gitignore — including nested files, each anchored to its own directory, and .git/info/exclude (--no-gitignore to disable).
3. .packdiffignore — gitignore syntax, read from the root of the first path you pass, for exclusions you do not want to commit (--ignore-file to point elsewhere).

--include and --exclude accept the same glob syntax and are applied last:

packdiff --include '*.py'                     # only Python files
packdiff --include '*.py' --include '*.toml'  # repeatable, additive
packdiff --exclude 'tests/**' --exclude '*.snap'
packdiff --exclude 'src/**' --exclude '!src/public/**'   # negation re-includes

Supported gitignore syntax: # comments, \# / \! escapes, backslash escaping anywhere in a pattern (foo\ bar matches foo bar), trailing-space stripping, ! negation, trailing / for directory-only patterns, leading / for anchoring, *, ?, [a-z] within a path segment, and ** as a whole segment. A ** at the end of a pattern matches everything inside a directory but not the directory itself; anywhere else it matches zero or more directories. A pattern that matches a directory excludes everything inside it, with or without a trailing slash. Re-including a file whose parent directory is excluded does not work — that is git's rule too, and packdiff enforces it by pruning excluded directories rather than descending into them.

This is the one part of the tool where "close enough" would quietly produce wrong answers, so it ships with an oracle rather than only with unit tests. Run it against any repository you trust and it will tell you where packdiff and git disagree:

$ python3 tools/gitignore_oracle.py /path/to/repo
compared 301 paths: 0 mismatches

Across this repository, a synthetic corpus of edge cases, and the checked-out trees of Flask, Requests, Express and Click — over a thousand paths in total — it reported zero disagreements. It found three real bugs during development: escapes inside patterns, the trailing-** rule, and contents of a matched directory. All three now have tests that cite git as the authority.

Safety

The output of this tool is, by construction, something you paste into someone else's model. That makes the secret filter a feature, not a nicety: files such as .env, *.pem, *.key, id_rsa, .netrc and credentials.json are skipped by default and reported by name. Templates stay: .env.example, .env.sample, id_rsa.pub.

packdiff --include-secrets      # only when you really do want them

The filter is a heuristic. It is not a substitute for reading what you are about to send.

Command reference

Exit status: 0 success, 1 runtime error, 2 usage error.

Recipes

Into a prompt from the shell

packdiff src/ --max-tokens 16k | pbcopy            # macOS
packdiff src/ --max-tokens 16k | clip              # Windows

A code review pack — just the change, for pasting into a reviewer:

packdiff --diff --max-tokens 60k -o review.md       # uncommitted work
packdiff --diff main --max-tokens 60k -o review.md   # a whole branch

Implementation plus tests, nothing else:

packdiff src/ tests/ --include '*.py' --max-tokens 100k -o review.md

Reproducible output for CI artefacts or committed snapshots: set SOURCE_DATE_EPOCH and the timestamp in the header becomes deterministic.

SOURCE_DATE_EPOCH=0 packdiff . -o context.md

See the cost before you commit to it:

packdiff --list          # tokens and bytes per file
packdiff --list | sort -k2 -n | tail   # the heaviest files

Design notes

A few decisions worth knowing about, in case you want to change them.

• No dependencies. Not even a TOML parser for config — .packdiffignore is gitignore syntax read by an engine that had to exist anyway.
• Prune, don't filter. Excluded directories are never descended into, which keeps the walk fast on node_modules and matches git's own re-inclusion semantics for free.
• Fences size themselves. A pack containing a Markdown file that talks about code fences would break with a fixed ```; the renderer emits one backtick longer than the longest run in the file.
• Unicode output, ASCII fallback. If the output stream cannot encode box-drawing characters (a Windows console, say), the tree silently degrades to |-- / `--.
• --diff asks git rather than guessing. Parsing the index by hand would be another thing to get wrong; git diff already knows, and the answer matches what the user saw in git status a second earlier.

Development

git clone https://github.com/muguang1224/packdiff
cd packdiff
python -m pip install -e ".[dev]"

python -m pytest            # the full suite; no network, no fixtures on disk
ruff check . && ruff format --check .

# Optional: check the ignore engine against real git (needs git on PATH)
python3 tools/gitignore_oracle.py .

# Optional: rebuild the committed example pack
python3 examples/regenerate.py

The test suite builds its fixtures in temporary directories, so it runs on Linux, macOS and Windows without any setup.

License

MIT — see LICENSE.