RepoHarvester
Clone a git repository (or point at a local folder) and compile all of its text files into one file. Handy for feeding a codebase to an LLM, code review, or offline reading.
Features
- Works with SSH/HTTPS git URLs (shallow clone into a temp dir) or local folders.
- Skips binary/noise by default: media, archives, executables, lock files,
node_modules, virtualenvs, caches,.git,.github, and more. - Optional comment removal for common languages. String literals are kept, so
"http://..."and"#fff"stay intact. Python uses the real tokenizer; other languages use regexes. - Size limit per file. Non-UTF-8 files are skipped and listed in
skipped_files.txt. - Respects
.gitignore: in a git repo only tracked and untracked-but-not-ignored files are harvested. - Logs a rough token estimate (~4 characters per token) so you know if it fits an LLM context.
- Each file is headed by its path relative to the repo root. The output folder is never harvested into itself.
- No runtime dependencies. The GUI is optional.
Install
Requires Python 3.12+ and git.
uv tool install repoharvester # or: pipx install repoharvester / uv run repoharvester ... from a clone
Usage
repoharvester <repo-url-or-folder> [options]
| Option | Description |
|---|---|
-r, --remove |
Remove comments from code files |
--no-skip GROUP[,GROUP] |
Include groups that are skipped by default (repeatable) |
--max-size KB |
Skip files larger than this (default 1000). Files over 500 KB are logged |
--exclude DIR[,DIR] |
Folder names to exclude, along with their contents (repeatable) |
-o, --output-dir |
Output directory (default output) |
--log FILE |
Also write the log to a file |
Groups: archive, audio, cache_files, certificate, compiled_code, configuration, database, executables, font, log_files, media, node_modules, office, package_locks, system, temporary, video, virtual_env. See EXTENSION_GROUPS, FILENAME_GROUPS and FOLDER_GROUPS in repoharvester.py.
Example:
repoharvester git@github.com:username/repo.git -r --no-skip configuration --max-size 250 --exclude vendor,docs
GUI
uv run --extra gui python gui.py
Paste a git URL or choose a folder, tick anything extra to include, press Harvest (or Enter), then Open file.
While a clone runs the button turns into Cancel. Settings are remembered in ~/.config/repoharvester/gui.json.
On Linux the project prefers the system Python: uv's bundled builds ship Tk without font antialiasing.
Output format
## <repo-name>
### <relative/path/to/file.ext>
<file contents>
### end of file
...
Written to <output-dir>/<repo-name>_all_files.txt, with skipped_files.txt next to it.
Development
uv run --group dev pytest
uvx ruff check .
License
MIT. See LICENSE.
Metadata
Release files for repoharvester 0.2.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| repoharvester-0.2.0.tar.gz | 15.7 kB | Without provenance | Details for repoharvester-0.2.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| repoharvester-0.2.0-py3-none-any.whl | Python 3 | none | any | Details for repoharvester-0.2.0-py3-none-any.whl |
Total release size: 24.4 kB
Release files / repoharvester-0.2.0.tar.gz
| Download URL | repoharvester-0.2.0.tar.gz |
|---|---|
| Size | 15.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
1697bb72eaedc89b1ffb4cb32ee45bef7ce0700a6121a8ad9caba6e5b56c9283
|
|
BLAKE2b-256 checksum How to use checksums |
184ab76b2f8b10a7d275c81d2d615a4644c8e0c4f625037432a90e22233d3e36
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.12.19 {"installer":{"name":"uv","version":"0.12.19","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Release files / repoharvester-0.2.0-py3-none-any.whl
| Download URL | repoharvester-0.2.0-py3-none-any.whl |
|---|---|
| Size | 8.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
a833b64e13279b499001afdfdda265ea2ff2cd71e2978bd17dbf2cb3c6aab68f
|
|
BLAKE2b-256 checksum How to use checksums |
022b8f566462b525eba060bc213a718c1504551baeead579affd81e19f355804
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.12.19 {"installer":{"name":"uv","version":"0.12.19","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Security for repoharvester 0.2.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.