iagitup

Archive git repositories to the Internet Archive.

iagitup clones a repository, packs it into a git bundle, and uploads it to archive.org with metadata. archive-watchlist does it in bulk: every repo of a user or organization, or the most-starred GitHub repos.

Features

• Any HTTPS git host -- GitHub, GitLab, Bitbucket, Codeberg, self-hosted Gitea/Forgejo, or a plain URL.
• Complete snapshots -- every branch, tag and ref in one verified bundle.
• Wikis, LFS, submodules -- archived alongside the main bundle.
• Releases, issues, pull requests -- optional GitHub exports, assets included.
• Rich metadata -- description, README, avatar, stars and forks from the platform API.
• No duplicates -- a snapshot is uploaded once; a new push creates a new item.
• Resumable -- keep the clone and retry an interrupted upload without re-cloning.
• Bulk mode -- every repo of a user or org, or the top 1000 GitHub repos. Parallel workers, JSON summaries.

Installation

pip install iagitup

Installs two commands: iagitup and archive-watchlist.

Requirements:

• Python 3.10+
• git on $PATH (git-lfs too, if you want LFS objects)
• An Internet Archive account
• HTTPS repository URLs (SSH syntax is not supported)

Quick Start

iagitup https://github.com/torvalds/linux
iagitup https://gitlab.com/inkscape/inkscape
iagitup https://codeberg.org/forgejo/forgejo

:: Downloading https://github.com/torvalds/linux ...
:: Uploading bundle: torvalds-linux_-_2026-02-28_10-00-00.bundle
:: Upload FINISHED.
   Archived repository: https://archive.org/details/github.com-torvalds-linux_-_2026-02-28_10-00-00
   Git bundle:          https://archive.org/download/github.com-torvalds-linux_-_2026-02-28_10-00-00/torvalds-linux_-_2026-02-28_10-00-00.bundle

Bulk mode:

archive-watchlist --dry-run --top 10          # preview
archive-watchlist --workers 8                 # top 100, all-time
archive-watchlist --days 7                    # trending this week
archive-watchlist --since 2025-01-01 --until 2025-06-30
archive-watchlist --owner torvalds --skip-archived --max-repos 20   # all repos of a user/org

Usage

iagitup [options] <repo_url>

Archive every repo of a user or organization by passing its URL:

iagitup https://github.com/torvalds
iagitup --skip-archived --max-repos 20 https://gitlab.com/gnome

Resume an interrupted upload:

iagitup --keep-work-dir https://github.com/user/repo
# ... fails; stderr shows ":: Work directory kept: /tmp/tmpabc123/repo"
iagitup --resume /tmp/tmpabc123/repo https://github.com/user/repo

archive-watchlist [options]

Cron, daily at 03:00:

GITHUB_TOKEN=ghp_your_token_here
0 3 * * * archive-watchlist --json-summary /var/log/watchlist.json >> watchlist.log 2>&1

Configuration

Internet Archive credentials. On first run you are asked to run ia configure. Keys are read from ~/.config/internetarchive/ia.ini, ~/.ia or ~/.config/ia.ini. Get them at https://archive.org/account/s3.php.

GITHUB_TOKEN raises the GitHub API limit from 60 to 5,000 requests per hour. No scopes needed for public repos. Get one at https://github.com/settings/tokens.

GITLAB_TOKEN authenticates GitLab API calls for private projects and higher limits.

What gets archived

Each snapshot is one IA item named {host}-{owner}-{repo}_-_{YYYY-MM-DD_HH-MM-SS}, dated by the last push.

Item metadata: title, date, year, creator, originalurl, pushed_date, subject (e.g. GitHub;code;software;git), stars_count, forks_count, has_lfs, and an HTML description built from the repo description, README and restore instructions. archive-watchlist adds language, topics and watchers_count, plus github_rank for top-starred runs. Fields passed with --metadata override these.

How it works

1. Fetch metadata from the platform API, or from git history if there is none.
2. Skip if that snapshot already exists on archive.org. For GitHub this happens before cloning.
3. Clone, fetch the wiki, avatar and LFS objects, and any requested extras.
4. Create and verify the bundles.
5. Upload everything to one item.

In owner mode, archive-watchlist first lists the repos through the platform API. It also keeps a local state file of pushed_at per repo, so unchanged repos are skipped with no network calls. Delete a repo's entry to force a re-archive.

Restoring

wget https://archive.org/download/<identifier>/<name>.bundle
git clone <name>.bundle my-repo

LFS objects:

cd my-repo
wget https://archive.org/download/<identifier>/<name>_lfs.tar.gz
tar -xzf <name>_lfs.tar.gz -C .git/
git lfs install && git lfs checkout

Submodules: clone each submodules/*.bundle into the path listed in <name>_gitmodules.txt.

Contributing

Issues and pull requests are welcome at https://github.com/gdamdam/iagitup. Tests run on Python 3.10 to 3.13.

License

GPLv3. Copyright (C) 2018-2026 Giovanni Damiola. See LICENSE.