snapshot — offline website snapshots
Take a fast offline copy of any public website, then serve it locally.
Install
Requires Python 3.10+.
pip install web-snapshot-cli
Then run:
snapshot https://example.com ./mirror
Command generator: snapshot.sushii.dev
Usage
Snapshot a single page
snapshot https://example.com ./mirror
Crawl an entire site
snapshot --crawl https://docs.example.com ./docs --max-pages 200 --depth 4
Save pages as Markdown
snapshot https://example.com ./mirror --lang md
Crawl with filters and politeness
snapshot --crawl --include '/docs/*' --exclude '/docs/drafts/*' \
--crawl-delay 1 --robots https://docs.example.com ./docs
Authenticated pages
snapshot --cookie session=abc123 --header "Authorization: Bearer TOKEN" \
https://app.example.com ./mirror
Sitemap-based crawl
snapshot --crawl --sitemap https://example.com ./mirror --max-pages 500
Gobuster-style path discovery
Brute-force common paths (including robots.txt, sitemap.xml, admin panels, etc.) using bundled wordlists (common + ~18k large entries). Add your own SecLists/gobuster wordlists with -w:
snapshot --crawl --gobuster https://example.com ./mirror
snapshot --crawl -w common -w large -w /path/to/SecLists/Discovery/Web-Content/common.txt URL ./out
snapshot --crawl --gobuster --wordlist-ext html,php,asp URL ./out
Resume an interrupted snapshot
snapshot --resume --crawl https://example.com ./mirror
Dry run (no writes)
snapshot --dry-run --verbose --crawl https://example.com ./mirror
Extra args (positional)
Any key=value pairs after the output directory are merged into options:
snapshot https://example.com ./mirror crawl=true max-pages=100 lang=html concurrency=32
Restore locally
snapshot -restore ./mirror
This starts a local HTTP server (default http://127.0.0.1:8080) and opens your browser.
snapshot -restore ./mirror --port 3000 --no-open
Options
| Flag | Description |
|---|---|
--crawl, -c |
Follow same-origin links and download all pages |
--lang, -l |
Output format: html (default) or md |
--max-pages |
Max pages when crawling (default: 50) |
--depth |
Max crawl depth (default: 3) |
--no-assets |
Skip CSS, JS, images, fonts |
--timeout |
HTTP timeout in seconds (default: 15) |
--concurrency |
Parallel downloads (default: 16) |
--same-origin / --no-same-origin |
Restrict crawl to same origin (default: on) |
--user-agent |
Custom User-Agent header |
--cookie |
Cookie as name=value (repeatable) |
--header |
Extra HTTP header (repeatable) |
--include |
Only fetch URLs matching glob (repeatable) |
--exclude |
Skip URLs matching glob (repeatable) |
--robots / --no-robots |
Respect robots.txt (default: on) |
--crawl-delay |
Seconds to wait after each request (default: 0) |
--sitemap |
Seed crawl from sitemap.xml |
--gobuster, -g |
Discover paths with bundled wordlists |
--wordlist, -w |
Custom or builtin wordlist (common, large) |
--wordlist-ext |
Extensions to append per word (html,php) |
--resume |
Skip pages/assets already saved |
--verbose, -v |
Detailed log output |
--dry-run |
Fetch without writing files |
-restore DIR |
Serve a saved snapshot from DIR |
--port |
Port for restore (default: 8080) |
--host |
Host for restore (default: 127.0.0.1) |
--no-open |
Don't open a browser on restore |
How it works
- snapshot fetches pages with async HTTP (httpx), rewrites links to local paths, and downloads linked assets in parallel.
- A
.snapshot.jsonmanifest is written to the output folder with metadata for restore. - snapshot -restore serves the saved files and maps
/back to the original root page.
Output layout
mirror/
.snapshot.json
example.com/
index.html
about/
index.html
_assets/
...
License
MIT
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
web_snapshot_cli-1.2.0.tar.gz
(80.0 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file web_snapshot_cli-1.2.0.tar.gz.
File metadata
- Download URL: web_snapshot_cli-1.2.0.tar.gz
- Upload date:
- Size: 80.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/6.1.0 CPython/3.13.12
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
be3e297114ad26f369d0e6c45f9bef94a48f2d1d73316c368971bc18d89f9fa4
|
|
| MD5 |
f7fbbe08ac3d8bb7c753ad6a9f1a6958
|
|
| BLAKE2b-256 |
5c7c1829089cfa424cb3145fedf50560c2e3fffdd2e6633379042580007c0ce8
|
Provenance
The following attestation bundles were made for web_snapshot_cli-1.2.0.tar.gz:
Publisher:
publish.yml on codingsushi79/Snapshot
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
web_snapshot_cli-1.2.0.tar.gz -
Subject digest:
be3e297114ad26f369d0e6c45f9bef94a48f2d1d73316c368971bc18d89f9fa4 - Sigstore transparency entry: 2104879435
- Sigstore integration time:
-
Permalink:
codingsushi79/Snapshot@61204d1205fd185dc145c37b18c8441753bbfe44 -
Branch / Tag:
refs/heads/main - Owner: https://github.com/codingsushi79
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish.yml@61204d1205fd185dc145c37b18c8441753bbfe44 -
Trigger Event:
workflow_dispatch
-
Statement type:
File details
Details for the file web_snapshot_cli-1.2.0-py3-none-any.whl.
File metadata
- Download URL: web_snapshot_cli-1.2.0-py3-none-any.whl
- Upload date:
- Size: 75.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/6.1.0 CPython/3.13.12
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
2d73c41e20fe618e8052b81b401b18942886dc7c4369545238277e7ea18d5217
|
|
| MD5 |
036038868c0724375d836c649dcde1ff
|
|
| BLAKE2b-256 |
73c0a806a359acdf2c9dd343c31ff7d744b0c95e7d2bb29a7a74fe9c4cf73782
|
Provenance
The following attestation bundles were made for web_snapshot_cli-1.2.0-py3-none-any.whl:
Publisher:
publish.yml on codingsushi79/Snapshot
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
web_snapshot_cli-1.2.0-py3-none-any.whl -
Subject digest:
2d73c41e20fe618e8052b81b401b18942886dc7c4369545238277e7ea18d5217 - Sigstore transparency entry: 2104879511
- Sigstore integration time:
-
Permalink:
codingsushi79/Snapshot@61204d1205fd185dc145c37b18c8441753bbfe44 -
Branch / Tag:
refs/heads/main - Owner: https://github.com/codingsushi79
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish.yml@61204d1205fd185dc145c37b18c8441753bbfe44 -
Trigger Event:
workflow_dispatch
-
Statement type:
