snapshot — offline website snapshots
Take a fast offline copy of any public website, then serve it locally.
Install
Requires Python 3.10+.
pip install web-snapshot-cli
Then run:
snapshot https://example.com ./mirror
Command generator: snapshot.sushii.dev
Usage
Snapshot a single page
snapshot https://example.com ./mirror
Crawl an entire site
snapshot --crawl https://docs.example.com ./docs --max-pages 200 --depth 4
Save pages as Markdown
snapshot https://example.com ./mirror --lang md
Crawl with filters and politeness
snapshot --crawl --include '/docs/*' --exclude '/docs/drafts/*' \
--crawl-delay 1 --robots https://docs.example.com ./docs
Authenticated pages
snapshot --cookie session=abc123 --header "Authorization: Bearer TOKEN" \
https://app.example.com ./mirror
Sitemap-based crawl
snapshot --crawl --sitemap https://example.com ./mirror --max-pages 500
Gobuster-style path discovery
Brute-force common paths (including robots.txt, sitemap.xml, admin panels, etc.) using bundled wordlists (common + ~18k large entries). Add your own SecLists/gobuster wordlists with -w:
snapshot --crawl --gobuster https://example.com ./mirror
snapshot --crawl -w common -w large -w /path/to/SecLists/Discovery/Web-Content/common.txt URL ./out
snapshot --crawl --gobuster --wordlist-ext html,php,asp URL ./out
Resume an interrupted snapshot
snapshot --resume --crawl https://example.com ./mirror
Dry run (no writes)
snapshot --dry-run --verbose --crawl https://example.com ./mirror
Extra args (positional)
Any key=value pairs after the output directory are merged into options:
snapshot https://example.com ./mirror crawl=true max-pages=100 lang=html concurrency=32
Restore locally
snapshot -restore ./mirror
This starts a local HTTP server (default http://127.0.0.1:8080) and opens your browser.
snapshot -restore ./mirror --port 3000 --no-open
Options
| Flag | Description |
|---|---|
--crawl, -c |
Follow same-origin links and download all pages |
--lang, -l |
Output format: html (default) or md |
--max-pages |
Max pages when crawling (default: 50) |
--depth |
Max crawl depth (default: 3) |
--no-assets |
Skip CSS, JS, images, fonts |
--timeout |
HTTP timeout in seconds (default: 15) |
--concurrency |
Parallel downloads (default: 16) |
--same-origin / --no-same-origin |
Restrict crawl to same origin (default: on) |
--user-agent |
Custom User-Agent header |
--cookie |
Cookie as name=value (repeatable) |
--header |
Extra HTTP header (repeatable) |
--include |
Only fetch URLs matching glob (repeatable) |
--exclude |
Skip URLs matching glob (repeatable) |
--robots / --no-robots |
Respect robots.txt (default: on) |
--crawl-delay |
Seconds to wait after each request (default: 0) |
--sitemap |
Seed crawl from sitemap.xml |
--gobuster, -g |
Discover paths with bundled wordlists |
--wordlist, -w |
Custom or builtin wordlist (common, large) |
--wordlist-ext |
Extensions to append per word (html,php) |
--resume |
Skip pages/assets already saved |
--verbose, -v |
Detailed log output |
--dry-run |
Fetch without writing files |
-restore DIR |
Serve a saved snapshot from DIR |
--port |
Port for restore (default: 8080) |
--host |
Host for restore (default: 127.0.0.1) |
--no-open |
Don't open a browser on restore |
How it works
- snapshot fetches pages with async HTTP (httpx), rewrites links to local paths, and downloads linked assets in parallel.
- A
.snapshot.jsonmanifest is written to the output folder with metadata for restore. - snapshot -restore serves the saved files and maps
/back to the original root page.
Output layout
mirror/
.snapshot.json
example.com/
index.html
about/
index.html
_assets/
...
License
MIT
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
web_snapshot_cli-1.1.0.tar.gz
(79.3 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file web_snapshot_cli-1.1.0.tar.gz.
File metadata
- Download URL: web_snapshot_cli-1.1.0.tar.gz
- Upload date:
- Size: 79.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/6.1.0 CPython/3.13.12
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
956525ee13b0fa2a6dd0f3789b376b61d52a1c69ff71cb0b709839c6558fe3b9
|
|
| MD5 |
deb1c7bea46010d8e40c7da235a02c93
|
|
| BLAKE2b-256 |
4e2014a595a480a21f7e9bf835beb3730bbe22e5d815ca9f2343c62f511388fe
|
Provenance
The following attestation bundles were made for web_snapshot_cli-1.1.0.tar.gz:
Publisher:
publish.yml on codingsushi79/Snapshot
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
web_snapshot_cli-1.1.0.tar.gz -
Subject digest:
956525ee13b0fa2a6dd0f3789b376b61d52a1c69ff71cb0b709839c6558fe3b9 - Sigstore transparency entry: 2104623135
- Sigstore integration time:
-
Permalink:
codingsushi79/Snapshot@f5e6f8bdeee373fdc410d95ba9c91ef29efe4530 -
Branch / Tag:
refs/tags/v1.1.0 - Owner: https://github.com/codingsushi79
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish.yml@f5e6f8bdeee373fdc410d95ba9c91ef29efe4530 -
Trigger Event:
release
-
Statement type:
File details
Details for the file web_snapshot_cli-1.1.0-py3-none-any.whl.
File metadata
- Download URL: web_snapshot_cli-1.1.0-py3-none-any.whl
- Upload date:
- Size: 75.1 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/6.1.0 CPython/3.13.12
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
a5a63bb8375f4855875ac2dd7aec109d1f2cd4773c5149ce4adee68686cd2f0e
|
|
| MD5 |
b97bb2350574b6132b1c324be093f3a2
|
|
| BLAKE2b-256 |
f46e73c0cb5ae656d2fa48114119f4d3f8ca8e6c1217245d00ccb05f894d525e
|
Provenance
The following attestation bundles were made for web_snapshot_cli-1.1.0-py3-none-any.whl:
Publisher:
publish.yml on codingsushi79/Snapshot
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
web_snapshot_cli-1.1.0-py3-none-any.whl -
Subject digest:
a5a63bb8375f4855875ac2dd7aec109d1f2cd4773c5149ce4adee68686cd2f0e - Sigstore transparency entry: 2104623245
- Sigstore integration time:
-
Permalink:
codingsushi79/Snapshot@f5e6f8bdeee373fdc410d95ba9c91ef29efe4530 -
Branch / Tag:
refs/tags/v1.1.0 - Owner: https://github.com/codingsushi79
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish.yml@f5e6f8bdeee373fdc410d95ba9c91ef29efe4530 -
Trigger Event:
release
-
Statement type:
