A command line utility for scraping wayback snapshots from archive.org.
Project description
wayback_scraper
Scraping urls from archive.org (Wayback Machine Snapshots)
Command-Line Interface
The usage information can be printed by running wayback-scraper -h.
usage: wayback-scraper [-h] -u URL [-l LIMIT] [-f Start TIMESTAMP]
[-t End TIMESTAMP] [-p DIRECTORY] [-n FILENAME] -o
Output type
optional arguments:
-h, --help show this help message and exit
-u URL, --url URL url for the wayback scraper to scrape. [eg:
http://www.example.com
-l LIMIT, --limit LIMIT
limit the number of results to scrape
-f Start TIMESTAMP, --from Start TIMESTAMP
start date, format - YYYYmmddHHMMSS. Example : -f 2011
-t End TIMESTAMP, --to End TIMESTAMP
end date, format - YYYYmmddHHMMSS. Example : -t 201205
-p DIRECTORY, --path DIRECTORY
Output file path
-n FILENAME, --name FILENAME
filename for the output data
-o Output type, --output Output type
output type
Examples
The usage can be perhaps be made more clear with a couple of concrete examples.
wayback-scraper -u https://www.archive.org -l 2 -f 2011 -t 2012 -n output-o json
which produces:
{
'archived_urls':
[
'url':'http://www.archive.org:80/',
'date':'1997-01-26',
'timestamp':'19970126045828'
],
[
'url':'http://www.archive.org:80/',
'date':'1997-10-01',
'timestamp':'19971011050034'
]
}
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
wayback_scraper-1.2.tar.gz
(2.9 kB
view hashes)
Built Distribution
Close
Hashes for wayback_scraper-1.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 09810552d864f52a9d13a0220575e25633b861d90bb22cc65b1a8325e1249ec9 |
|
MD5 | 0c5a91e2775eb53b98af268446015b35 |
|
BLAKE2b-256 | d6ec51ee1d29032711d58bdafe4b5f2fa38273f99db0a6be0e5063f989f4aa48 |