Python package to scrape NRC Event Reports.
Project description
Set of modules to scrape Event Reports from the NRC.gov website.
Tests
pytest
Usage
Download all event reports into yearly csv files:
python bulk_downloader --start_year 2004 --end_year 2020
# generate single event notification report event
url3 = 'https://www.nrc.gov/reading-rm/doc-collections/event-status/event/2019/20190612en.html'
h = EventNotificationReport.from_url(url3, headers)
# get event notification urls from 2019 to 2020
er_urls = generate_nrc_event_report_urls(2019, 2020)
from random import sample
urls = sample(list(extract_nested_values(er_urls)), 10)
sl = logging.getLogger('success_log')
el = logging.getLogger('error_log')
fl = logging.getLogger('fof_log')
# get a subsample of the urls, and log attempts to request
enrs, errors, fofs = fetch_enrs(urls)
# convert an event to dataframe
df = enrs[0].events[0].to_dataframe()
# event notification report to dataframe
enr_df = enrs[1].to_dataframe()
# event notification reports to dataframe
enrs_df = pd.concat([enr.to_dataframe() for enr in enrs])
#dump to csv
enrs_df.to_csv('enrs.csv')
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
nrc-scrape-0.0.4.tar.gz
(9.1 kB
view hashes)
Built Distribution
nrc_scrape-0.0.4-py3-none-any.whl
(10.9 kB
view hashes)
Close
Hashes for nrc_scrape-0.0.4-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 03a7dd7c73624f28688b4fe54e33f7fc767c58c988cede949cab81550d504470 |
|
MD5 | c740d5dcd88d13aa2b4fe8e2474f60ed |
|
BLAKE2b-256 | 951f1325873de53329f55a7b16e67f69618a2dd917233153805029f8f1b8b1e4 |