Python package to scrape NRC Event Reports.
Project description
Set of modules to scrape Event Reports from the NRC.gov website.
Tests
pytest
Usage
# generate single event notification report event
url3 = 'https://www.nrc.gov/reading-rm/doc-collections/event-status/event/2019/20190612en.html'
h = EventNotificationReport.from_url(url3, headers)
# get event notification urls from 2019 to 2020
er_urls = generate_nrc_event_report_urls(2019, 2020)
from random import sample
urls = sample(list(extract_nested_values(er_urls)), 10)
sl = logging.getLogger('success_log')
el = logging.getLogger('error_log')
fl = logging.getLogger('fof_log')
# get a subsample of the urls, and log attempts to request
enrs, errors, fofs = fetch_enrs(urls)
# convert an event to dataframe
df = enrs[0].events[0].to_dataframe()
# event notification report to dataframe
enr_df = enrs[1].to_dataframe()
# event notification reports to dataframe
enrs_df = pd.concat([enr.to_dataframe() for enr in enrs])
#dump to csv
enrs_df.to_csv('enrs.csv')
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
nrc-scrape-0.0.3.tar.gz
(7.9 kB
view hashes)
Built Distribution
Close
Hashes for nrc_scrape-0.0.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 98bbd702d56592ffc11ebd9f7d44bf5fce503a01d5da4b24b5dfeef68bef0135 |
|
MD5 | 79c6586520fd05032d35eb4a9400a12e |
|
BLAKE2b-256 | 33697740e980221ff058e9cf7b239114f3a31b5cae35a0efa8e899a68c7a3426 |