CORD 19 tools and utilities
Project description
COVID-19 Data Tools
Tools for making COVID 19 data slightly easier for everyone! If you A) think something would be useful in your research or B) have some helpful code to contribute, make an issue or PR ASAP so we can get your code shared!
Installation
pip install cord-19-tools
Downloading the data
To download and extract the data, use the download
function:
import cotools
from pprint import pprint
cotools.download(dir="data")
For now this just downloads the data from the CORD-19 dataset, metadata is not included (will be by end of day), extracts all the tarfiles, and places them in a directory
The Paperset class
This is a class for lazily loading papers from the CORD-19 dataset.
# no `/` at the end please!
data = cotools.Paperset("data/comm_use_subset")
# indexes with ints
pprint(data[0])
# returns a dict
# and slices!
pprint(data[:2])
# returns a list of dicts
print(len(data))
# takes about 5gb in memory
alldata = data[:]
Lets talk for a bit about how it works, and why it doesnt take a gigantic amount of memory. The files are not actually loaded into python until the data is indexed. Upon indexing, the files at those indexes are read into python, resulting in a list of dictionaries. This means you can still contribute while working on a low resource system.
Hopkins data
The Hopkins data can be loaded with load_hopkins
. It loads three dicts, each containing data from the hopkins dataset:
confirmed, deaths, recoverise = cotools.load_hopkins()
TODO
- Metadata
- Other data, for example data from this aggregate site and this google spreadsheet
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for cord_19_tools-0.0.6-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0b978d73bd8cd509fa628d8ad5c2381c86384fc331523b33c642130b850c64c8 |
|
MD5 | c022c4f7e02d8bc7b38ffd78b8380dde |
|
BLAKE2b-256 | fd19e8162ba5bbae73962ab935650742e0a203f03a64159f3af42247d7ec4a91 |