Automated Lit Review
Jason Portenoy 2018
This is code and sample data accompanying the paper:
Supervised Learning for Automated Literature Review
published in the proceedings of the 4th Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural Language Processing for Digital Libraries (BIRNDL 2019) co-located with the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2019)
Starting with a list of seed papers, get candidate papers by following in- and out-citations (2 degrees). Then, train a classifier to rank the candidate papers. Repeat this a number of times to get an aggregate ranking for many candidate papers.
Example script in scripts/run_autoreview.py
Inputs:
List of paper IDs for the seed set.
Data for paper citations.
Paper data to be used as features for the classifiers (e.g., clusters, eigenfactor, titles, etc.)
Parameters:
Size of the initial split
Number of times to perform the overall process of collecting candidate papers and training a classifier
Output:
List of papers not in the seed set, ordered descending by relevance score.
Installation
Install via PyPI:
pip install autoreview
Example
Apache Spark (https://spark.apache.org/downloads.html) must be installed to run the example.
The environment variable SPARK_HOME must be set (preferably in a .env file) with the path to Spark.
Java version 8 is required to be used with Spark. Make sure Java 8 is installed and point to its path with the environment variable JAVA_HOME.
Example .env file:
SPARK_HOME=/home/spark-2.4.0-bin-hadoop2.7 JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
Create a virtual environment and install the required libraries:
virtualenv venv source venv/bin/activate pip install -r requirements.txt
Run the full autoreview pipeline using sample data:
python scripts/run_autoreview.py --id-list sample_data/sample_IDs_MAG.txt --citations sample_data/MAG_citations_sample --papers sample_data/MAG_papers_sample --sample-size 15 --random-seed 999 --id-colname Paper_ID --cited-colname Paper_Reference_ID --outdir sample_data/sample_output --debug
This is just meant to show how the system operates. It will not provide meaningful results with such a small sample of paper and citation data.
It will output the top predictions in sample_data/sample_output/predictions.tsv.
Development
For new releases:
# increment the version number bump2version patch
Replace patch with minor or major as needed. Then:
# push new release to github git push --tags # build and upload to PyPI python setup.py sdist bdist_wheel twine check dist/* twine upload dist/*
Metadata
Release files for autoreview 0.2.5
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| autoreview-0.2.5.tar.gz | 18.9 kB | Without provenance | Details for autoreview-0.2.5.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| autoreview-0.2.5-py2.py3-none-any.whl | Python 2, Python 3 | none | any | Details for autoreview-0.2.5-py2.py3-none-any.whl |
Total release size: 41.1 kB
Release files / autoreview-0.2.5.tar.gz
| Download URL | autoreview-0.2.5.tar.gz |
|---|---|
| Size | 18.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
5477a052cd9b6ae57bd475d40a03d3d861b7f6a55167b77fd824eb6cbd4da5fd
|
|
BLAKE2b-256 checksum How to use checksums |
8ebfe88f7ad1eddfbec00fdf027fdc14f280d49524c449f13fc4d92a9e13301d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/45.2.0 requests-toolbelt/0.9.1 tqdm/4.42.1 CPython/3.7.1
|
Release files / autoreview-0.2.5-py2.py3-none-any.whl
| Download URL | autoreview-0.2.5-py2.py3-none-any.whl |
|---|---|
| Size | 22.2 kB |
| Tags | Python 2 Python 3 |
|
SHA-256 checksum How to use checksums |
2178c3816ab167cd5a6c788aa74e68c25c0b65fc3e3b017830851251a3cdde62
|
|
BLAKE2b-256 checksum How to use checksums |
3c76888c9b38eca9577a2f36dd2fb18b5ef1852c719625d0ee8ad6f401d767fe
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/45.2.0 requests-toolbelt/0.9.1 tqdm/4.42.1 CPython/3.7.1
|
Security for autoreview 0.2.5
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.