asreview 0.4

Automated Systematic Review

ASReview: Active learning for systematic reviews

This project is work in progress and not production ready.

---

Check out our new tutorial "10 minutes into ASReview"

Systematic Reviews are “top of the bill” in research. The number of systematic reviews published by researchers increases year after year. But performing a sound systematic review is a time-consuming and sometimes boring task. Our software is designed to accelerate the step of screening abstracts and titles with a minimum of papers to be read by a human with no or very few false negatives.

The Automated Systematic Review (ASReview) project implements learning algorithms that interactively query the researcher. This way of interactive training is known as Active Learning. ASReview offers support for classical learning algorithms and state-of-the-art learning algorithms like neural networks. The following image gives an overview of the process.

Our ASReview software implements two different modes:

• Oracle The oracle modus is used to perform a systematic review with interaction by the reviewer (the 'oracle' in literature on active learning). The software presents papers to the reviewer, whereafter the reviewer classifies them.
• Simulate The simulation modus is used to measure the performance of our software on existing systematic reviews. The software shows how many papers you could have potentially skipped during the systematic review.

Installation

The ASReview software requires Python 3.6+. The project is available on Pypi. Install the project with:

pip install asreview

Or, install the development version of the Automated Systematic Review project directly from this Github page.

pip install git+https://github.com/msdslab/automated-systematic-review.git

Quick start

The quickest way to start using the Automated Systematic Review (ASR) software is the Command Line Interface (CLI). Start an interactive systematic review (Oracle mode) with the following line in CMD or shell:

asreview oracle YOUR_DATA.csv --log_file results.json

This command (asreview oracle) runs the software in oracle mode on the YOUR_DATA.csv dataset.

The higher the number of papers that you manually include in ASReview, the quicker the ASReview software will understand your choices for inclusion. The IDs are the identifiers of papers, starting from 0 for the first paper found in the dataset.

To benchmark an already executed review, use the simulation modus (asreview simulation). The dataset then needs an additional column ("label_included") to signify their inclusion in the final review. The command for the simulation modus is similar to the oracle mode:

asreview simulate YOUR_DATA.csv --n_prior_included 5 --n_prior_excluded 5 --log_file results.h5

Resources

• The full documentation is available at asreview.rtfd.io
• 10 Minutes into ASReview An introduction into ASReview for new users.
• automated-systematic-review-datasets A project for collection, preprocessing and publication of systematic review datasets. The project describes the data storage format used by the software.
• automated-systematic-review-simulations A repository with scripts for a simulation study and scripts for the aggregation and visualisation of the results.

License

LICENSE

Publications

• Dutch newspaper NRC on this project "Software vist de beste artikelen uit een bibliotheek van duizenden."
• News site of Utrecht University: ASReview: Smart source screening software for academia and beyond
• News site of Utrecht University: "A digital tracker dog for datasets"

Citation

A research paper is coming up for this project. In the mean time, it can be cited with (fill in x and y for the version number):

ASReview Core Development Team (2019). ASReview: Software for automated systematic reviews [version 0.x.y]. Utrecht University, Utrecht, The Netherlands. Available at https://github.com/msdslab/automated-systematic-review.

BibTeX:

@Manual{,
    title = {ASReview: Active learning for systematic reviews},
    author = {{ASReview Core Development Team}},
    organization = {Utrecht University},
    address = {Utrecht, The Netherlands},
    year = 2019,
    doi = {10.5281/zenodo.3345592},
    url = {https://doi.org/10.5281/zenodo.3345592}
} 

Contact and contributors

This project is part of the research work conducted by the Department of Methodology & Statistics, Faculty of Social and Behavioral Sciences, Utrecht University, The Netherlands in collaboration with Utrecht Applied Data Science, Information and Technology Services, and Utrecht University Library.

For any questions or remarks, please send an email to [e-mail].

Coordination

• Rens van de Schoot (Main coordinator, @Rensvandeschoot)
• Daniel Oberski (Scientific Director, @daob)

Engineers

• Jonathan de Bruin (Lead engineer, @J535D165)
• Parisa Zahedi (@parisa-zahedi)
• Raoul Schram (@qubixes)
• Kees van Eijden (@KvEijden)

Librarians

• Jan de Boer
• Edu Hackenitz
• Felix Weijdema
• Bianca Kramer

Affiliated Researchers

• Pim Huijnen (Digital Cultural History at the Department of History and Art History)
• Lars Tummers (Public Management and Behavior at Utrecht University, School of Governance)

Students

• Gerbrich Ferdinands (@GerbrichFerdinands)
• Qixiang Fang (@fqixiang)
• Albert Harkema (@sasafrass)