autora 3.0.0a10

Autonomous Research Assistant (AutoRA) is a framework for automating steps of the empirical research process.

Automated Research Assistant

AutoRA (Automated Research Assistant) is an open-source framework for automating multiple stages of the empirical research process, including model discovery, experimental design, data collection, and documentation for open science.

AutoRA implements the Autonomous Empirical Research Paradigm, which involves a dynamic interplay between two artificial agents. The first agent, a theorist, is primarily responsible for constructing computational models by relying on existing data to link experimental conditions to dependent measures. The second agent, an experimentalist, is tasked with designing follow-up experiments that can refine and validate the models generated by the theorist. Together, these agents implement an automated scientific discovery process. To enable closed-loop empirical research, AutoRA interfaces with platforms for automated data collection, such as Prolific or Amazon Mechanical Turk, which enable the efficient acquisition of behavioral data from human participants. Finally, AutoRA is designed to support the automated documentation and dissemination of steps in the empirical research process.

AutoRA was initially intended for accelerating research in the behavioral and brain sciences. However, AutoRA is designed as a general framework that enables automation of the research processes in other empirical sciences, such as material science or physics.

Features

AutoRA consists of different modules that can be used independently or in combination, such as:

• Automated theorists that support the discovery of formal scientific models from data
• Automated experimentalists that support the design of follow-up experiments
• Interfaces for automated data collection, e.g., for behavioral experiments via Prolific or Amazon Mechanical Turk
• Workflow logic for defining interactions between different components of the research process
• Interfaces for automated documentation of the research process

Usages

AutoRA can be used for a variety of research purposes in empirical sciences, such as psychology, neuroscience, economics, physics, or material science. Usages include:

• Equation discovery from empirical data
• Experimental design for follow-up experiments
• Research documentation and dissemination
• Closed-loop empirical research
• Computational analyses of the scientific process (metascience, computational philosophy of science)

Motivation

Various empirical sciences are beset by a replication crisis, which can be attributed to inadequately precise hypotheses, lack of transparency in research procedures, and insufficient rigor in testing findings. These limitations are the result of three primary bottlenecks—a lack of formal modeling, the demanding requirements of open science, and a shortage of resources to reproduce individual studies. Empirical scientists face difficulties in formalizing their theories, find it arduous to document their research activities, and often lack time and funds to conduct follow-up experiments to test and revise their hypotheses. These limitations impede scientific progress and hinder the development of new knowledge. We seek to overcome these limitations by providing a tool for the generation, estimation, and empirical testing of scientific models. It is our hope that AutoRA will help accelerate scientific discovery by overcoming these limitations and promoting greater transparency and rigor in empirical research.

Pointers

• AutoRA Pip Package
• GitHub Repository
• Autonomous Empirical Research Group

About

This project is in active development by the Autonomous Empirical Research Group, led by Sebastian Musslick, in collaboration with the Center for Computation and Visualization at Brown University.

The development of this package is supported by Schmidt Science Fellows, in partnership with the Rhodes Trust, as well as the Carney BRAINSTORM program at Brown University.