Open-ended Scientific Discovery via Bayesian Surprise

Asta Autodiscovery is an autonomous agent that performs data exploration on arbitrary datasets. The agent will generate hypotheses and run experiments to test each one. Surprising outcomes generate follow-up hypotheses in a recursive exploration.

Link to our NeurIPS 2025 paper: AutoDiscovery: Open-ended Scientific Discovery via Bayesian Surprise

Upgrading from 1.x

2.0.0 has no default model. A command line that relied on the 1.x default stops at startup with No model chosen: add --model <provider>/<model>, or export AUTODISCOVERY_MODEL. --belief_model and --vision_model follow --model unless set, and --embedding_model is needed only with --dedupe. See Selecting models below.

Upgrading from 0.2.x

1.0.0 changes two things that every existing command line depends on:

• Model flags now name their provider. --model gemini-3.1-pro-preview becomes --model vertex_ai/gemini-3.8-flash. A bare model name is rejected at startup. See Selecting models below.
• Vertex AI authenticates via Application Default Credentials only. VERTEX_ACCESS_TOKEN, GOOGLE_OAUTH_ACCESS_TOKEN and VERTEX_OPENAI_BASE_URL are no longer read.

Full notes: CHANGELOG.

Installation

Requires Python 3.13 or newer.

pip install asta-autodiscovery

This installs the auto-discovery command-line tool.

Quick start

Point auto-discovery at one or more dataset files and describe what you want explored:

auto-discovery \
    --model openai/gpt-5.4-mini \
    --name "Plant growth study" \
    --description "Field trial measurements of plant height under varying fertilizer dosage" \
    --intent "Focus on dose-response relationships" \
    --n_experiments 20 \
    --out_dir ./results \
    data/measurements.csv data/treatments.csv

CSV/TSV column headers are detected automatically. Datasets can also be directories — every file under them will be included.

Dataset files/directories can have different descriptions for each one listed. Use a repeated --dataset_description parameter in place of the overall --description.

A static HTML report is written to <out_dir>/report when the run starts and rewritten as each experiment finishes. Open its index.html in a browser: it is the web UI's view of the run, it works offline from the files in that directory, and while the run goes on it picks up new findings by itself.

Common options

Run auto-discovery --help to see the full set of options.

Authentication

All model traffic goes through litellm, and every model flag names its provider explicitly as <provider>/<model> — for example vertex_ai/gemini-3.8-flash, openai/o4-mini, github_copilot/claude-haiku-4.5. You only need to configure the providers you actually name in --model, --belief_model, --vision_model and --embedding_model.

Vertex AI

Used when a model flag names vertex_ai/....

Pick one of the following. In all cases, set the project and location so the agent knows which Vertex endpoint to call — these are litellm's own variable names, read by litellm itself:

export VERTEXAI_PROJECT=your-gcp-project-id
export VERTEXAI_LOCATION=global   # `global` serves the Gemini models

Both are required for a vertex_ai/ model, and a run that names one without them stops at startup naming whichever is missing. Neither is inferred: Application Default Credentials carry a project of their own, which is frequently not the project you meant, and litellm's fallback location is us-central1, which does not serve every Gemini model. Relying on either turns a missing setting into a confusing 404 mid-run.

Service account key file (recommended for non-interactive use):

Create a service account in your GCP project, grant it the Vertex AI User role, download a JSON key, and point Google's standard ADC env var at it:

export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account-key.json

User credentials via gcloud (recommended for local development):

gcloud auth application-default login

Vertex authenticates via Application Default Credentials, so either GOOGLE_APPLICATION_CREDENTIALS or gcloud auth application-default login is required. Raw bearer tokens (VERTEX_ACCESS_TOKEN) are no longer accepted.

OpenAI

Used when a model flag names openai/... (e.g. openai/gpt-4o).

export OPENAI_API_KEY=sk-...

Anthropic

Used when a model flag names anthropic/.... Anthropic has no embedding models, so --dedupe needs an --embedding_model from another provider.

export ANTHROPIC_API_KEY=sk-ant-...

Azure OpenAI

Used when a model flag names azure/<deployment name>. Name deployments after the model they serve, so the model's request handling resolves from litellm's registry.

export AZURE_API_KEY=...
export AZURE_API_BASE=https://<resource>.openai.azure.com

GitHub Copilot

Used when a model flag names github_copilot/.... litellm reads a GitHub OAuth token from a file. On an interactive terminal it runs GitHub's device-code login on first use and caches the token itself, so no setup is needed. For non-interactive runs, pre-seed it:

export GITHUB_COPILOT_TOKEN_DIR=/path/to/dir   # must contain a file named `access-token`

There is no TTY detection, so a headless run without a cached token prints a device code and blocks for about three minutes before failing.

Selecting models

Because the provider travels with each flag, mixing providers is supported — for example, --model openai/gpt-4o --belief_model vertex_ai/gemini-3.8-flash uses OpenAI for the main loop and Vertex AI for belief updates, with both OPENAI_API_KEY and the Vertex variables set.

Each flag is validated against litellm's offline model registry at startup, before the first model call.

Citation

If you find this work useful, please cite:

@inproceedings{
agarwal2025autodiscovery,
title={AutoDiscovery: Open-ended Scientific Discovery via Bayesian Surprise},
author={Dhruv Agarwal and Bodhisattwa Prasad Majumder and Reece Adamson and Megha Chakravorty and Satvika Reddy Gavireddy and Aditya Parashar and Harshit Surana and Bhavana Dalvi Mishra and Andrew McCallum and Ashish Sabharwal and Peter Clark},
booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems},
year={2025},
url={https://openreview.net/forum?id=kJqTkj2HhF}
}