denia 0.1.0

DENIA: Data-Efficient Next-experiment Intelligent Agent - an agentic framework for AI-guided experimentation

DENIA

Data-Efficient Next-experiment Intelligent Agent

DENIA is an open-source agentic framework for AI-guided experimentation. It places a decision engine at the heart of any experimental campaign: intelligent agents understand your data as it is, uncertainty-aware models learn the landscape of what you have already measured, and active learning recommends the experiments most worth running next — with the loop tightening after every new result.

The goal is to make AI-driven experimental optimization, until now the preserve of large industrial R&D platforms, available to every laboratory. Point DENIA at your experimental record, state your objective, and get back a ranked, explained batch of next experiments.

Architecture: agents at the edges, classical core

DENIA places LLM agents where language understanding adds value and keeps the experiment-selection core classical and reproducible:

Layer	Component	Role
Agentic edge	SchemaAgent	Reads arbitrary column headers and sample rows; returns a validated, machine-actionable campaign schema (roles, units, composition pairings, optimization direction)
Agentic edge	ReporterAgent	Turns a recommendation batch into a natural-language briefing for the experimentalist
Classical core	Featurizer	Schema-driven composition vectors, one-hot encodings, standardized conditions
Classical core	EnsembleSurrogate / GPSurrogate	Uncertainty-aware prediction (random-forest dispersion / Matérn-5/2 GP)
Classical core	Acquisition	Expected improvement / UCB with diversity-aware batch selection

Agent output is always validated against the data before use — the LLM proposes, the code verifies. Without an ANTHROPIC_API_KEY, every agent degrades gracefully to deterministic heuristics, so the full loop runs offline and every benchmark result is reproducible without an LLM in the loop.

Installation

pip install denia            # core (offline-capable)
pip install "denia[agents]"  # + LLM-agentic ingestion and reporting

Quickstart

from denia import Campaign

# 1. Ingest your spreadsheet, as-is
camp = Campaign.from_file("my_lab_history.xlsx", target="Y(C2), %")
print(camp.schema.summary())

# 2. Recommend the next batch from a pool of runnable experiments
recommendations = camp.recommend(candidate_pool, batch_size=5, report=True)
print(recommendations)                    # ranked, with predictions ± uncertainty
print(recommendations.attrs["briefing"])  # natural-language briefing

# 3. Run them in the lab, then close the loop
camp.tell(measured_results)

DENIA handles the situations real campaigns present:

• Multiple objectives — pass target=["Y(C2), %", "S(C2), %"] and DENIA fits per-target surrogates and spreads the batch across Pareto trade-offs (ParEGO-style Chebyshev scalarization); camp.pareto() returns the non-dominated experiments so far.
• No candidate list needed — with candidates=None, DENIA proposes new experiments from your observed design space under practical Constraints: numeric bounds, allowed/forbidden categories, fixed settings, and mixture totals (compositions renormalized to 100%).
• Ongoing campaigns — camp.save("project.denia") / Campaign.load(...) persist everything (history, schema, constraints) in one portable, transparent file (plain CSV + JSON inside a zip - no lock-in). Recommend, run, tell the results, save, resume anytime.
• Several data sources — Campaign.from_file([file_a, file_b]) fuses records from multiple files/teams into one campaign, tracking provenance.
• Cost awareness — pass cost_column="cost_eur" and DENIA ranks by expected improvement per unit cost, favoring informative-but-cheap experiments.
• Understanding, not just suggestions — denia.drivers(history, schema) reports which variables the model relies on for your target, at both the species/setting level and the original-column level (descriptive, not causal - and labeled as such).

Or from the command line:

denia schema    --file my_lab_history.xlsx
denia recommend --file my_lab_history.xlsx --target "Y(C2), %" --batch-size 5 --report
denia recommend --file my_lab_history.xlsx --target "Y(C2), %" --target "S(C2), %"   # multi-objective
denia recommend --file my_lab_history.xlsx --target "Y(C2), %" --save proj.denia
denia tell      --campaign proj.denia --results measured_today.csv
denia recommend --campaign proj.denia                                      # resume

Flagship benchmark: retrospective rediscovery of OCM catalysts

Validated on the open literature compilation of oxidative coupling of methane (OCM) catalysis (Zavyalova et al., ChemCatChem 3 (2011) 1935; 2019 update): ~4,750 experiments from 543 publications, in exactly the wide, sparse, heterogeneous spreadsheet format experimental groups actually use (multi-slot compositions in mol%, preparation methods, reaction conditions, correlated targets).

Protocol: the dataset is hidden; the campaign is seeded with 20 random experiments; DENIA selects 5 experiments per iteration from the hidden pool.

Averaged over 5 seeds, DENIA reaches 90% of the dataset optimum within ~170 experiments; random selection does not reach it within the same budget, plateauing at ~80% of the optimum. Reproduce with:

python examples/ocm_retrospective.py path/to/OCM_dataset.xlsx

(The dataset is not redistributed here; see the reference above for the original compilation.)

Scope and roadmap

DENIA operates on any campaign expressible as a table of inputs (compositions, categorical choices, process conditions) and one or more numeric targets: heterogeneous catalysis, synthesis optimization, formulations, device processing. Roadmap: mixed-provenance data fusion, cost-aware acquisition, and richer chemistry-aware featurizations.

Citation

If you use DENIA, please cite it via the metadata in CITATION.cff (Zenodo DOI upon release).

License

MIT — see LICENSE.