malbec

Multi-Agent LLM Behavioural Experiment Comparison

malbec is a lightweight, open Python testbed for running experiments on groups of interacting LLM agents. Install it, configure agents and an environment, and run a reproducible multi-agent experiment. Easily compare results across models, population sizes, and conditions.

Status

Under development. This is an early placeholder release; the API is not yet stable and there is no usable functionality at this version.

Planned design

• Agents composed of an LLM (frontier via API, or open-weight served locally) and a fully configurable harness: prompts, tools, information, and memory.
• Environments defining how agents and other entities interact, including principals that deploy and control agents.
• A controller that runs experiments from simple configuration files, with reproducible seeds and easy parameter sweeps.
• Automatic logging at multiple levels of detail, supporting open-data practices.
• A growing library of experiment templates to get researchers most of the way to a new experiment.

Background

malbec grows out of research on selection dynamics in populations of LLM agents. See the pilot study: The Risks of Evolving AI are Near-Term and Broad.

Author

Charlie Pilgrim (Google Scholar)

License

Apache 2.0