AgentLab

A modular, composable framework for building, evaluating, and comparing LLM-powered agents.

Installation

pip install agentlab

Or with optional provider extras:

pip install agentlab[llm]      # OpenAI, Anthropic, Google SDKs
pip install agentlab[rerank]   # Cohere + CrossEncoder rerankers
pip install agentlab[dev]      # Development tools

Package Structure

AgentLab is organized like scoped packages — each sub-package has one clear responsibility:

Quick Start

from agentlab.agents import agent
from agentlab.llm import get_llm
from agentlab.embedding import get_embedding
from agentlab.retriever import get_retriever
from agentlab.reranker import get_reranker
from agentlab.execution import run_experiment

# 1. Define an agent using the @agent decorator
@agent(name="my_researcher")
class MyResearcher:
    def __init__(self, llm):
        self.llm = llm

    def run(self, query: str) -> str:
        return self.llm.generate(f"Research: {query}")

# 2. Build components
llm       = get_llm(provider="openai", model="gpt-4o")
embedding = get_embedding(provider="openai", model="text-embedding-3-small")
retriever = get_retriever(provider="faiss")
reranker  = get_reranker(provider="cohere")

# 3. Run an experiment
result = run_experiment({
    "agents": ["my_researcher"],
    "llms": [{"provider": "openai", "model": "gpt-4o", "temperature": 0.2}],
    "embeddings": [{"provider": "openai", "model": "text-embedding-3-small"}],
    "retrievers": [{"provider": "faiss"}],
    "mode": "Sequential"
})

Environment Variables

Create a .env file in your project root:

OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
GOOGLE_API_KEY=AI...
COHERE_API_KEY=...
VOYAGE_API_KEY=...

License

MIT