llm-agent-base

A lightweight Python library for building LLM agents with tool calling and retrieval-augmented generation (RAG). Works with any OpenAI-compatible API (OpenRouter, OpenAI, Ollama, etc.).

Available on PyPI: https://pypi.org/project/llm-agent-base/

Installation

pip install llm-agent-base

Features

• Simple LLM calls — single-call ask() with optional knowledge retrieval for straightforward completions
• Agentic loop — run() with knowledge retrieval and tool calling until a final text response
• Conversational chat — chat() maintains conversation history across calls; reset_conversation() starts fresh
• File attachments — pass images, PDFs, and text files directly to ask(), run(), or chat() via the files parameter
• Tool calling — register plain Python functions as LLM-callable tools; schemas are built automatically from type hints and docstrings, including per-parameter descriptions and enum values
• Robust inline tool calls — handles models that emit tool calls as JSON text (e.g. GLM), including batched and variant-encoded arguments, while gating on the tool registry so genuine JSON output is never mistaken for a malformed call
• RAG — ingest a folder of documents (.txt, .md, .json, .pdf) into a FAISS vector index and inject relevant chunks into every prompt
• Incremental re-indexing — edit a document and only that file is re-embedded; unchanged chunks keep their vectors
• Knowledge search tool — when a knowledge base is configured, the agent automatically gains a search_knowledge tool (semantic/vector) and a read_knowledge_files tool (keyword search returning full file contents); both are optional and independently toggleable
• Pipelines — chain multiple agents so each agent's output becomes the next agent's input
• Temperature control — set per-agent temperature for precise or creative responses
• Response format — enforce JSON output or structured schemas via the OpenAI response format API
• Debug mode — optional logging of tool calls and knowledge retrievals

Quick start

from llm_agent_base import AgentBase, LLMConnectionConfig

config = LLMConnectionConfig(model="openai/gpt-4o-mini", api_key="...")

agent = AgentBase(
    system_prompt="You are a helpful assistant.",
    llm_config=config,
)

# Simple one-shot call
print(agent.ask("What is the capital of France?"))

# Full agentic loop (knowledge retrieval + tool calling)
print(agent.run("What is the capital of France?"))

The default base_url points to OpenRouter, which gives access to many models through a single API key. You can swap it for the OpenAI base URL or any other compatible endpoint.

Usage

Simple LLM call

ask() makes a single completion call with no tool calling. If a knowledge base is configured, relevant chunks are automatically retrieved and injected as context.

from llm_agent_base import AgentBase, LLMConnectionConfig

config = LLMConnectionConfig(model="openai/gpt-4o-mini", api_key="...")

agent = AgentBase(system_prompt="You are a helpful assistant.", llm_config=config)
print(agent.ask("Summarise the water cycle in one sentence."))

Temperature and response format

# Low temperature for deterministic, factual responses
precise = AgentBase(
    system_prompt="You are a helpful assistant.",
    llm_config=config,
    temperature=0.1,
)

# High temperature for creative responses
creative = AgentBase(
    system_prompt="You are a poet.",
    llm_config=config,
    temperature=1.4,
)

# Enforce JSON output
json_agent = AgentBase(
    system_prompt="Always respond with valid JSON.",
    llm_config=config,
    response_format={"type": "json_object"},
)
print(json_agent.ask('Return {"city": "Paris", "country": "France"}'))

Tool calling

Register any Python function as a tool. The function name becomes the tool name, the docstring becomes its description, and the type hints define the parameter schema.

Schema generation reads more than the bare type. A docstring Args: section (Google style) or :param name: lines (Sphinx style) become per-parameter descriptions, Literal and Enum annotations become enum constraints, and list[str] gets a proper items type — which strict endpoints require and which measurably improves how accurately the model fills arguments. Returns:/Raises: sections are kept out of the tool description.

@agent.register_tool
def search_orders(customer: str, status: Literal["open", "shipped", "cancelled"] = "open") -> str:
    """Look up a customer's orders.

    Args:
        customer: The customer's account ID.
        status: Which order state to filter on.
    """

The tool-calling loop handles both the standard OpenAI tool_calls field and the fallback case where a model emits a tool call as inline JSON text in the message content — a behaviour seen on some OpenAI-compatible endpoints (e.g. GLM models). Both of these inline shapes are recognised:

{"name": "get_weather", "arguments": {"city": "Tokyo"}}
{"get_weather": {"city": "Tokyo"}}

The second form — the tool name as the single top-level key, arguments as the nested object — is also produced by GLM models. Either shape may batch several calls to the same tool by supplying a list of argument objects ({"get_weather": [{"city": "Tokyo"}, {"city": "Paris"}]}); each is executed and fed back with a distinct id, and a batch counts as one round against the safety cap. The arguments payload is also normalised across common variants — a single-element list wrapper ([{"city": "Tokyo"}]) or a double-encoded JSON string — so a recoverable call is not silently dropped.

Crucially, an inline call is only intercepted when its tool name matches a registered tool. Because both shapes are indistinguishable from ordinary JSON a model might emit as its genuine answer ({"result": {...}} or {"name": "Alice", "arguments": {...}}), gating on the tool registry ensures real JSON output is returned unchanged rather than mistaken for a malformed tool call. When a call is detected, the tool runs, the result is fed back, and the loop continues exactly as with a structural call.

Tool results are made actionable for the model: a tool that raises gets an error message naming the tool and instructing the model to proceed, and a tool that returns None or an empty string is reported as an explicit "no results" message so the model never confuses "found nothing" with "the call failed" — for both inline and structural calls.

The loop is bounded by max_iterations (default 10) and a safety cap of 5 consecutive text-emitted calls. Once either budget is spent, the tools are withheld and the model is told to give a final answer in text — so even a model that would otherwise keep calling tools forever always terminates with a real answer instead of leaking its last tool-call JSON. The loop always returns a string.

agent = AgentBase(..., max_iterations=25)  # allow longer tool chains

from llm_agent_base import AgentBase, LLMConnectionConfig

config = LLMConnectionConfig(model="openai/gpt-4o-mini", api_key="...")

agent = AgentBase(
    system_prompt="You are a helpful assistant. Use the available tools when needed.",
    llm_config=config,
)

@agent.register_tool
def get_weather(city: str) -> str:
    """Return the current weather for a given city."""
    return f"The weather in {city} is sunny and 22°C."

@agent.register_tool
def add(a: int, b: int) -> int:
    """Add two integers and return the result."""
    return a + b

print(agent.run("What is the weather in Tokyo and what is 10 + 20?"))

register_tool can also be called directly:

agent.register_tool(get_weather)

Conversational chat

chat() works like run() — knowledge retrieval and tool calling are both active — but it accumulates the conversation history across calls so the LLM retains context between turns. Call reset_conversation() to wipe the history and start a new session.

from llm_agent_base import AgentBase, LLMConnectionConfig

config = LLMConnectionConfig(model="openai/gpt-4o-mini", api_key="...")

agent = AgentBase(
    system_prompt="You are a helpful assistant. Keep answers concise.",
    llm_config=config,
)

print(agent.chat("My name is Alice."))
print(agent.chat("What is my name?"))  # agent remembers: Alice

agent.reset_conversation()

print(agent.chat("What is my name?"))  # history cleared, agent no longer knows

History is unbounded by default and will eventually exceed the model's context window in a long session. Set max_history_messages to cap it — the oldest turns are dropped once the limit is passed:

agent = AgentBase(..., max_history_messages=20)  # keep the last 10 turns

RAG (knowledge base)

Place your documents in a folder (organised into subdirectories by topic). Call ingest_knowledge once to embed and index them, then use run, ask, or chat — relevant chunks are automatically retrieved and injected into the system prompt on every call.

When a knowledge base is configured, the agent registers two tools the LLM can call mid-reasoning:

• search_knowledge — semantic vector search; returns the most relevant chunks for a query. Requires an ingested FAISS index.
• read_knowledge_files — keyword search by filename or file content; returns complete file text. Works directly on files without a vector index.

Both tools are enabled by default. Use knowledge_search_tool=False or knowledge_file_tool=False to disable either one.

read_knowledge_files returns at most knowledge_file_max_chars (default 20 000) per call, so a broad keyword cannot return the entire knowledge base and overflow the context window. When the limit is hit, the result ends with a note telling the model how many files were omitted and to narrow its search.

knowledge/
├── products/
│   ├── faq.md
│   └── pricing.json
└── support/
    └── sla.md

from llm_agent_base import AgentBase, LLMConnectionConfig

config = LLMConnectionConfig(model="openai/gpt-4o-mini", api_key="...")

agent = AgentBase(
    system_prompt="You are a product assistant. Answer using only the provided context.",
    llm_config=config,
    knowledge_folder_path="knowledge",
    knowledge_index_dir=".kb_index",  # where the FAISS index is saved
    knowledge_top_k=3,                # number of chunks injected per prompt
)

# Load from disk if a saved index exists, otherwise ingest and save automatically
agent.load_or_ingest_knowledge()

print(agent.run("Who founded the company and when?"))

Pass auto_load_or_ingest=True to do this in the constructor:

agent = AgentBase(
    system_prompt="You are a product assistant. Answer using only the provided context.",
    llm_config=config,
    knowledge_folder_path="knowledge",
    auto_load_or_ingest=True,
)

print(agent.run("Who founded the company and when?"))

For more control, call ingest_knowledge() and load_knowledge() directly:

# Build and persist the index (run once, or when documents change)
agent.ingest_knowledge(save=True)

# On subsequent runs, load from disk instead of re-embedding
agent.load_knowledge()

Keeping the index current

load_or_ingest_knowledge() compares a content hash of every source file against the saved index. If documents were added, edited, or deleted, only those files are re-embedded — everything else keeps its existing vector. Editing one file in a 500-document corpus costs one file's worth of embedding calls, not 500.

Call sync_knowledge() to do it explicitly:

summary = agent.sync_knowledge()
# {'added': 1, 'updated': 1, 'removed': 0,
#  'reused_chunks': 128, 'embedded_chunks': 4, 'total_chunks': 132}

Embeddings are sent in batches (100 chunks per request by default), so a first-time ingest of 1 000 chunks is 10 requests rather than 1 000.

Filtering weak matches

retrieve returns chunks in similarity order, each carrying its cosine similarity in score. By default it always returns knowledge_top_k chunks, even when none are actually relevant. Set knowledge_min_score to drop weak matches instead:

agent = AgentBase(
    ...,
    knowledge_top_k=5,
    knowledge_min_score=0.35,  # return fewer than 5 rather than pad with noise
)

Keyword file search without a vector index

read_knowledge_files searches filenames and file contents directly — no embedding or FAISS index needed. Disable search_knowledge to use only this tool:

agent = AgentBase(
    system_prompt="You are a product assistant. Answer using only the provided context.",
    llm_config=config,
    knowledge_folder_path="knowledge",
    knowledge_search_tool=False,  # no vector index required
)

print(agent.run("What are the support SLA terms?"))

The LLM can search with fine-grained control over where to look and how many keywords must match:

# filename or content, at least 2 of the 3 keywords must match
read_knowledge_files(
    keywords=["pricing plan", "enterprise", "discount"],
    search_in="both",
    match_mode="min",
    min_matches=2,
)

File attachments

All three call methods accept an optional files parameter — a list of file paths to attach to the prompt. Files are embedded directly in the message sent to the model.

Passing a file with any other extension raises a ValueError.

# Describe an image (requires a vision-capable model)
print(agent.ask("What's in this diagram?", files=["architecture.png"]))

# Summarise a PDF
print(agent.run("Summarise the key findings.", files=["report.pdf"]))

# Multi-turn with an attached file — the file stays in conversation history
agent.chat("Here's our codebase overview.", files=["overview.md"])
agent.chat("Which module handles authentication?")

Agent pipelines

Chain agents so the output of one becomes the input of the next:

from llm_agent_base import AgentBase, AgentPipelineBase, LLMConnectionConfig

config = LLMConnectionConfig(model="openai/gpt-4o-mini", api_key="...")

researcher = AgentBase(
    system_prompt="Extract the key facts from the user's question.",
    llm_config=config,
)
writer = AgentBase(
    system_prompt="Turn the provided facts into a concise, friendly summary.",
    llm_config=config,
)

pipeline = AgentPipelineBase(agents=[researcher, writer])
print(pipeline.run("Tell me about the Acme Corp product lineup."))

Debug mode

Pass debug=True to any agent to print tool invocations and knowledge retrievals to stdout:

agent = AgentBase(..., debug=True)

[debug] Retrieving knowledge
[debug] tool 'get_weather' args={'city': 'Tokyo'} result=The weather in Tokyo is sunny and 22°C.

Upgrading to 0.3.0

The knowledge index catalog is now stored as catalog.json instead of catalog.pkl. Loading an index required unpickling it, which meant a saved index could execute arbitrary code — so the pickle path was removed rather than kept as a fallback.

Existing indexes are not readable. Delete the index directory and re-ingest once:

agent.ingest_knowledge(save=True)   # or just delete .kb_index and run as usual

load_knowledge() raises a FileNotFoundError explaining this if it finds a legacy catalog.pkl.

API reference

LLMConnectionConfig

AgentBase

read_knowledge_files tool parameters (called by the LLM, not directly):

Other exports