Agent LLM Service
Agent LLM Service is a robust, clean, and extensible Python library designed for executing LLM requests across multiple language models (e.g., OpenAI, Groq, Gemini). It offers reliable fallback mechanics, model round-robins, API rate-limit management, built-in tool/function-calling structures, and exponential backoff retry logic.
It is built as the core execution engine for any multi-agent system, strictly adhering to modern Python typing (Pydantic v2) and asynchronous request handling (httpx, asyncio).
Key Features
- Execution Pool & Failover Strategy: Automatically cycle through fallback models when encountering rate limits or temporary provider issues (
LlmExecutionPool). - Resilient Polling & Backoffs: Built-in retry with exponential backoff (
LlmRunner). - Modular Provider Interface: Easily write adapters and unified
RawLlmProviderhandlers that work with OpenAI-spec endpoints. - Native Tool Calling Definitions: Extensively typed schemas to define custom tools (
BaseTool) and dispatch LLM decisions automatically mappings to real functions (ToolRegistry). - Asynchronous by Default: Built utilizing async (
httpx.AsyncClient) for scaling safely across multi-agent fleets.
Getting Started
Installation
Ensure you are using Python >= 3.12. Install dependencies using uv or pip:
uv add agent-llm-service
# or you can also use with pip
# pip install agent-llm-service
Quick Usage
from agent_llm_service import LlmProviderConfig, RawLlmProvider, LlmRunner
from dotenv import load_dotenv
load_dotenv() # Load environment variables from .env file
async def main():
# Configure the provider (e.g., Groq via OpenAI schema)
config = LlmProviderConfig(
name="Groq",
slug="groq",
api_key_env_var="GROQ_API_KEY",
base_url="https://api.groq.com/openai/v1",
enabled=True,
)
provider = RawLlmProvider(config=[config])
runner = LlmRunner(provider=provider)
# Async LLM Run
response = await runner.acall(
model="groq/openai/gpt-oss-20b",
messages=[{"role": "user", "content": "What is the capital of France?"}],
)
print(response.content)
if __name__ == "__main__":
import asyncio
asyncio.run(main())
Documentation
Note: If not shown directly on PyPi package, try seeing in Github repo: SurajAiri/Agent-LLM-Service
- Architecture Review: Deep dive into the modular structure, object domains, and the pool failover strategies.
- How to Use (Examples & Tool Calling): Extensive recipes on handling single LLM runs, pools, and tool registry integrations.
Contributing
- Fork the repository and create an issue.
- Ensure you run the linter (
uv run ruff check src --fix). - Submit a PR.
License
MIT License.
Metadata
Release files for agent-llm-service 0.0.4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| agent_llm_service-0.0.4.tar.gz | 12.4 kB | Without provenance | Details for agent_llm_service-0.0.4.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| agent_llm_service-0.0.4-py3-none-any.whl | Python 3 | none | any | Details for agent_llm_service-0.0.4-py3-none-any.whl |
Total release size: 27.4 kB
Release files / agent_llm_service-0.0.4.tar.gz
| Download URL | agent_llm_service-0.0.4.tar.gz |
|---|---|
| Size | 12.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
96e5fe686c2f11ad2c3a74c934aa2652b69c953b961a376d84e58ee0caafdc5e
|
|
BLAKE2b-256 checksum How to use checksums |
e3a00b7650660515f748438b8837fc68ebb4e2b523aac27ed54442f94e5d07f3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.8.8
|
Release files / agent_llm_service-0.0.4-py3-none-any.whl
| Download URL | agent_llm_service-0.0.4-py3-none-any.whl |
|---|---|
| Size | 15.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
a53d59017ae486af940a682f2678dc9802bc54af928ac6e1a2c77176b73ec59e
|
|
BLAKE2b-256 checksum How to use checksums |
aeafe1c024ecd27ab4451807141447a88e2f7cf74f782bd896ba551af6fa4cb4
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.8.8
|
Security for agent-llm-service 0.0.4
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.