FAIR-LLM: A Flexible, Agnostic, and Interoperable Reasoning Framework

FAIR-LLM is a Python framework designed to accelerate the development of powerful, consistent, and modular agentic applications. It provides a structured, interface-driven architecture that allows developers to easily build and customize agents with sophisticated reasoning capabilities, long-term memory, and collaborative multi-agent systems. Crafted to help engineers build powerful AI systems without vendor lock-in.

Core Principles

The FAIR framework is built on four core principles that guide its architecture and empower developers:

• Flexible: The framework is highly modular. Every core component is built upon an abstract interface, allowing developers to easily swap implementations (e.g., swapping a simple memory system for a summarizing one) without altering the agent's core logic.
• Agnostic: You are not locked into a single LLM provider. The Model Abstraction Layer (MAL) enables seamless switching between different models—from OpenAI and Anthropic to local models via HuggingFace or Ollama—ensuring you can always use the best tool for the job.
• Interoperable: Components are designed to work together seamlessly. A standardized set of data structures, like the Message and Document types, ensures that data flows consistently between the agent's memory, planner, and tools.
• Reasoning: At its heart, the framework is built to support sophisticated reasoning patterns. The default ReActPlanner allows agents to "think" step-by-step to solve complex problems, and the architecture is extensible enough to support more advanced paradigms like "Plan-and-Execute".

Key Features

• Advanced Agent Patterns: Built-in support for the powerful ReAct (Reason+Act) cognitive cycle. Two planner variants are provided: ReActPlanner (JSON-based, for powerful models like GPT-4 and Claude) and SimpleReActPlanner (key-value format, for smaller or local models that may struggle with strict JSON).
• Multi-Agent Collaboration: Orchestrate teams of specialized agents with workers-as-tools fan-out. WorkerAgentTool wraps any agent as a typed tool and build_worker_manager wires a manager that delegates independent sub-tasks concurrently in a single turn, enabling the system to tackle complex, multi-faceted problems.
• Retrieval-Augmented Generation (RAG): Easily ground agents in your own documents and data. The framework includes all necessary components for a robust RAG pipeline, including document loaders, text splitters, embedders (SentenceTransformerEmbedder), and two queryable vector store backends (FaissVectorStore and ChromaDBVectorStore) made available to the agent through the KnowledgeBaseQueryTool. FAISS is the recommended default backend.
• Pluggable Model Support: The Model Abstraction Layer (MAL) features concrete adapters for OpenAI, Anthropic, HuggingFace Transformers, Ollama (local models), and LoadBalancerAdapter (distributed vLLM inference clusters), allowing for unparalleled flexibility in model selection and deployment topology.
• Modular & Extensible by Design: Every core component (LLM, Memory, Tools, Planner) is defined by an Abstract base class in the core/interfaces/ directory. This interface-driven design makes the framework easy to extend and customize.
• Prompt Engineering System: A first-class prompt engineering system lives in core/prompts/, providing composable building blocks: PromptBuilder, RoleDefinition, FormatInstruction, Example, ToolInstruction, WorkerInstruction, and more. Planners merge their own mandatory format instructions with the application content these blocks provide, so prompt content and parse format cannot drift apart.
• MCP (Model Context Protocol) Integration: A full modules/mcp/ package provides first-class support for MCP tool servers. Agents can combine local FAIR tools with tools from any MCP-compatible server (stdio or SSE transport) using MCPToolRegistry, CompositeToolRegistry, and create_mcp_enhanced_registry. Configure MCP connections via MCPServerConfig in settings.yml.
• Streaming Responses: Opt-in end-to-end token streaming through the agent loop. Enable streaming.enabled (or pass stream=True to a planner or SimpleAgent) and every model call is consumed incrementally and published as typed events (ModelStreamStartEvent, ModelStreamChunkEvent, ModelStreamEndEvent) on the agent event bus, with KVFinalAnswerStreamFilter extracting just the final-answer text for chat surfaces. The run's return value is unchanged. See demos/demo_streaming_agent.py.
• Secure by Design: The framework includes foundational components for security, such as the BasicSecurityManager for input validation and explicit warnings and placeholders for sandboxed tool execution to mitigate risks.
• Reliable Structured Output: Comes with patterns and demos (see demo_structured_output.py) for compelling LLMs to return clean, Pydantic-validated JSON, which is essential for reliable data extraction and tool integration.
• Agent Configuration Persistence: Save and reload complete agent configurations — including prompt definitions, tool setup, and model settings — via utilities in fairlib.modules.agent.factory. Enables reproducible agent setups, team sharing, and prompt optimization workflows.

Getting Started: Your First Agent (5-Minute Quickstart)

Follow these steps to get your first agent running.

1. Prerequisites

• Python 3.12+
• Git

2. Installation

Install warning: This framework is published on PyPI as fair-llm (hyphen), imported in Python as fairlib. The unrelated PyPI package fairlib (0.1.0) is a different project. Always pin the correct distribution, e.g. fair-llm==0.6.0 in requirements.txt or constraints.txt, and verify with pip show fair-llm before debugging import errors.

Package name vs import name: install with pip install fair-llm (hyphen). Import in Python as import fairlib. The unrelated PyPI package fairlib (0.1.0) is not this framework; pin fair-llm==0.6.0 in constraints.txt to avoid the footgun.

The slim core install ships anthropic and httpx as first-class dependencies so OpenAI, Anthropic, and Ollama adapters work without optional extras.

Clone the repository and install the required dependencies. It is highly recommended to use a virtual environment.

git clone https://github.com/USAFA-AI-Center/fair_llm.git
cd fair_llm
pip install -e .

For the default slim install (OpenAI, Anthropic, Ollama adapters without local torch/RAG), use:

pip install -e .

For local HuggingFace models (torch, transformers):

pip install -e ".[local]"

For RAG (chromadb, sentence-transformers, faiss):

pip install -e ".[rag]"

For the full stack (local HuggingFace models plus RAG; [all] adds faiss-cpu on top of [local]; anthropic is in the base install):

pip install -e ".[all]"

Network egress allowlist: BasicSecurityManager(allowed_egress=...) and tool network_egress declarations validate declared targets only. This is NOT a network sandbox: tools must call validate_network_egress inside acall for input-derived hosts, and the default posture permits all egress until you opt into an allowlist.

Configure the allowlist in YAML (security.allowed_egress) or at construction:

from fairlib import settings
from fairlib.modules.security.basic_security_manager import BasicSecurityManager

security = BasicSecurityManager.from_settings(settings)

Or pass allowed_egress=settings.security.allowed_egress explicitly.

A requirements.txt is provided in the repository root for development installs with the full dependency set.

3. Configuration

The framework uses a centralized YAML configuration file for API keys and model settings.

1. Either edit the packaged file at fairlib/config/settings.yml, or set FAIR_LLM_SETTINGS to a consumer-owned YAML path, or call configure_settings(path=...) to replace the module singleton, or pass explicit kwargs when constructing adapters. Whoever controls the process environment controls which YAML is loaded (including security-related fields); parsing uses yaml.safe_load only.
2. Add your API keys (e.g., for OpenAI or Anthropic).
3. Review the model definitions and other settings (RAG paths, cache, security, MCP) to suit your environment.

Local Ollama + SimpleReActPlanner

SimpleReActPlanner uses a simplified key-value response format for smaller local models that struggle with strict JSON. The agent loop tolerates consecutive PlannerParseError failures up to SimpleAgent(max_parse_attempts=...) (default 2 total parse tries before abort). Retryable transient tool failures can be retried within a turn via SimpleAgent(max_tool_retries=...) (default 0; set to a positive value to enable).

1B models: Do not use llama3.2:1b (or similar 1B checkpoints) with SimpleReActPlanner for capstone-style agent chat. They lack reliable KV compliance. Prefer 3B+ for local agent loops, or switch to ReActPlanner (JSON) with a cloud model when format compliance is critical. If you must experiment with 1B, raise max_parse_attempts and expect frequent aborts — that is a degrade path, not a supported configuration.

3B tool_input mangling: Executor smoke tests (tool.acall / tool.invoke / direct ToolExecutor calls) validate tool wiring only; they do not exercise planner format compliance on the chat path. When tool_input arrives malformed from the model, use ReActPlanner (JSON) or a larger local model rather than trusting executor-only tests.

Recommendations: prefer 3B+ models for interactive agent chat; use ReActPlanner (JSON) with cloud models when format compliance is critical; increase max_parse_attempts if you accept longer re-prompt loops on borderline models. Validate chat-path behavior separately from tool-only smoke tests.

4. Run Your First Agent

The following code assembles and runs a simple agent that can use a calculator. Save it as main.py in the root of the project directory.

# main.py
import asyncio
from fairlib import (
    settings,
    OpenAIAdapter,
    ToolRegistry,
    SafeCalculatorTool,
    ToolExecutor,
    WorkingMemory,
    ReActPlanner,
    SimpleAgent
)

async def main():
    print("Initializing a single agent...")

    # 1. The "Brain": Initialize the LLM adapter from the settings file
    llm = OpenAIAdapter(
        api_key=settings.api_keys.openai_api_key,
        model_name=settings.models["openai_gpt4"].model_name
    )

    # 2. The "Toolbelt": Create a registry and add tools
    tool_registry = ToolRegistry()
    tool_registry.register_tool(SafeCalculatorTool())

    # 3. The "Hands": Create an executor that uses the toolbelt
    executor = ToolExecutor(tool_registry)

    # 4. The "Memory": Set up short-term memory for the conversation
    memory = WorkingMemory()

    # 5. The "Mind": Create the planner that uses the brain and tools
    planner = ReActPlanner(llm, tool_registry)

    # 6. Assemble the Agent: Combine all parts into a functional unit
    agent = SimpleAgent(
        llm=llm,
        planner=planner,
        tool_executor=executor,
        memory=memory
    )
    print(" Agent created. Ask a math question or type 'exit'.")

    # 7. Run the agent in a loop
    while True:
        try:
            user_input = input("\n You: ")
            if user_input.lower() == "exit":
                break
            response = await agent.arun(user_input)
            print(f" Agent: {response}")
        except KeyboardInterrupt:
            break

if __name__ == "__main__":
    asyncio.run(main())

Run the agent from your terminal:

python main.py

Security

Network egress policy is opt-in and layered; it is not a sandbox.

What the allowlist checks: static network_egress tuples declared on each tool at registration time (when reaches_network=True), plus any runtime validate_network_egress calls tools make for input-derived hosts inside acall.

SideEffect.EXTERNAL_READ: idempotent network reads (WebSearcherTool, etc.) may run concurrently when tool_dispatch.parallel_read_only is enabled. Tools must use async I/O (or asyncio.to_thread for blocking clients) or parallelism only schedules overlap without speeding up blocking calls.

Shipped tools: WebSearcherTool declares Google PSE egress; grading tools derive egress from the injected MAL adapter; WebDataExtractor validates fetch URLs at runtime; WorkerAgentTool is in-process by default (reaches_network=False); MCP SSE adapters declare transport hosts (stdio MCP uses VERIFIED_NO_AGENT_SIDE_EGRESS).

Compensating controls you still need: OS/network firewalls, secret management, MCP server trust boundaries, sandboxed code execution (the shipped BasicSecurityManager.asandbox_code_execution is a demo stub), and reviewing tool implementations for undeclared outbound calls. Multi-agent setups must set security_manager on each agent level (build_worker_manager forwards the manager's policy recursively to nested worker executors).

Point the framework at your own config instead of the packaged settings.yml:

export FAIR_LLM_SETTINGS=/path/to/your/settings.yml

Or construct adapters with explicit kwargs and skip the fairlib.settings singleton.

Learn with Demos

The demos/ directory contains executable scripts that are the best way to learn the framework's capabilities.

• demo_single_agent_calculator.py: The best place to start. Learn the fundamentals of building and running a single agent with a simple tool.
• demo_advanced_calculator_calculus.py: Shows how to equip an agent with multiple, more complex tools.
• demo_rag_from_documents.py: A deep dive into Retrieval-Augmented Generation using ChromaDB.
• demo_faiss_rag_from_readme.py: RAG using the FAISS vector store backend with cross-encoder re-ranking (recommended for most use cases).
• demo_structured_output.py: Learn how to reliably extract structured JSON data from unstructured text, complete with Pydantic validation and a self-correction loop.
• demo_model_comparison.py: See the Model Abstraction Layer in action by comparing responses from different LLMs side-by-side.
• demo_multi_agent.py: The most advanced agent demo. Learn how to orchestrate a team of specialized agents to solve a complex problem that's impossible for a single agent.
• demo_committee_of_agents_*.py: Practical examples of the multi-agent architecture applied to autograding essays and code.
• demo_web_search_plot_agent.py: An agent that combines web search and graphing tools to answer data-driven questions.

Learn with Our Developer's Guide

Our comprehensive developer's guide, "A Guide to the FAIR Agentic Framework", is written for developers of all skill levels—from newcomers to experienced AI engineers. Whether you're building your first chatbot with a tool or architecting a complex network of distributed agents, this guide will give you the principles and practical techniques you need. The guide is available in the docs/ directory.

Framework Architecture

The FAIR Agentic Framework is organized into a modular architecture. Below is an overview of the most critical components:

• core/: Contains the architectural DNA of the framework.
  • interfaces/: Abstract Base Classes defining the "contract" for every major component (AbstractChatModel, AbstractPlanner, AbstractMemory, AbstractTool, AbstractEmbedder, AbstractPerception, AbstractSecurityManager). This is the key to the framework's modularity.
  • message.py: Core conversational data structures — Message, Thought, Action, Observation, and FinalAnswer — that flow between all components.
  • types.py: Domain data types such as Document, GradingCriterion, and FinalGrade.
  • prompts/: The prompt engineering package — prompt item types in items.py, builders in builders.py, and lazy compatibility re-exports from __init__.py. History now lives under memory and AgentCapability under the capability surface while remaining importable from fairlib.core.prompts.
  • base_agent.py: The abstract BaseAgent class all agents inherit from.
  • config.py / config_schemas.py: Pydantic-validated configuration loading, including MCPServerConfig and MCPSettings schemas.
• modules/: Contains the concrete implementations of the interfaces.
  • mal/: The Model Abstraction Layer, with adapters for OpenAIAdapter, AnthropicAdapter, OllamaAdapter, HuggingFaceAdapter, and LoadBalancerAdapter (distributed vLLM inference).
  • agent/: Agent definitions (SimpleAgent), the workers-as-tools fan-out path (WorkerAgentTool, build_worker_manager), and the agent factory (factory - build agents from config files, save them back).
  • planning/: Reasoning engines — ReActPlanner (JSON-based, for powerful models), SimpleReActPlanner (key-value format, for smaller/local models), and MultiActionReActPlanner (several tool calls per turn), all sharing BaseTextPlanner. Planners contribute only their mandatory format rules and the live tool/worker catalog to the prompt - all other content (role, examples) is supplied by your application.
  • mcp/: Full Model Context Protocol support — MCPClient, MCPToolAdapter, MCPToolRegistry, MCPServer, and create_mcp_enhanced_registry. Supports both stdio and SSE transports.
  • action/: The ToolExecutor, ToolRegistry, CompositeToolRegistry (merges multiple registries for hybrid local+MCP setups), and the tool library — built-in tools (SafeCalculatorTool, WebSearcherTool, WeatherTool, GraphingTool, WebDataExtractor) and domain tools (KnowledgeBaseQueryTool, GradeEssayFromRubricTool, GradeCodeFromRubricTool, CodeExecutionTool, AdvancedCalculusTool).
  • memory/: Short-term (WorkingMemory, SummarizingMemory) and long-term/RAG memory components (FaissVectorStore — import via fairlib.modules.memory.vector_faiss, ChromaDBVectorStore, InMemoryVectorStore, SentenceTransformerEmbedder, SimpleRetriever, retriever_rerank).
  • perception/: Input preprocessing components (TextParser, EchoPreprocessor).
  • communication/: Agent-to-agent communication (InMemoryCommunicator).
  • learning/: Model selection and learning utilities.
  • security/: Security components like the BasicSecurityManager.
• config/: Centralized YAML configuration (settings.yml), including MCP server configuration.
• utils/: Utility code — document_processor, autograder_utils, math_expression_parser, rag_prompts, chart_utils, WolframGPT, and more.
• fairlib/__init__.py: The central, user-facing API providing lazy-loaded access to all framework components.
• demos/: A collection of scripts that serve as practical, executable examples of the framework's capabilities.

Folder Structure

fair_llm/
├── fairlib/
│   ├── config/
│   │   └── settings.yml
│   ├── core/
│   │   ├── interfaces/
│   │   │   ├── communicator.py
│   │   │   ├── embedder.py
│   │   │   ├── executor.py
│   │   │   ├── llm.py
│   │   │   ├── memory.py
│   │   │   ├── model_manager.py
│   │   │   ├── perception.py
│   │   │   ├── planner.py
│   │   │   ├── security.py
│   │   │   └── tools.py
│   │   ├── base_agent.py
│   │   ├── config.py
│   │   ├── config_schemas.py      (includes MCPServerConfig, MCPSettings)
│   │   ├── message.py
│   │   ├── prompts/               (prompt items + builders + compatibility re-exports)
│   │   └── types.py
│   ├── modules/
│   │   ├── action/
│   │   │   ├── tools/
│   │   │   │   ├── builtin_tools/
│   │   │   │   │   ├── data_extractor.py
│   │   │   │   │   ├── final_answer.py
│   │   │   │   │   ├── safe_calculator.py
│   │   │   │   │   ├── weather.py
│   │   │   │   │   └── web_searcher.py
│   │   │   │   ├── advanced_calculus_tool.py
│   │   │   │   ├── code_execution_tool.py
│   │   │   │   ├── composite_registry.py  (CompositeToolRegistry)
│   │   │   │   ├── grading_tool.py
│   │   │   │   ├── graphing_tool.py
│   │   │   │   ├── knowledge_tool.py
│   │   │   │   └── registry.py
│   │   │   └── executor.py
│   │   ├── agent/
│   │   │   ├── factory.py                 (save_agent_config, load_agent)
│   │   │   ├── simple_agent.py
│   │   │   ├── worker_manager.py          (build_worker_manager)
│   │   │   └── worker_tool.py             (WorkerAgentTool)
│   │   ├── communication/
│   │   │   └── in_memory_communicator.py
│   │   ├── learning/
│   │   │   ├── base.py
│   │   │   └── model_selector.py
│   │   ├── mal/
│   │   │   ├── anthropic_adapter.py
│   │   │   ├── huggingface_adapter.py
│   │   │   ├── load_balancer_adapter.py   (LoadBalancerAdapter — vLLM)
│   │   │   ├── local_llama_adapter.py     (OllamaAdapter)
│   │   │   └── openai_adapter.py
│   │   ├── mcp/
│   │   │   ├── client/
│   │   │   │   ├── mcp_client.py
│   │   │   │   ├── mcp_tool_adapter.py
│   │   │   │   └── mcp_tool_registry.py
│   │   │   └── server/
│   │   │       └── mcp_server.py
│   │   ├── memory/
│   │   │   ├── base.py
│   │   │   ├── embedder.py
│   │   │   ├── retriever.py
│   │   │   ├── retriever_rerank.py
│   │   │   ├── summarization.py
│   │   │   ├── vector_faiss.py
│   │   │   └── vector_store.py            (ChromaDB)
│   │   ├── perception/
│   │   │   ├── echo_preprocessor.py
│   │   │   └── text_parser.py
│   │   ├── planning/
│   │   │   ├── base_text_planner.py         (shared planner shell + prompt cache)
│   │   │   ├── multi_action_planner.py    (MultiActionReActPlanner)
│   │   │   ├── react_planner.py           (ReActPlanner)
│   │   │   └── simple_react_planner.py    (SimpleReActPlanner)
│   │   └── security/
│   │       └── basic_security_manager.py
│   ├── utils/
│   │   ├── autograder_utils.py
│   │   ├── chart_utils.py
│   │   ├── document_processor.py
│   │   ├── math_expression_parser.py
│   │   ├── nuextract.py
│   │   ├── rag_prompts.py
│   │   ├── speech_util.py
│   │   └── WolframGPT.py
│   └── __init__.py                        (central API with lazy loading)
├── demos/
├── docs/
│   ├── Agentic System Design Blueprint.pdf
│   ├── fair_lib_dev_doc.pdf
│   └── latex/
├── tests/
├── Jenkinsfile
├── LICENSE.md
├── pyproject.toml
├── README.md
├── requirements.txt
└── uv.lock

Contributors

Developed by the USAFA AI Center team:

• Ryan R (rrabinow@uccs.edu)
• Austin W (austin.w@ardentinc.com)
• Eli G (elijah.g@ardentinc.com)
• Chad M (Chad.Mello@afacademy.af.edu)

Contributing

Contributions are welcome! Please open an issue or submit a pull request with any enhancements, bug fixes, or new features.