FAIR-LLM: A Flexible, Agnostic, and Interoperable Reasoning Framework

FAIR-LLM is a Python framework designed to accelerate the development of powerful, consistent, and modular agentic applications. It provides a structured, interface-driven architecture that allows developers to easily build and customize agents with sophisticated reasoning capabilities, long-term memory, and collaborative multi-agent systems. Crafted to help engineers build powerful AI systems without vendor lock-in.

Core Principles

The FAIR framework is built on four core principles that guide its architecture and empower developers:

• Flexible: The framework is highly modular. Every core component is built upon an abstract interface, allowing developers to easily swap implementations (e.g., swapping a simple memory system for a summarizing one) without altering the agent's core logic.
• Agnostic: You are not locked into a single LLM provider. The Model Abstraction Layer (MAL) enables seamless switching between different models - from OpenAI, Anthropic, and Google Gemini to local models via HuggingFace or Ollama - ensuring you can always use the best tool for the job.
• Interoperable: Components are designed to work together seamlessly. A standardized set of data structures, like the Message and Document types, ensures that data flows consistently between the agent's memory, planner, and tools.
• Reasoning: At its heart, the framework is built to support sophisticated reasoning patterns. The default ReActPlanner allows agents to "think" step-by-step to solve complex problems, and the architecture is extensible enough to support more advanced paradigms like "Plan-and-Execute".

Key Features

• Advanced Agent Patterns: Built-in support for the powerful ReAct (Reason+Act) cognitive cycle. Two planner variants are provided: ReActPlanner (JSON-based, for powerful models like GPT-4 and Claude) and SimpleReActPlanner (key-value format, for smaller or local models that may struggle with strict JSON).
• Multi-Agent Collaboration: Orchestrate teams of specialized agents with workers-as-tools fan-out. WorkerAgentTool wraps any agent as a typed tool and build_worker_manager wires a manager that delegates independent sub-tasks concurrently in a single turn, enabling the system to tackle complex, multi-faceted problems.
• Retrieval-Augmented Generation (RAG): Easily ground agents in your own documents and data. The framework includes all necessary components for a robust RAG pipeline, including document loaders, text splitters, embedders (SentenceTransformerEmbedder), and two queryable vector store backends (FaissVectorStore and ChromaDBVectorStore) made available to the agent through the RAGQueryTool. Retrieved hits are numbered [S1]..[Sn] with a cite-only instruction; the tool accumulates a session-scoped grounded_context so answers can be verified after multiple retrieves. CitationVerifier then checks generated citations and reports a fabrication rate that distinguishes not_found from could_not_check and source_unreachable. External locator lookup is an optional AbstractSourceLookup extension point. See demos/demo_two_sided_grounding.py. FAISS is the recommended default backend.
• Pluggable Model Support: The Model Abstraction Layer (MAL) features concrete adapters for OpenAI, Anthropic, Google Gemini, HuggingFace Transformers, Ollama (local models), and LoadBalancerAdapter (distributed vLLM inference clusters), allowing for unparalleled flexibility in model selection and deployment topology.
• Modular & Extensible by Design: Every core component (LLM, Memory, Tools, Planner) is defined by an Abstract base class in the core/interfaces/ directory. This interface-driven design makes the framework easy to extend and customize.
• Prompt Engineering System: A first-class prompt engineering system lives in core/prompts/, providing composable building blocks: PromptBuilder, RoleDefinition, FormatInstruction, Example, ToolInstruction, WorkerInstruction, and more. Planners merge their own mandatory format instructions with the application content these blocks provide, so prompt content and parse format cannot drift apart.
• MCP (Model Context Protocol) Integration: A full modules/mcp/ package provides first-class support for MCP tool servers. Agents can combine local FAIR tools with tools from any MCP-compatible server (stdio or SSE transport) using MCPToolRegistry, CompositeToolRegistry, and create_mcp_enhanced_registry. Configure MCP connections via MCPServerConfig in settings.yml.
• Streaming Responses: Opt-in end-to-end token streaming through the agent loop. Enable streaming.enabled (or pass stream=True to a planner or SimpleAgent) and every model call is consumed incrementally and published as typed events (ModelStreamStartEvent, ModelStreamChunkEvent, ModelStreamEndEvent) on the agent event bus, with KVFinalAnswerStreamFilter extracting just the final-answer text for chat surfaces. SimpleReAct streams use the default unanchored filter; ReActPlanner chat surfaces must construct KVFinalAnswerStreamFilter(anchored=True) so a prose-prefixed format echo is not shown as a phantom final answer. The run's return value is unchanged. See demos/demo_streaming_agent.py.
• Secure by Design: The framework includes foundational components for security, such as the BasicSecurityManager for input validation and explicit warnings and placeholders for sandboxed tool execution to mitigate risks.
• Reliable Structured Output: Comes with patterns and demos (see demo_structured_output.py) for compelling LLMs to return clean, Pydantic-validated JSON, which is essential for reliable data extraction and tool integration.
• Agent Configuration Persistence: Save and reload complete agent configurations - including prompt definitions, tool setup, and model settings - via utilities in fairlib.modules.agent.factory. Enables reproducible agent setups, team sharing, and prompt optimization workflows.

Getting Started: Your First Agent (5-Minute Quickstart)

Follow these steps to get your first agent running.

1. Prerequisites

• Python 3.12+
• Git

2. Installation

Install warning: This framework is published on PyPI as fair-llm (hyphen), imported in Python as fairlib. The unrelated PyPI package fairlib (0.1.0) is a different project. Always pin the correct distribution, e.g. fair-llm==0.6.3 in requirements.txt or constraints.txt, and verify with pip show fair-llm before debugging import errors.

Package name vs import name: install with pip install fair-llm (hyphen). Import in Python as import fairlib. The unrelated PyPI package fairlib (0.1.0) is not this framework; pin fair-llm==0.6.3 in constraints.txt to avoid the footgun.

The slim core install ships anthropic, openai, google-genai, and httpx as first-class dependencies so the OpenAI, Anthropic, Gemini, and Ollama adapters work without optional extras. Use a virtual environment either way.

Use fairlib in a project

You want to build on the framework: install the published package, no clone needed.

pip install fair-llm                # slim: OpenAI, Anthropic, Gemini, Ollama adapters
pip install "fair-llm[local]"       # + local HuggingFace models (torch, transformers)
pip install "fair-llm[rag]"         # + RAG (chromadb, sentence-transformers, faiss)
pip install "fair-llm[documents]"   # + document ingestion (pdfplumber, pdf2image, pytesseract)
pip install "fair-llm[mcp]"         # + Model Context Protocol SDK (mcp)
pip install "fair-llm[all]"         # everything above

Note: pdf2image and pytesseract also need the poppler and tesseract system binaries, which pip cannot install (for example apt install poppler-utils tesseract-ocr).

Develop fairlib

You want to change the framework itself: clone it and install it editable, so edits under fairlib/ are live in your environment, with the test extras.

git clone https://github.com/USAFA-AI-Center/fair_llm.git
cd fair_llm
python -m venv .venv && source .venv/bin/activate
pip install -e ".[all,test]"

The same extras apply (-e ., -e ".[local]", -e ".[rag]", -e ".[documents]", -e ".[mcp]") if you want a lighter development install. Prefer pyproject.toml + uv.lock as the resolved source of truth. requirements.txt is generated from uv.lock by make requirements and is never hand-edited; refresh it at each release.

Network egress allowlist: BasicSecurityManager(allowed_egress=...) and tool network_egress declarations validate declared targets only. This is NOT a network sandbox: tools must call validate_network_egress inside acall for input-derived hosts, and the default posture permits all egress until you opt into an allowlist.

Configure the allowlist in YAML (security.allowed_egress) or at construction:

from fairlib import BasicSecurityManager, settings

security = BasicSecurityManager.from_settings(settings)

Or pass allowed_egress=settings.security.allowed_egress explicitly.

Install troubleshooting

If import fairlib fails or imports an unexpected API, verify you installed the correct PyPI distribution:

pip show fair-llm    # must show Name: fair-llm and your pinned version
python -c "import fairlib; print(fairlib.__version__)"

Common mistakes:

• pip install fairlib installs an unrelated 0.1.0 project on PyPI, not this framework.
• Installing from a capstone repo without -c constraints.txt can miss the shared fair-llm pin.

Downstream apps (for example the 2026 fall capstone templates) pin fair-llm==0.6.x in a root constraints.txt; always use pip install -r requirements.txt so the constraint file is honored.

3. Configuration

The framework uses a centralized YAML configuration file for API keys and model settings.

1. Either edit the packaged file at fairlib/config/settings.yml, or set FAIR_LLM_SETTINGS to a consumer-owned YAML path, or call configure_settings(path=...) to replace the module singleton, or pass explicit kwargs when constructing adapters. Whoever controls the process environment controls which YAML is loaded (including security-related fields); parsing uses yaml.safe_load only.
2. Add your API keys (e.g., for OpenAI, Anthropic, or Gemini).
3. Review the model definitions and other settings (RAG paths, cache, security, MCP) to suit your environment.

Local Ollama + SimpleReActPlanner

SimpleReActPlanner uses a simplified key-value response format for smaller local models that struggle with strict JSON. The agent loop tolerates consecutive PlannerParseError failures up to SimpleAgent(max_parse_attempts=...) (default 2 total parse tries before abort). Retryable transient READ_ONLY / EXTERNAL_READ tool failures can be retried within a turn via SimpleAgent(max_tool_retries=...) (default 0; set to a positive value to enable). EXTERNAL failures are never auto-retried.

1B models: Do not rely on llama3.2:1b (or similar 1B checkpoints) with SimpleReActPlanner for capstone-style agent chat. They lack reliable KV compliance; a parse retry may recover on attempt 2 but aborts are still common. Prefer 3B+ for local agent loops, or switch to ReActPlanner (JSON) with a cloud model when format compliance is critical.

3B tool_input mangling: Small models sometimes pack several schema fields into one tool_input line (host=ok-plc, port=502). SimpleReActPlanner repairs common comma-separated key=value blobs into a dict before dispatch. Executor smoke tests (tool.invoke) still do not exercise the chat path - validate with a live model when format matters.

Recommendations: prefer 3B+ models for interactive agent chat; use ReActPlanner (JSON) with cloud models when format compliance is critical; increase max_parse_attempts if you accept longer re-prompt loops on borderline models. Validate chat-path behavior separately from tool-only smoke tests.

4. Run Your First Agent

The following code assembles and runs a simple agent that can use a calculator. Save it as main.py in the root of the project directory.

# main.py
import asyncio
from fairlib import (
    settings,
    OpenAIAdapter,
    ToolRegistry,
    SafeCalculatorTool,
    ToolExecutor,
    WorkingMemory,
    ReActPlanner,
    SimpleAgent
)

async def main():
    print("Initializing a single agent...")

    # 1. The "Brain": Initialize the LLM adapter from the settings file
    llm = OpenAIAdapter(
        api_key=settings.api_keys.openai_api_key,
        model_name=settings.models["openai_gpt4"].model_name
    )

    # 2. The "Toolbelt": Create a registry and add tools
    tool_registry = ToolRegistry()
    tool_registry.register_tool(SafeCalculatorTool())

    # 3. The "Hands": Create an executor that uses the toolbelt
    executor = ToolExecutor(tool_registry)

    # 4. The "Memory": Set up short-term memory for the conversation
    memory = WorkingMemory()

    # 5. The "Mind": Create the planner that uses the brain and tools
    planner = ReActPlanner(llm, tool_registry)

    # 6. Assemble the Agent: Combine all parts into a functional unit
    agent = SimpleAgent(
        llm=llm,
        planner=planner,
        tool_executor=executor,
        memory=memory
    )
    print(" Agent created. Ask a math question or type 'exit'.")

    # 7. Run the agent in a loop
    while True:
        try:
            user_input = input("\n You: ")
            if user_input.lower() == "exit":
                break
            response = await agent.arun(user_input)
            print(f" Agent: {response}")
        except KeyboardInterrupt:
            break

if __name__ == "__main__":
    asyncio.run(main())

Run the agent from your terminal:

python main.py

Security

Network egress policy is opt-in and layered; it is not a sandbox.

What the allowlist checks: static network_egress tuples declared on each tool at registration time (when reaches_network=True), plus any runtime validate_network_egress calls tools make for input-derived hosts inside acall.

SideEffect.EXTERNAL_READ: idempotent network reads (WebSearcherTool, etc.) may run concurrently when tool_dispatch.parallel_read_only is enabled. Tools must use async I/O (or asyncio.to_thread for blocking clients) or parallelism only schedules overlap without speeding up blocking calls.

Shipped tools: WebSearcherTool declares Google PSE egress; grading tools derive egress from the injected MAL adapter; WebDataExtractor validates fetch URLs at runtime; WorkerAgentTool is in-process by default (reaches_network=False); MCP SSE adapters declare transport hosts (stdio MCP uses VERIFIED_NO_AGENT_SIDE_EGRESS).

Compensating controls you still need: OS/network firewalls, secret management, MCP server trust boundaries, sandboxed code execution (the shipped BasicSecurityManager.asandbox_code_execution is a demo stub), and reviewing tool implementations for undeclared outbound calls. Multi-agent setups must set security_manager on each agent level (build_worker_manager forwards the manager's policy recursively to nested worker executors).

Point the framework at your own config instead of the packaged settings.yml:

export FAIR_LLM_SETTINGS=/path/to/your/settings.yml

Or construct adapters with explicit kwargs and skip the fairlib.settings singleton.

Learn with Demos

The demos/ directory contains executable scripts that are the best way to learn the framework's capabilities.

• demo_single_agent_calculator.py: The best place to start. Learn the fundamentals of building and running a single agent with a simple tool.
• demo_advanced_calculator_calculus.py: Shows how to equip an agent with multiple, more complex tools.
• demo_rag_from_documents.py: A deep dive into Retrieval-Augmented Generation using ChromaDB.
• demo_two_sided_grounding.py: Cite-only source injection plus post-generation citation verification with a local HuggingFace model (FAIR_LLM_DEMO_MODEL).
• demo_agent_usage_telemetry.py: A SimpleAgent reports token usage and a forced LENGTH truncation through ModelInvocationEvent on its own bus, under Ollama and HuggingFace with no provider-specific code (FAIR_LLM_DEMO_MODEL, FAIR_LLM_DEMO_HF_MODEL).
• demo_model_manager.py: Three role aliases (tutor, student, judge) from a temp settings YAML via ModelManager / load_settings, then the same app against a complete lab settings file where the student role moves to HuggingFace. Uses FAIR_LLM_DEMO_MODEL and FAIR_LLM_DEMO_HF_MODEL; run python demos/demo_model_manager.py with no arguments.
• demo_faiss_rag_from_readme.py: RAG using the FAISS vector store backend with cross-encoder re-ranking (recommended for most use cases).
• demo_structured_output.py: Learn how to reliably extract structured JSON data from unstructured text, complete with Pydantic validation and a self-correction loop.
• demo_model_comparison.py: See the Model Abstraction Layer in action by comparing responses from different LLMs side-by-side.
• demo_multi_agent.py: The most advanced agent demo. Learn how to orchestrate a team of specialized agents to solve a complex problem that's impossible for a single agent.
• demo_committee_of_agents_*.py: Practical examples of the multi-agent architecture applied to autograding essays and code.
• demo_web_search_plot_agent.py: An agent that combines web search and graphing tools to answer data-driven questions.
• demo_orchestration_layer.py: Leased durable work queue with CheckpointWorkRunner crash/resume via session store and CostBudget (FAIR_LLM_DEMO_MODEL).

Learn with Our Developer's Guide

Our comprehensive developer's guide, "A Guide to the FAIR Agentic Framework", is written for developers of all skill levels - from newcomers to experienced AI engineers. Whether you're building your first chatbot with a tool or architecting a complex network of distributed agents, this guide will give you the principles and practical techniques you need. The guide is available in the docs/ directory.

Framework Architecture

The FAIR Agentic Framework is organized into a modular architecture. Below is an overview of the most critical components:

• core/: Contains the architectural DNA of the framework.
  • interfaces/: Abstract Base Classes defining the "contract" for every major component (AbstractChatModel, AbstractPlanner, AbstractMemory, AbstractTool, AbstractEmbedder, AbstractPerception, AbstractSecurityManager). This is the key to the framework's modularity.
  • message.py: Core conversational data structures - Message, Thought, Action, Observation, and FinalAnswer - that flow between all components.
  • types.py: Domain data types such as Document, GradingCriterion, and FinalGrade.
  • prompts/: The prompt engineering package - prompt item types in items.py, builders in builders.py, and lazy compatibility re-exports from __init__.py. History now lives under memory and AgentCapability under the capability surface while remaining importable from fairlib.core.prompts.
  • base_agent.py: The abstract BaseAgent class all agents inherit from.
  • config.py / config_schemas.py: Pydantic-validated configuration loading, including MCPServerConfig and MCPSettings schemas.
• modules/: Contains the concrete implementations of the interfaces.
  • mal/: The Model Abstraction Layer, with adapters for OpenAIAdapter, AnthropicAdapter, GeminiAdapter, OllamaAdapter, HuggingFaceAdapter, and LoadBalancerAdapter (distributed vLLM inference).
  • agent/: Agent definitions (SimpleAgent), the workers-as-tools fan-out path (WorkerAgentTool, build_worker_manager), and the agent factory (factory - build agents from config files, save them back).
  • planning/: Reasoning engines - ReActPlanner (JSON-based, for powerful models), SimpleReActPlanner (key-value format, for smaller/local models), and MultiActionReActPlanner (several tool calls per turn), all sharing BaseTextPlanner. Planners contribute only their mandatory format rules and the live tool/worker catalog to the prompt - all other content (role, examples) is supplied by your application.
  • mcp/: Full Model Context Protocol support - MCPClient, MCPToolAdapter, MCPToolRegistry, MCPServer, and create_mcp_enhanced_registry. Supports both stdio and SSE transports.
  • action/: The ToolExecutor, ToolRegistry, CompositeToolRegistry (merges multiple registries for hybrid local+MCP setups), and the tool library - built-in tools (SafeCalculatorTool, WebSearcherTool, WeatherTool, GraphingTool, WebDataExtractor) and domain tools (RAGQueryTool, GradeEssayFromRubricTool, GradeCodeFromRubricTool, CodeExecutionTool, AdvancedCalculusTool).
  • memory/: Short-term (WorkingMemory, SummarizingMemory) and long-term/RAG memory components (FaissVectorStore, ChromaDBVectorStore, InMemoryVectorStore, SentenceTransformerEmbedder, SimpleRetriever, CrossEncoderRerankingRetriever), all importable from fairlib.
  • perception/: Input preprocessing components (TextParser, EchoPreprocessor).
  • communication/: Agent-to-agent communication (InMemoryCommunicator).
  • learning/: Model selection and learning utilities.
  • security/: Security components like the BasicSecurityManager.
• config/: Centralized YAML configuration (settings.yml), including MCP server configuration.
• utils/: Utility code - document_processor, autograder_utils, math_expression_parser, rag_prompts, chart_utils, WolframGPT, and more.
• fairlib/__init__.py: The central, user-facing API providing lazy-loaded access to all framework components.
• demos/: A collection of scripts that serve as practical, executable examples of the framework's capabilities.

Folder Structure

fair_llm/
|-- fairlib/
|   |-- config/
|   |   `-- settings.yml
|   |-- core/
|   |   |-- interfaces/
|   |   |   |-- communicator.py
|   |   |   |-- embedder.py
|   |   |   |-- executor.py
|   |   |   |-- llm.py
|   |   |   |-- memory.py
|   |   |   |-- model_manager.py
|   |   |   |-- perception.py
|   |   |   |-- planner.py
|   |   |   |-- security.py
|   |   |   `-- tools.py
|   |   |-- base_agent.py
|   |   |-- config.py
|   |   |-- config_schemas.py      (includes MCPServerConfig, MCPSettings)
|   |   |-- message.py
|   |   |-- prompts/               (prompt items + builders + compatibility re-exports)
|   |   `-- types.py
|   |-- modules/
|   |   |-- action/
|   |   |   |-- tools/
|   |   |   |   |-- builtin_tools/
|   |   |   |   |   |-- data_extractor.py
|   |   |   |   |   |-- safe_calculator.py
|   |   |   |   |   |-- weather.py
|   |   |   |   |   `-- web_searcher.py
|   |   |   |   |-- advanced_calculus_tool.py
|   |   |   |   |-- code_execution_tool.py
|   |   |   |   |-- composite_registry.py  (CompositeToolRegistry)
|   |   |   |   |-- grading_tool.py
|   |   |   |   |-- graphing_tool.py
|   |   |   |   |-- rag_tool.py
|   |   |   |   `-- registry.py
|   |   |   `-- executor.py
|   |   |-- agent/
|   |   |   |-- factory.py                 (save_agent_config, load_agent)
|   |   |   |-- simple_agent.py
|   |   |   |-- worker_manager.py          (build_worker_manager)
|   |   |   `-- worker_tool.py             (WorkerAgentTool)
|   |   |-- communication/
|   |   |   `-- in_memory_communicator.py
|   |   |-- learning/
|   |   |   |-- base.py
|   |   |   `-- model_selector.py
|   |   |-- mal/
|   |   |   |-- anthropic_adapter.py
|   |   |   |-- gemini_adapter.py
|   |   |   |-- huggingface_adapter.py
|   |   |   |-- load_balancer_adapter.py   (LoadBalancerAdapter - vLLM)
|   |   |   |-- local_llama_adapter.py     (OllamaAdapter)
|   |   |   `-- openai_adapter.py
|   |   |-- mcp/
|   |   |   |-- client/
|   |   |   |   |-- mcp_client.py
|   |   |   |   |-- mcp_tool_adapter.py
|   |   |   |   `-- mcp_tool_registry.py
|   |   |   `-- server/
|   |   |       `-- mcp_server.py
|   |   |-- memory/
|   |   |   |-- base.py
|   |   |   |-- embedder.py
|   |   |   |-- retriever.py
|   |   |   |-- retriever_rerank.py
|   |   |   |-- summarization.py
|   |   |   |-- vector_faiss.py
|   |   |   `-- vector_store.py            (ChromaDB)
|   |   |-- perception/
|   |   |   |-- echo_preprocessor.py
|   |   |   `-- text_parser.py
|   |   |-- planning/
|   |   |   |-- base_text_planner.py         (shared planner shell + prompt cache)
|   |   |   |-- multi_action_planner.py    (MultiActionReActPlanner)
|   |   |   |-- react_planner.py           (ReActPlanner)
|   |   |   `-- simple_react_planner.py    (SimpleReActPlanner)
|   |   `-- security/
|   |       `-- basic_security_manager.py
|   |-- utils/
|   |   |-- autograder_utils.py
|   |   |-- chart_utils.py
|   |   |-- document_processor.py
|   |   |-- math_expression_parser.py
|   |   |-- nuextract.py
|   |   |-- rag_prompts.py
|   |   |-- speech_util.py
|   |   `-- WolframGPT.py
|   `-- __init__.py                        (central API with lazy loading)
|-- demos/
|-- docs/
|   |-- Agentic System Design Blueprint.pdf
|   |-- fair_lib_dev_doc.pdf
|   `-- latex/
|-- tests/
|-- Jenkinsfile
|-- LICENSE.md
|-- pyproject.toml
|-- README.md
|-- requirements.txt
`-- uv.lock

Contributors

Developed by the USAFA AI Center team:

• Ryan R (rrabinow@uccs.edu)
• Austin W (austin.w@ardentinc.com)
• Eli G (elijah.g@ardentinc.com)
• Chad M (Chad.Mello@afacademy.af.edu)

Contributing

Contributions are welcome! Please open an issue or submit a pull request with any enhancements, bug fixes, or new features.

Before opening a PR, run the local QA entrypoints (same recipes as CI):

make qa-fast
make qa

See docs/QA_AND_DEPENDENCIES.md and SECURITY.md for coverage floors, Dependabot policy, and dependency-update expectations.