Skip to main content

Self-improving retrieval orchestration framework with RL-based routing, conditional graph activation, and evaluation-driven learning.

Project description

adaptive-intelligence

Self-improving retrieval orchestration framework for document intelligence. Drop documents, ask questions, the system learns how to retrieve better over time.

RL-based retrieval routing, conditional graph activation, evaluation-driven learning, and zero-configuration architecture. Works with any LLM.

Open In Colab

Install

pip install adaptive-intelligence
pip install adaptive-intelligence[pdf]          # adds PDF support
pip install adaptive-intelligence[all]          # all document formats
pip install adaptive-intelligence[huggingface]  # local HuggingFace models

Quick Start

from adaptive_intelligence import AdaptiveAI

# Zero config — defaults to Ollama (free, local, private)
engine = AdaptiveAI()
engine.ingest("./documents")
response = engine.ask("What are the key operational risks?")

print(response.answer)
print(f"Confidence: {response.confidence:.0%}")
print(response.evaluation.display())
# With Grok API
engine = AdaptiveAI(
    llm_backend="openai",
    llm_model="grok-3-mini",
    api_key="xai-...",
    base_url="https://api.x.ai/v1",
    domain="financial",
)

# With OpenAI
engine = AdaptiveAI(
    llm_backend="openai",
    llm_model="gpt-4o",
    api_key="sk-...",
)

What Makes This Different

Traditional RAG Adaptive Intelligence
Retrieval Static vector similarity RL-learned routing (6 strategies)
Graph None Conditional activation (5-signal gate)
Prompts Fixed template Domain-adaptive, evolving
Learning Same performance forever Improves with each query
Evaluation Manual Automatic 6-metric + RL reward

Three Core Innovations

1. RL Policy Engine

Contextual bandits with Thompson Sampling learn which retrieval strategy works best for each query type. First 15 queries use heuristic defaults, then RL takes over. No hardcoded rules.

# The RL policy decides per query:
# - Retrieval route: vector, keyword, hybrid, table-first, graph-hybrid
# - Retrieval depth: 3, 5, 8, 10, 15 chunks
# - Graph activation: on/off
# - Prompt template: extraction, analysis, summary, comparison
# - Verification level: none, citation, full

2. Conditional Graph Activation

Knowledge graph built automatically during ingestion. Activated only when the query needs relational reasoning — not wasted on simple factual lookups.

Five signals gate activation: relationship words, query analysis, entity density, complexity, historical success rate.

3. Self-Adaptive Retrieval

Every response evaluated on 6 metrics. Composite score becomes RL reward. System measurably improves over queries.

# Evaluation metrics (all automatic, no ground truth needed):
# - Faithfulness: grounded in source documents?
# - Relevance: addresses the query?
# - Citation accuracy: sources cited?
# - Hallucination risk: fabricated content?
# - Retrieval precision: relevant chunks retrieved?
# - Retrieval recall: query terms covered?

Supported Formats

S.No. Format Extension Required Package
1 Text / Markdown .txt, .md
2 CSV .csv
3 JSON .json
4 HTML .html
5 XML .xml
6 PDF .pdf PyMuPDF or pdfplumber
7 Word .docx python-docx
8 Excel .xlsx openpyxl
9 PowerPoint .pptx python-pptx
10 Images (OCR) .png, .jpg pytesseract, Pillow

Supported LLM Providers

S.No. Provider Backend Local? Free?
1 Ollama ollama Yes Yes
2 OpenAI openai No No
3 Grok (xAI) openai No No
4 Azure OpenAI azure_openai No No
5 Groq groq No Free tier
6 Together AI together No Free tier
7 HuggingFace huggingface Yes Yes
8 Any OpenAI-compatible custom Varies Varies

Code Examples

Inspect the Full Pipeline

response = engine.ask("Compare Q2 and Q3 revenue")

# What did the system understand?
print(response.query_analysis)

# What strategy did the RL policy choose?
pd = response.policy_decision
print(f"Route: {pd.retrieval_route}")
print(f"Graph: {pd.graph_activation}")
print(f"Explored: {pd.was_exploration}")

# Evaluation scores
print(response.evaluation.display())

# Citations
for c in response.citations:
    print(f"  {c.source_document} ({c.confidence:.0%})")

Dashboard and Monitoring

# System dashboard
print(engine.dashboard())

# RL policy stats
stats = engine.rl.get_stats()
print(f"Warmup: {stats['is_warmup']}")
print(f"Arms learned: {stats['total_arms']}")
print(f"Exploration: {stats['exploration_rate']:.1%}")

# Learning curve data
curve = engine.learning_curve()

# Learning memory
print(engine.memory.get_learning_summary())

# Audit trail
print(engine.audit.display_query_trail(response.query_id))
engine.audit.export("audit.json")

Advanced Configuration

from adaptive_intelligence.core.config import (
    AdaptiveConfig, RLConfig, GraphConfig, EvaluationConfig,
    LLMBackend, Domain, SecurityLevel,
)

config = AdaptiveConfig(
    llm_backend=LLMBackend.OLLAMA,
    llm_model="llama3.2",
    domain=Domain.FINANCIAL,
    security_level=SecurityLevel.HIGH,

    rl=RLConfig(
        warmup_queries=20,
        exploration_rate=0.15,
        algorithm="thompson_sampling",
    ),

    graph=GraphConfig(
        enabled=True,
        conditional_activation=True,
        max_hops=3,
    ),

    evaluation=EvaluationConfig(
        faithfulness_weight=0.35,
        enable_llm_judge=True,
    ),
)

engine = AdaptiveAI(config=config)

Architecture

Query → Trigger Interpreter → RL Policy Decision → Retrieval
                                    ↓                   ↓
                              Graph Traversal    Vector + Keyword
                              (conditional)      (hybrid RRF)
                                    ↓                   ↓
                              Adaptive Prompt ← ────────┘
                                    ↓
                              LLM Generation
                                    ↓
                              Evaluation Engine
                                    ↓
                    ┌───────────────┼───────────────┐
                    ↓               ↓               ↓
              RL Update      Memory Update    Prompt Evolution
              (reward)       (patterns)       (template scores)

Also by the Author

Citation

@software{venkatkumar2026adaptive,
  title={Adaptive Intelligence: Self-Improving Retrieval Orchestration via Evaluation-Driven Policy Learning},
  author={Venkatkumar, Rajan},
  year={2026},
  url={https://github.com/VK-Ant/adaptive-intelligence}
}

License

Apache License 2.0

Author

Venkatkumar Rajan

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

adaptive_intelligence-1.0.0.tar.gz (64.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

adaptive_intelligence-1.0.0-py3-none-any.whl (61.9 kB view details)

Uploaded Python 3

File details

Details for the file adaptive_intelligence-1.0.0.tar.gz.

File metadata

  • Download URL: adaptive_intelligence-1.0.0.tar.gz
  • Upload date:
  • Size: 64.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.14.2

File hashes

Hashes for adaptive_intelligence-1.0.0.tar.gz
Algorithm Hash digest
SHA256 ee6a53e53cf8a7867256176213d9267b51d4607d69d619da480b5ed55902f386
MD5 1f21355e9791b35160de3deb0c7f278d
BLAKE2b-256 b5e4bee2375028eb48e5e520c3f6faedae6cac138d775269927009297ab717ce

See more details on using hashes here.

File details

Details for the file adaptive_intelligence-1.0.0-py3-none-any.whl.

File metadata

File hashes

Hashes for adaptive_intelligence-1.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 eedc4de75b6c01e705f1a6f71e217838218a0f45725e9b4a25f7d49aa0eed806
MD5 d4e05e90b5c43930d9526d24b8ea88c5
BLAKE2b-256 2731226a00641f1c2ba743dcc106cb8dd9b90f879c2ba4fdf849d393ed4c2606

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page