Skip to main content

Self-improving retrieval orchestration framework with RL-based routing, conditional graph activation, and evaluation-driven learning.

Project description

adaptive-intelligence

Self-improving retrieval orchestration framework for document intelligence. Drop documents, ask questions, the system learns how to retrieve better over time.

RL-based retrieval routing, conditional graph activation, evaluation-driven learning, and zero-configuration architecture. Works with any LLM.

Open In Colab

Install

pip install adaptive-intelligence
pip install adaptive-intelligence[pdf]          # adds PDF support
pip install adaptive-intelligence[all]          # all document formats
pip install adaptive-intelligence[huggingface]  # local HuggingFace models

Quick Start

from adaptive_intelligence import AdaptiveAI

# Zero config — defaults to Ollama (free, local, private)
engine = AdaptiveAI()
engine.ingest("./documents")
response = engine.ask("What are the key operational risks?")

print(response.answer)
print(f"Confidence: {response.confidence:.0%}")
print(response.evaluation.display())
# With Grok API
engine = AdaptiveAI(
    llm_backend="openai",
    llm_model="grok-3-mini",
    api_key="xai-...",
    base_url="https://api.x.ai/v1",
    domain="financial",
)

# With OpenAI
engine = AdaptiveAI(
    llm_backend="openai",
    llm_model="gpt-4o",
    api_key="sk-...",
)

What Makes This Different

Traditional RAG Adaptive Intelligence
Retrieval Static vector similarity RL-learned routing (6 strategies)
Graph None Conditional activation (5-signal gate)
Prompts Fixed template Domain-adaptive, evolving
Learning Same performance forever Improves with each query
Evaluation Manual Automatic 6-metric + RL reward

Three Core Innovations

1. RL Policy Engine

Contextual bandits with Thompson Sampling learn which retrieval strategy works best for each query type. First 15 queries use heuristic defaults, then RL takes over. No hardcoded rules.

# The RL policy decides per query:
# - Retrieval route: vector, keyword, hybrid, table-first, graph-hybrid
# - Retrieval depth: 3, 5, 8, 10, 15 chunks
# - Graph activation: on/off
# - Prompt template: extraction, analysis, summary, comparison
# - Verification level: none, citation, full

2. Conditional Graph Activation

Knowledge graph built automatically during ingestion. Activated only when the query needs relational reasoning — not wasted on simple factual lookups.

Five signals gate activation: relationship words, query analysis, entity density, complexity, historical success rate.

3. Self-Adaptive Retrieval

Every response evaluated on 6 metrics. Composite score becomes RL reward. System measurably improves over queries.

# Evaluation metrics (all automatic, no ground truth needed):
# - Faithfulness: grounded in source documents?
# - Relevance: addresses the query?
# - Citation accuracy: sources cited?
# - Hallucination risk: fabricated content?
# - Retrieval precision: relevant chunks retrieved?
# - Retrieval recall: query terms covered?

Supported Formats

S.No. Format Extension Required Package
1 Text / Markdown .txt, .md
2 CSV .csv
3 JSON .json
4 HTML .html
5 XML .xml
6 PDF .pdf PyMuPDF or pdfplumber
7 Word .docx python-docx
8 Excel .xlsx openpyxl
9 PowerPoint .pptx python-pptx
10 Images (OCR) .png, .jpg pytesseract, Pillow

Supported LLM Providers

S.No. Provider Backend Local? Free?
1 Ollama ollama Yes Yes
2 OpenAI openai No No
3 Grok (xAI) openai No No
4 Azure OpenAI azure_openai No No
5 Groq groq No Free tier
6 Together AI together No Free tier
7 HuggingFace huggingface Yes Yes
8 Any OpenAI-compatible custom Varies Varies

Code Examples

Inspect the Full Pipeline

response = engine.ask("Compare Q2 and Q3 revenue")

# What did the system understand?
print(response.query_analysis)

# What strategy did the RL policy choose?
pd = response.policy_decision
print(f"Route: {pd.retrieval_route}")
print(f"Graph: {pd.graph_activation}")
print(f"Explored: {pd.was_exploration}")

# Evaluation scores
print(response.evaluation.display())

# Citations
for c in response.citations:
    print(f"  {c.source_document} ({c.confidence:.0%})")

Dashboard and Monitoring

# System dashboard
print(engine.dashboard())

# RL policy stats
stats = engine.rl.get_stats()
print(f"Warmup: {stats['is_warmup']}")
print(f"Arms learned: {stats['total_arms']}")
print(f"Exploration: {stats['exploration_rate']:.1%}")

# Learning curve data
curve = engine.learning_curve()

# Learning memory
print(engine.memory.get_learning_summary())

# Audit trail
print(engine.audit.display_query_trail(response.query_id))
engine.audit.export("audit.json")

Advanced Configuration

from adaptive_intelligence.core.config import (
    AdaptiveConfig, RLConfig, GraphConfig, EvaluationConfig,
    LLMBackend, Domain, SecurityLevel,
)

config = AdaptiveConfig(
    llm_backend=LLMBackend.OLLAMA,
    llm_model="llama3.2",
    domain=Domain.FINANCIAL,
    security_level=SecurityLevel.HIGH,

    rl=RLConfig(
        warmup_queries=20,
        exploration_rate=0.15,
        algorithm="thompson_sampling",
    ),

    graph=GraphConfig(
        enabled=True,
        conditional_activation=True,
        max_hops=3,
    ),

    evaluation=EvaluationConfig(
        faithfulness_weight=0.35,
        enable_llm_judge=True,
    ),
)

engine = AdaptiveAI(config=config)

Architecture

systemarchitecure


Also by the Author

Citation

@software{venkatkumar2026adaptive,
  title={Adaptive Intelligence: Self-Improving Retrieval Orchestration via Evaluation-Driven Policy Learning},
  author={Venkatkumar, Rajan},
  year={2026},
  url={https://github.com/VK-Ant/adaptive-intelligence}
}

License

Apache License 2.0

Author

Venkatkumar Rajan

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

adaptive_intelligence-1.0.1.tar.gz (64.1 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

adaptive_intelligence-1.0.1-py3-none-any.whl (61.8 kB view details)

Uploaded Python 3

File details

Details for the file adaptive_intelligence-1.0.1.tar.gz.

File metadata

  • Download URL: adaptive_intelligence-1.0.1.tar.gz
  • Upload date:
  • Size: 64.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.14.2

File hashes

Hashes for adaptive_intelligence-1.0.1.tar.gz
Algorithm Hash digest
SHA256 713e1cb862931e8957f33ca7ea42809ddd4cab4b0924a9265e04161e84d3c657
MD5 c18402ca3a2c96458103690423deebd1
BLAKE2b-256 ba855babe6e333fa4538548cbd273c946fe6beafbe9e3fee491aa268951231f7

See more details on using hashes here.

File details

Details for the file adaptive_intelligence-1.0.1-py3-none-any.whl.

File metadata

File hashes

Hashes for adaptive_intelligence-1.0.1-py3-none-any.whl
Algorithm Hash digest
SHA256 d175aac3952c7f457f039daa1762ca749c728891c82bfab08834eb87bd2a6633
MD5 86d4e96a8e9c448f4ed41f5584aa2d78
BLAKE2b-256 2823eb5e2474dbd2559dded02983987c7608cecb0a511ffdbbe2359fee5defb1

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page