llm-context-toolkit

Version: 0.1.0
Author: sreeyenan
License: MIT
Python: 3.10+

A comprehensive toolkit for LLM context management with intelligent metadata selection and contextual question resolution.

🎯 What This Library Does

llm-context-toolkit solves two critical problems when building LLM-powered AI systems:

1. Context Resolution - Converts short, contextual messages into complete standalone instructions
2. Metadata Selection - Intelligently selects only relevant metadata from large catalogs

The Problem

When building Text-to-SQL, RAG systems, or dashboard AI assistants, you face these challenges:

• Contextual questions are hard to understand: "make it monthly", "what else?", "same for payments"
• Full metadata catalogs are too large: Sending 50+ tables with 500+ columns to an LLM wastes tokens, increases cost, and reduces accuracy
• Chat history grows unbounded: Passing full conversation history is expensive and often unnecessary

The Solution

from llm_context_toolkit import resolve_and_select

# One line to resolve context + select metadata
resolution, selection = resolve_and_select(
    current_question="make it monthly",
    metadata_catalog=tenant_catalog,
    context_cards=[previous_turn_card],
    database="client_db"
)

# Result: 
# - Resolved question: "Show monthly revenue by region for last 6 months"
# - Selected metadata: 1 table, 3 columns (from 42 tables, 890 columns)
# - Token reduction: 95.5% (47000 → 2100 tokens)

🚀 Key Features

Context Resolution

• ✅ Converts contextual messages into standalone questions
• ✅ Detects follow-ups, modifications, fresh questions, and suggestion requests
• ✅ Maintains state using compact TurnContextCard objects
• ✅ No full chat history needed (90%+ memory reduction)
• ✅ Rule-based resolution (no API calls required)
• ✅ Optional LLM fallback for ambiguous cases

Metadata Selection

• ✅ Selects only relevant tables and columns from large catalogs
• ✅ 90-98% token reduction for LLM prompts
• ✅ Multiple strategies: lexical, TF-IDF, embeddings, hybrid
• ✅ No API key required for core functionality
• ✅ Supports business metadata (synonyms, descriptions, roles)
• ✅ Works with any database schema (SQL, NoSQL, graph, etc.)

Framework-Agnostic

• ✅ Works with any LLM (OpenAI, Anthropic, Gemini, Groq, local models)
• ✅ Works with any SQL engine (PostgreSQL, MySQL, ClickHouse, etc.)
• ✅ Works with any framework (FastAPI, Flask, Django, etc.)
• ✅ Works with any vector database for RAG
• ✅ Pure Python with zero required dependencies

📦 Installation

Minimal Installation (Pure Python)

pip install llm-context-toolkit

With Optional Enhanced Features

# TF-IDF strategy for better lexical matching
pip install llm-context-toolkit[tfidf]

# Local embedding support for semantic matching
pip install llm-context-toolkit[embeddings]

# Everything
pip install llm-context-toolkit[all]

🎓 Quick Start

1. Basic Context Resolution

from llm_context_toolkit import ContextResolver, TurnContextCard

# Previous conversation turn
previous_card = TurnContextCard(
    turn_id=1,
    user_question="Show total revenue by region",
    resolved_question="Show total revenue by region",
    query_state={
        "active_tables": ["sales_orders"],
        "metrics": ["revenue"],
        "group_by": ["region"],
    }
)

# Resolve a contextual follow-up
resolver = ContextResolver()
resolution = resolver.resolve(
    current_question="make it monthly",
    context_cards=[previous_card]
)

print(resolution.resolved_question)
# "Show monthly revenue by region"

print(resolution.metadata_search_query)
# "monthly revenue region order_date sales_orders"

2. Basic Metadata Selection

from llm_context_toolkit import MetadataSelector

catalog = {
    "database": "client_db",
    "tables": [
        {
            "table_name": "sales_orders",
            "columns": [
                {"name": "revenue", "role": "metric", "synonyms": ["sales", "income"]},
                {"name": "region", "role": "dimension", "synonyms": ["zone", "area"]},
                {"name": "order_date", "role": "date"},
            ]
        }
    ]
}

selector = MetadataSelector()
result = selector.select(
    question="Show total sales by region for last 6 months",
    catalog=catalog
)

print(result.selected_metadata)
# Only relevant table and columns selected
print(result.metrics.summary())
# "tables 42 → 1 (97.6% cut), columns 890 → 3 (99.7% cut), tokens ~47000 → ~500 (98.9% cut)"

3. Combined Workflow (Recommended)

from llm_context_toolkit import resolve_and_select, TurnContextCard

# One call to handle both resolution and selection
resolution, selection = resolve_and_select(
    current_question="consider monthly?",
    metadata_catalog=tenant_catalog,
    context_cards=[previous_card],
    database="client_abc_db"
)

# Handle clarification requests
if resolution.route == "clarify":
    return {"message": resolution.clarification_question}

# Now pass to your LLM
prompt = f"""
Question: {resolution.resolved_question}

Available Schema:
{selection.selected_metadata}

Generate SQL for this question.
"""

# Send to OpenAI, Anthropic, Gemini, etc.
sql = your_llm.generate(prompt)

🔄 Typical Pipeline

User Message: "make it monthly"
        ↓
ContextResolver
        ↓
Resolved: "Show monthly revenue by region for last 6 months"
Search Query: "monthly revenue region order_date"
        ↓
MetadataSelector
        ↓
Selected: 1 table (sales_orders), 3 columns (revenue, region, order_date)
Token Reduction: 47000 → 2100 tokens (95.5% reduction)
        ↓
LLM Prompt
        ↓
SQL/Config/Answer Generation
        ↓
Save new TurnContextCard

📖 Use Cases

Text-to-SQL

# User: "Show revenue by region"
# → SQL: SELECT region, sum(revenue) FROM sales_orders GROUP BY region

# User: "make it monthly"
# → SQL: SELECT toStartOfMonth(order_date) AS month, region, sum(revenue) 
#        FROM sales_orders GROUP BY month, region

Text-to-Query-Config

# User: "Show revenue by region as a bar chart"
# → Config: {"type": "bar", "x": "region", "y": "sum(revenue)"}

RAG / Document Q&A

# Previous: "What are the compliance requirements?"
# User: "what about privacy only?"
# → Resolved: "What are the privacy-specific compliance requirements?"

Dashboard AI Assistant

# User: "what else can we analyze?"
# → Suggestions: ["Revenue by product category", "Monthly trend analysis", "Top 10 regions"]

🏗️ Architecture

Context Resolution Flow

User Message
    ↓
Detect explicit references (turn #3, chart 2, etc.)
    ↓
Rank recent context cards by relevance
    ↓
Classify: fresh | follow-up | modify | suggest | clarify
    ↓
Extract intent delta (granularity, filters, etc.)
    ↓
Compose resolved question + metadata search query
    ↓
Output: ContextResolution

Metadata Selection Flow

Question + Full Catalog (42 tables, 890 columns)
    ↓
Parse intent (metrics, dates, grouping, trends)
    ↓
Score tables (lexical + TF-IDF + embeddings)
    ↓
Select top K tables (default: 3)
    ↓
Score columns within selected tables
    ↓
Select top N columns per table (default: 25)
    ↓
Include relationships between selected tables
    ↓
Output: Selected metadata (1-3 tables, 5-30 columns)

🎯 Core Concepts

TurnContextCard

Compact memory object storing structured context from a previous AI turn.

TurnContextCard(
    turn_id=1,
    user_question="Show revenue by region",
    resolved_question="Show total revenue by region for last 6 months",
    task_type="text_to_sql",
    artifact_type="sql",
    topic_keywords=["revenue", "region", "sales"],
    entities=["sales_orders", "revenue", "region"],
    query_state={
        "active_tables": ["sales_orders"],
        "metrics": ["revenue"],
        "dimensions": ["region"],
        "filters": [{"column": "order_date", "operation": "last_6_months"}],
    },
    sql="SELECT region, sum(revenue) FROM sales_orders WHERE order_date >= ...",
)

Why not store full chat history?

• Full history: 1000+ tokens per turn × 10 turns = 10,000+ tokens
• Context cards: ~200 tokens per turn × 10 turns = 2,000 tokens
• 80% reduction before metadata selection even starts

ContextType

Classification of how the current question relates to previous context:

• fresh_question - New independent question
• follow_up_latest - Continues from latest result
• follow_up_specific_turn - References a specific older turn
• modify_previous_query - Modifies a previous result
• suggestion_request - Asks for related suggestions
• clarification_needed - Too ambiguous to resolve

Selection Strategies

• lexical - Pure Python keyword matching (no dependencies)
• tfidf - TF-IDF scoring (requires scikit-learn)
• embedding - Semantic similarity (requires sentence-transformers)
• hybrid - Combines all available strategies (recommended)

🔧 Configuration

Resolver Configuration

from llm_context_toolkit import ResolverConfig, ContextResolver

config = ResolverConfig(
    max_recent_cards=10,              # How many recent cards to consider
    min_confidence_threshold=0.75,    # Minimum confidence to resolve
    allow_llm_fallback=False,         # Use LLM for ambiguous cases
    enable_debug=True,                # Include debug information
)

resolver = ContextResolver(config=config)

Selector Configuration

from llm_context_toolkit import SelectorConfig, MetadataSelector

config = SelectorConfig(
    strategy="hybrid",                # lexical | tfidf | embedding | hybrid
    top_k_tables=3,                   # Max tables to select
    max_columns_per_table=25,         # Max columns per table
    max_output_chars=10000,           # Budget limit for output
    use_embeddings_if_available=True, # Use embeddings if installed
    embedding_model_name_or_path="./models/all-MiniLM-L6-v2",  # Local model
    include_debug=True,               # Include scoring details
)

selector = MetadataSelector(config=config)

📊 Real-World Results

Token Reduction Examples

Cost Impact

Assuming GPT-4 pricing ($10/1M input tokens):

• Large catalog: 30,000 tokens → 1,200 tokens = $0.30 → $0.012 per query (96% cost reduction)
• 10,000 queries/month: $3,000 → $120/month savings = $34,560/year

🔬 Advanced Usage

Custom LLM Fallback for Ambiguous Cases

from llm_context_toolkit import ContextResolver

def my_llm_fallback(question, card_summaries, preliminary):
    # Call your LLM to resolve ambiguous cases
    prompt = f"Question: {question}\nContext: {card_summaries}\nResolve this."
    response = your_llm_client.generate(prompt)
    return {"resolved_question": response["question"], ...}

resolver = ContextResolver(llm_fallback=my_llm_fallback)

Multi-Tenant Isolation

# Always pass database override from your tenant resolver
client_id = request.headers["X-Client-ID"]
database = tenant_resolver.get_database(client_id)

resolution, selection = resolve_and_select(
    current_question=user_question,
    metadata_catalog=tenant_catalogs[client_id],
    database=database,  # Ensures LLM only sees this tenant's data
)

State Extraction for Next Turn

# After generating SQL/config, extract state for next turn
from llm_context_toolkit import TurnContextCard

next_card = TurnContextCard(
    turn_id=current_turn_id + 1,
    user_question=resolution.current_question,
    resolved_question=resolution.resolved_question,
    query_state={
        "active_tables": [t["table_name"] for t in selection.selected_metadata["tables"]],
        "metrics": extract_metrics_from_sql(generated_sql),
        "dimensions": extract_dimensions_from_sql(generated_sql),
        # ... extract other state
    },
    sql=generated_sql,
)

# Store for next turn
context_store.save(session_id, next_card)

🧪 Testing

# Run tests
pytest

# Run with coverage
pytest --cov=llm_context_toolkit

# Run specific test
pytest tests/test_context_resolver.py::test_fresh_question

📚 Documentation

• Full User Manual - Complete reference guide
• Context Resolver Guide - Detailed context resolution documentation
• Metadata Selector Guide - Detailed metadata selection documentation
• Examples - Code examples for common use cases

🤝 Contributing

Contributions are welcome! Please:

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

This library was built to solve real-world problems in multi-tenant analytics systems with Text-to-SQL, dashboard AI assistants, and natural language query interfaces.

📧 Support

• 📖 Documentation
• � Changelog
• 🔧 Environment Variables
• 🐛 Issue Tracker
• 💬 Discussions

Made with ❤️ by sreeyenan