cognition-horizon

A linter for LLM output quality.

Detects when an AI response drifts from the original intent, hedges instead of answers, or escalates in intensity without adding information.

Install

pip install cognition-horizon

Quick Start

from cognition_horizon.engine import CognitionHorizon

analyzer = CognitionHorizon()

intent = "Explain how to implement rate limiting in a FastAPI application"
output = """
Rate limiting is a crucial concept in modern API design. It's important to 
consider various factors when implementing it. There are many approaches 
available, and the best one depends on your specific use case. Generally 
speaking, you might want to think about using middleware, which can be 
quite effective in many scenarios. The implementation details can vary 
significantly based on your requirements, and it's worth exploring the 
different options available to you.
"""

result = analyzer.reflect(intent=intent, output=output)
print(result.to_dict())

What It Measures

Chi Drift - keyword distance between what was asked and what was answered. A score of 0.0 means the response directly addresses the intent. A score of 1.0 means the response is about something else entirely.

Hedging Density - ratio of qualifying language to total content. "might", "could", "generally speaking", "it depends" - these are hedges. High hedging density means the model is avoiding commitment.

Escalation - language intensity increases in the second half. Models sometimes generate excitement rather than precision. Escalation detection catches this pattern.

Preamble - filler at the start before the actual answer begins. "Great question! I'd be happy to help..." adds no information.

The ?h(t)dt Accumulator

The integral of the cognition horizon accumulates a running score of AI quality across traces, stripping away excess verbiage to surface overlooked insights.

Use Cases

• Automated QA Pipeline: Fail PRs where LLM output drift exceeds 0.4
• Prompt Iteration: Compare hedging density across prompt versions
• RAG Refinement: Detect when a model hallucinates excitement instead of providing facts

API Reference

Start the API:

uvicorn cognition_horizon.app:app --reload

Endpoint: POST /api/v1/reflect Payload: {"intent": "...", "output": "..."} Returns: JSON with drift, compression ratio, excess markers, and insights.

Philosophy

"System illuminates. Human recognizes." This package provides the illumination - making the invisible patterns of AI drift visible, so you can recognize when it's failing.