persiq 0.1.0

Semantic persona drift detection for LLM character and agent applications

🎭 PersIQ

Stop your AI characters from breaking character. Real-time semantic persona drift detection for LLM apps.

---

The Problem

You build an AI character — a Victorian detective, a pirate, a customer support agent with a specific tone. It works great for the first 10 messages. Then slowly it starts acting like a generic chatbot.

Persona drift is silent, gradual, and almost impossible to catch manually.

Turn 01: "Elementary, my dear Watson."                    ✅ Perfect
Turn 08: "I deduce the suspect arrived from the east."    ✅ Good
Turn 15: "Sure! I'd be happy to help you with that! 😊"  ❌ Drifting
Turn 20: "lol yeah totally sounds good to me"            🚨 Severe Drift

The Solution

PersIQ embeds your character definition as an anchor vector, then scores every assistant message for semantic distance from that anchor using a sliding window. When drift is detected, it fires configurable alerts.

Character Definition  →  Anchor Vector
Each LLM Response     →  Response Vector  
Cosine Distance       →  Drift Score (0.0 = perfect, 1.0 = max drift)
Sliding Window Mean   →  Smoothed Score
Threshold Check       →  Alert / Log / Callback / Exception

---

Install

pip install persiq

With OpenAI embeddings:

pip install persiq[openai]

---

Quickstart

Real-Time Tracking

from persiq import PersonaDefinition, SentenceTransformerEmbedder, Tracker

persona = PersonaDefinition(
    name="Sherlock Holmes",
    description="Cold, analytical, brilliant Victorian detective.",
    traits=["logical", "observant", "aloof", "sardonic"],
    example_phrases=[
        "Elementary, my dear Watson.",
        "When you eliminate the impossible, whatever remains must be the truth.",
    ],
)

embedder = SentenceTransformerEmbedder()
tracker  = Tracker(
    persona    = persona,
    embedder   = embedder,
    threshold  = 0.25,
    alert_mode = "callback",
    callback   = lambda a: print(f"⚠️ Drift at turn {a.turn_index}!"),
)

tracker.add_turn("user",      "Holmes, what do you deduce?")
tracker.add_turn("assistant", "Elementary. The mud on your boots places you in Kensington.")

state = tracker.state()
print(f"Drift: {state.current_drift:.4f}")
print(f"Drifting: {state.is_drifting}")

Batch Analysis

from persiq import (
    PersonaDefinition,
    SentenceTransformerEmbedder,
    SlidingWindowScorer,
    DriftReport,
)

persona      = PersonaDefinition.from_json("sherlock.json")
conversation = [
    {"role": "user",      "content": "What do you observe?"},
    {"role": "assistant", "content": "Elementary. The footprints tell us everything."},
    {"role": "user",      "content": "And your hobbies?"},
    {"role": "assistant", "content": "lol I love Netflix and pizza honestly"},
]

embedder = SentenceTransformerEmbedder()
scorer   = SlidingWindowScorer(persona, embedder, window_size=5)
scores   = scorer.score_conversation(conversation)
report   = DriftReport.from_scores(scores, persona, threshold=0.25)

print(report)
report.plot()

CLI

persiq init --output my_persona.json

persiq check \
  --persona my_persona.json \
  --convo   my_chat.json    \
  --threshold 0.25          \
  --plot

---

Drift Score Reference

Score	Meaning
0.00 – 0.15	✅ Excellent — strongly on-persona
0.15 – 0.25	🟡 Moderate — minor deviations
0.25 – 0.35	🟠 Concerning — noticeable drift
0.35 – 1.00	🔴 Severe — significant character breakdown

---

Alert Modes

Mode	Behavior
"log"	Python logger.warning
"callback"	Calls your function with DriftAlert object
"raise"	Raises DriftAlertException
"warn"	Python warnings.warn
"silent"	Collects internally, you poll manually

---

Embedding Backends

Backend	Cost	Speed
SentenceTransformerEmbedder	Free	Fast
OpenAIEmbedder	Paid	Faster

---

Contributing

git clone https://github.com/himachhatbar17/persIQ
cd persIQ
pip install -e ".[dev]"
pytest tests/ -v

See CONTRIBUTING.md for full guide.

---

License

MIT © 2024 Hima Chhatbar