Skip to main content

whowhen-rag

Find the "who" and "when" in a search query, and only pay for an LLM when there is one.

whowhen-rag turns questions like "what did Priya change in the pricing docs last week?" into metadata filters for your RAG system:

f = detector.detect("what did Priya change in the pricing docs last week?")
f.conditions     # [Condition('modified_on', '>=', '2026-11-09'),
                 #  Condition('modified_on', '<', '2026-11-16'),
                 #  Condition('last_editor', '=', 'priya@example.com')]
f.cleaned_query  # 'pricing docs'  (use this for the semantic search)
f.to_sql()       # ('"modified_on" >= %s AND "modified_on" < %s AND "last_editor" = %s', [...])

It works with any vector store or database, any LLM, and any column names.

Why it is cheap

Most questions sent to a knowledge base have no time or author condition at all. Framework tools for this (LangChain's SelfQueryRetriever, LlamaIndex auto-retrieval) send every query to an LLM to find out. whowhen-rag puts two cheap gates in front of the LLM:

query
  │
  ├─ 1. Regex gate ......... free.  "last week", "written by", "latest" ... ───────┐
  │      miss                                                                       │ hit
  ├─ 2. Jev classifier ..... a yes/no answer, no text generation. ─────────────────┤ yes
  │      no: return "no filter", no LLM call                                        │
  │                                                                                 ▼
  └─ 3. Your LLM ........... extracts dates, names, sort order. Only for likely filters.
  • Gate 1 catches the common wording for free.
  • Gate 2 (Jev) catches unusual wording ("anything Priya put together on onboarding?"). It answers yes or no with a probability. It cannot pull out the dates or names, which is why step 3 exists.
  • Step 3 runs only when a gate says yes, so plain questions ("how does caching work?") never reach a paid model.

Every result carries a trace (gate, classifier_called, llm_called), so you can measure the savings on your own traffic.

Measured

On the synthetic benchmark sets (details and history in benchmarks/RESULTS.md):

  • Plain questions: 20 of 20 stopped by the gates. No LLM call.
  • Unusual wording the regex misses: 8 of 8 caught by the Jev classifier.
  • Extraction (Claude Haiku 4.5, prompt v2): 13 of 15 exactly right (dates, people, sort) on a fresh set written after the prompt was final, and 45 of 45 on the sets used while writing it. Misses on the fresh set: "since September" read as last year, and one arguable person filter.
  • Cost of an extraction: about 750 input and 75 output tokens, about $0.0011 at Haiku 4.5 list price (about $1.10 per 1,000 queries that reach the LLM), median 1.6 s.

These are small, hand-written sets. Measure on your own queries before relying on the numbers.

Install

pip install whowhen-rag

Quick start

from whowhen_rag import Detector, Field

detector = Detector(
    fields=[
        Field("created_on",  role="created_at"),
        Field("modified_on", role="updated_at"),
        Field("owner_email", role="created_by", resolve=lookup_email),  # name -> your stored value
    ],
    llm=my_llm,  # any function: prompt string in, answer string out
)

f = detector.detect("pages Priya wrote this month about onboarding")
if f:                                    # same as f.needs_filter
    where, params = f.to_sql()
    search_text = f.cleaned_query        # "onboarding"

Set TYPESAFE_API_KEY for the Jev classifier. A runnable example with sqlite is in examples/quickstart.py.

Your columns, your names

You declare your columns once. The LLM never sees them: it answers in terms of four roles, and whowhen-rag maps each role to your column.

Role Meaning
created_at when the document was created
updated_at when the document was last edited
created_by who created or wrote it
updated_by who last edited it

Declare only the roles you have. If you have one date column, "changed last week" and "created last week" both use it.

Field(
    "author_id",                               # your column or metadata key
    role="created_by",
    resolve=lambda name: users.find_id(name),  # str, int, list of IDs, or None if unknown
)
Field(
    "updated_ts",
    role="updated_at",
    value_format="epoch",                      # "date" (default), "datetime", "epoch", "epoch_ms"
    description="when the ticket was last touched",  # optional wording for the LLM
)
  • resolve turns a name from the query into what your column stores. Without it, the name is used as written. If it returns None, that condition is dropped and the name is listed in f.trace.unresolved_people.
  • value_format matches how your dates are stored. Dates are midnight UTC.

Bring your own LLM

llm is any function that takes the prompt and returns the model's text. whowhen-rag writes the prompt and validates the answer. Tested with Claude Haiku 4.5; any model that follows JSON instructions should work.

Anthropic

import anthropic

client = anthropic.Anthropic()

def llm(prompt: str) -> str:
    msg = client.messages.create(model="claude-haiku-4-5", max_tokens=512,
                                 messages=[{"role": "user", "content": prompt}])
    return msg.content[0].text

OpenAI or OpenRouter

from openai import OpenAI

client = OpenAI()  # OpenRouter: OpenAI(base_url="https://openrouter.ai/api/v1", api_key=...)

def llm(prompt: str) -> str:
    r = client.chat.completions.create(model="gpt-4o-mini", temperature=0,
                                       messages=[{"role": "user", "content": prompt}])
    return r.choices[0].message.content or ""

Prompts

The extraction prompt is a plain text file, versioned and measured. v2 is the default; v1 is kept so you can see what changed and why.

Detector(fields, llm, prompt="v1")             # an older built-in version
Detector(fields, llm, prompt="my_prompt.txt")  # your own template

v2 fixed five failure patterns found by the benchmark ("in October" read as last year, a year in a topic name read as a filter, and others), going from 37 to 45 of 45 correct on the sets used while writing it, and 13 of 15 on a fresh set. See docs/prompts.md for the before and after, the template placeholders, and how to score your own prompt.

The classifier (Jev)

On by default. It reads TYPESAFE_API_KEY, or you pass the key:

from whowhen_rag import Detector, JevClassifier

Detector(fields, llm, classifier=JevClassifier(api_key="...", threshold=0.5))
Detector(fields, llm, use_classifier=False)      # regex gate only
Detector(fields, llm, classifier=my_classifier)  # any function: query -> bool

Without a key, whowhen-rag logs one warning and carries on with the regex gate only; queries the regex misses then get no filter.

Using the result

f.needs_filter, bool(f) any condition, sort or list request found
f.conditions Condition(column, op, value); op is >=, <, = or in; combine with AND
f.sort Sort(column, descending) for "latest" or "oldest", else None
f.cleaned_query the query without its time and person words, for semantic search
f.list_only the query only asks to list documents ("pages Priya created this week"); you can skip semantic search
f.to_sql(placeholder="%s") (where_clause, params): columns quoted, values parameterised; "?" for sqlite
f.order_by_sql() ORDER BY "col" DESC, or ""
f.to_chroma() a Chroma where dict (Chroma ranges need numbers: use value_format="epoch")
f.to_qdrant() a Qdrant filter dict: models.Filter(**f.to_qdrant())
f.to_dict() plain dict, JSON-serialisable
f.trace which gate fired, whether the classifier and LLM ran, errors

Only need the yes or no? detector.has_signal(query) runs the cheap gates and never calls the LLM.

Failure behaviour

Search should never break because of filtering. If the classifier or LLM fails, times out or returns something unusable, detect() returns "no filter" with the original query and logs a warning (logger whowhen_rag). The LLM's answer is validated: unknown roles, invalid dates and empty ranges are dropped, never guessed.

Limitations

  • English queries only (the regex gate and the prompt are English).
  • Synchronous. Use asyncio.to_thread from async code.
  • Four roles (created and updated, time and person). "My documents" needs the current user, which whowhen-rag does not know.

Development

uv sync
uv run pytest                        # no network or API keys needed
uv run python benchmarks/gates.py    # set TYPESAFE_API_KEY to include the classifier

License

MIT

Metadata

Release files for whowhen-rag 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for whowhen-rag 0.1.0
File Size Uploaded Provenance Actions
whowhen_rag-0.1.0.tar.gz 62.0 kB With provenance Details for whowhen_rag-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for whowhen-rag 0.1.0
File Interpreter ABI Platform Actions
whowhen_rag-0.1.0-py3-none-any.whl Python 3 none any Details for whowhen_rag-0.1.0-py3-none-any.whl

Total release size: 81.8 kB

Release files / whowhen_rag-0.1.0.tar.gz

Download URL whowhen_rag-0.1.0.tar.gz
Size 62.0 kB
Tags Source
SHA-256 checksum
How to use checksums
adb39a58c6678e5c09d3b8eefe56b180291662a75e712291ee79d32c18ca64a9
BLAKE2b-256 checksum
How to use checksums
46181ded0c18b47372ab5996ea1e0732dc214417b3b11392fc5e00810415c659
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 22, 2026.

Transparency log

Release files / whowhen_rag-0.1.0-py3-none-any.whl

Download URL whowhen_rag-0.1.0-py3-none-any.whl
Size 19.8 kB
Tags Python 3
SHA-256 checksum
How to use checksums
c59c4b6e4b95966c1979929c22177f3504df9675414eb7226a555777a9893f5a
BLAKE2b-256 checksum
How to use checksums
513691d2b2a6372855a961ef047f5d9765f2dfc891d82ec0739ac7b0eb4399c4
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 22, 2026.

Transparency log

Security for whowhen-rag 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for whowhen_rag-0.1.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 22, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page