quotegate

中文 · English

Let a language model answer only with sentences copied from your passages, and check every one in code before it reaches a user.

The model is told to reply with lines like [2] a sentence, copied. quotegate.check() keeps a line only when the sentence is really in passage 2 and still reads right on its own. Everything else is dropped with a reason. No second model, no network, no dependencies: plain string work that gives the same answer every time.

It was pulled out of the 马莫百科 WeChat bot, which answers visa, enrolment and exam questions for Monash University students and parents, where a confident extra step is worse than no answer.

Why not let the model write and cite?

In the bot this came from, a free model asked to answer in its own words added steps no passage has ("the lecturer will apply for special consideration for you") and turned "ISP takes your passport when the sticker has 14 days or less left" into "there is a 14-day grace period after it expires". A second model checking the sentences missed some of these and cut correct ones. Quoting plus a string check had neither problem.

What gets dropped

A quote is widened to the whole sentence it sits in, then dropped when:

Line breaks end a sentence unless the passage says otherwise (soft_breaks=True for WeChat articles and PDF text, where lines break mid-sentence). A latin full stop ends one only before whitespace, so "50.0" and "monash.edu" don't.

Use

pip install quotegate

from quotegate import Passage, check
from quotegate.prompts import EN, user_message

passages = [
    Passage("Special consideration\nApply online within two working days of the due date. "
            "This must include supporting documents.", title="Special consideration"),
    Passage("Exams\nFinal exams run in weeks 13 and 14."),
]
messages = [
    {"role": "system", "content": EN},
    {"role": "user", "content": user_message("How long do I have to apply?", [p.text for p in passages])},
]
reply = call_your_model(messages)   # any chat model, e.g. a free tier
# reply == "[1] Apply online within two working days\n[1] This must include supporting documents"

result = check(reply, passages)
result.status    # "ok"
result.kept      # [Quote(passage=1, text="Apply online within two working days of the due date.", start=22, end=75)]
result.dropped   # [Dropped(passage=1, quote="This must include supporting documents.", reason="needs context")]

When result.status isn't "ok", send nothing from the model: "unsure" (it said it doesn't know), "uncited" (no quote lines), "unsupported" (no quote is in its passage) or "fragment" (found, but none reads right alone). The bot this came from forwards those questions to a person.

start and end locate the sentence in the passage, so you can link or highlight it. Send kept with the pages they came from and a line saying they are excerpts.

Rules

check(reply, passages, Rules(...)):

Passage(text, title=None, soft_breaks=False, qa=False): a text starting with its title line never has a sentence widened into the title; qa=True marks a passage that answers the question in its title, so "Yes, ..." or "可以，..." right after "答：" is fine.

Numbers

From the bot's evaluation (docs/EVAL.md, 50 questions, 2026-10-25, Groq's free qwen/qwen3.8-27b):

The check itself takes about 0.3 ms for a reply of four quotes against five 500-character passages (measured 2026-10-26 on one core of a cloud container).

Not for

• Answers that need reasoning or arithmetic across passages. A quote can only say what a passage says.
• Deciding whether a sentence answers the question. Bad retrieval gives true, irrelevant quotes.
• Translation: quote in the passages' language and translate the passages ahead of time.

Development

uv venv && uv pip install -e . pytest ruff
.venv/bin/pytest -q && .venv/bin/ruff check .

Releases: bump __version__ in src/quotegate/__init__.py, add a CHANGELOG.md section, merge, then push a v<version> tag. release.yml checks the tag against the version, runs the tests and publishes to PyPI with Trusted Publishing.

License

MIT © Waldo