varga

varga (वर्ग, “class”) answers two questions about a document. The first is what kind of document it is: an invoice, a contract, a research paper, a bank statement, or one of 31 kinds in all. guess_type answers it by counting evidence in the text, such as cue phrases, amounts of money, dates and reference numbers. It uses no model and no network, and it says whether the evidence was clear enough to trust.

The second question is what the document says, in a fixed set of fields: an invoice’s number, date, vendor and total, for instance. as_schema gives you the set of fields, called a schema, either one of the built-in ones or one you write as a short string. structured asks a language model you supply to fill it in, and check_fields then checks the dates and amounts the model returned.

Install

uv add varga            # document types, with no model and no network
uv add varga rishi      # and a chat model for `structured` to ask

varga depends only on fastcore and rahasya. structured needs a chat object, such as a rishi.Chat, but varga imports nothing to use it: any object with a .structured method and a call that returns its reply will do.

What kind of document is it

INVOICE = '''# INVOICE
Invoice No: ACM-2024-0117
Date: 2024-03-01
Bill to: Contoso GmbH, Berlin
From: Acme Supplies Ltd
Total due: 1,240.00 EUR
Payment terms: net 30
'''
r = guess_type(INVOICE)
r.doctype, r.decisive, round(r.margin, 2)

('invoice', True, 0.4)

doctype is the best guess and margin is how far its score is ahead of the runner-up. decisive says whether to trust the guess: it is True when the best score is at least 0.4 and at least 0.12 ahead of the next. When two kinds tie, margin is 0 and decisive is False, which tells you the text alone has not settled the question. guess_type never calls a model itself. If you want a second opinion, TYPE_SP in varga.core is a ready-made system prompt for asking a model to choose one label.

The evidence comes from signals, which counts what the text contains:

signals(INVOICE).counts

{'money': 1, 'date': 1, 'ref': 1, 'heading': 1, 'org': 2}

What is in it

Each built-in schema is a Python dataclass, so its fields have names and types:

from dataclasses import fields
[f.name for f in fields(as_schema('invoice'))]

['number',
 'date',
 'due_date',
 'vendor',
 'vendor_tax_id',
 'bill_to',
 'ship_to',
 'currency',
 'subtotal',
 'tax',
 'total',
 'payment_terms',
 'items']

There are fifteen built-in schema names: invoice, purchase_order, quote, receipt, catalogue, contract, resume, paper, meeting_notes, id_card, passport, bank_statement, payslip, tax_return and other. Some share a schema: purchase orders and quotes use the invoice schema. You can also write a schema on the spot as 'name:type, name:type':

[(f.name, f.type.__name__) for f in fields(as_schema('vendor:str, total:float, items:list'))]

[('vendor', 'str'), ('total', 'float'), ('items', 'list')]

structured(chat, prompt, schema) asks the model to fill the schema and returns a dictionary. It first uses the model’s constrained mode, which forces the reply into the schema’s shape. If that call raises, for a model without tool calling for instance, it warns and asks again for a plain JSON reply.

With cite=True it returns {field: {value, quote}}, where the quote is the text the model says it read the value from. locate(fields, pages) then finds the page and block holding each quote by plain string matching, with no model, so that each value can be checked against the source.

The document types

DOCTYPES holds all 31 kinds and the evidence for each:

len(DOCTYPES), sorted(DOCTYPES)

Ten of them describe work documents: proposal, presentation, requirements_spec, technical_design, regulatory_guidance, procedure, qa_artifact, roadmap, claim and clinical_record. The tests check that each is decided correctly from its own text without a model, and that ordinary prose which happens to use their vocabulary scores under 0.2 against all of them.

Six kinds are records about one person: id_card, passport, bank_statement, payslip, tax_return and clinical_record. They are listed in SENSITIVE, and the result of guess_type has sensitive=True when its answer is one of them. varga does not act on this itself. It is there so that your program can, for example, keep such a document on the machine instead of sending it to a hosted model through structured.

Where text goes

guess_type, signals and check_fields run entirely in your own process and send nothing anywhere. structured sends its prompt, which normally contains the document, to the chat object you pass it, so the text goes wherever that model runs. varga does not store documents or results.

The name finder in signals uses rahasya’s honorific pattern, so the two packages agree on what counts as a name.

Development

The notebooks in nbs/ are the source, and the Python modules are generated from them.

pip install -e .
nbdev-prepare