varga

varga (वर्ग, “class”) answers two questions about a document.

The first question is what kind of document it is. The answer is one of 31 kinds, such as an invoice, a contract, a research paper or a bank statement. guess_type answers it by counting evidence in the text, such as cue phrases, amounts of money, dates and reference numbers. It uses no model and no network. It also says whether the evidence is clear enough to trust.

The second question is what the document says, in a fixed set of fields. For an invoice, the fields are the number, the date, the vendor and the total. The set of fields is a schema. as_schema returns a built-in schema, or a schema that you write as a short string. structured asks a language model that you supply to fill in the schema. check_fields then checks the dates and amounts that the model returned.

Install

uv add varga            # document types, with no model and no network
uv add varga rishi      # and a chat model for `structured` to ask

varga depends only on fastcore and rahasya. structured needs a chat object, such as a rishi.Chat. varga does not import rishi. Any object that has a .structured method and can be called to return a reply works.

What kind of document is it

INVOICE = '''# INVOICE
Invoice No: ACM-2024-0117
Date: 2024-03-01
Bill to: Contoso GmbH, Berlin
From: Acme Supplies Ltd
Total due: 1,240.00 EUR
Payment terms: net 30
'''
r = guess_type(INVOICE)
r.doctype, r.decisive, round(r.margin, 2)

('invoice', True, 0.4)

doctype is the best guess. margin is the score of the best guess minus the score of the runner-up. decisive is True when the best score is at least 0.4 and the margin is at least 0.12. When two kinds tie, margin is 0 and decisive is False. The evidence in the text has then not settled the question.

guess_type never calls a model. TYPE_SP in varga.core is a system prompt that asks a model to choose one label. You can use it when decisive is False.

The evidence comes from signals, which counts what the text contains:

signals(INVOICE).counts

{'money': 1, 'date': 1, 'ref': 1, 'heading': 1, 'org': 2}

What is in it

Each built-in schema is a Python dataclass, so its fields have names and types:

from dataclasses import fields
[f.name for f in fields(as_schema('invoice'))]

['number',
 'date',
 'due_date',
 'vendor',
 'vendor_tax_id',
 'bill_to',
 'ship_to',
 'currency',
 'subtotal',
 'tax',
 'total',
 'payment_terms',
 'items']

There are fifteen built-in schema names: invoice, purchase_order, quote, receipt, catalogue, contract, resume, paper, meeting_notes, id_card, passport, bank_statement, payslip, tax_return and other. Some names share a schema. purchase_order and quote use the invoice schema. You can also write a schema as 'name:type, name:type':

[(f.name, f.type.__name__) for f in fields(as_schema('vendor:str, total:float, items:list'))]

[('vendor', 'str'), ('total', 'float'), ('items', 'list')]

structured(chat, prompt, schema) asks the model to fill the schema and returns a dictionary. It first uses the constrained mode of the model, which forces the reply into the shape of the schema. If that call raises, structured warns and asks again for a plain JSON reply.

With cite=True, structured returns {field: {value, quote}}. The quote is the text that the model says it read the value from. locate(fields, pages) finds the page and block of each quote by string matching, with no model. You can then check each value against the source.

The document types

DOCTYPES holds all 31 kinds and the evidence for each:

len(DOCTYPES), sorted(DOCTYPES)

Eight kinds describe work documents: proposal, presentation, requirements_spec, technical_design, regulatory_guidance, procedure, qa_artifact and roadmap. Two more kinds, claim and clinical_record, have a similar shape. The tests check that guess_type decides each of these ten kinds correctly from its own text without a model. The tests also check that ordinary prose with the same vocabulary scores under 0.2 against all ten.

Six kinds are records about one person: id_card, passport, bank_statement, payslip, tax_return and clinical_record. SENSITIVE lists them. The result of guess_type has sensitive=True when its answer is one of them. varga takes no action on this flag. Your program can use it, for example, to keep such a document on the machine and not send it to a hosted model through structured.

Where text goes

guess_type, signals and check_fields run entirely in your own process and send nothing anywhere. structured sends its prompt to the chat object that you pass. The prompt normally contains the document. The text goes to the place where that model runs. varga does not store documents or results.

The name finder in signals uses the honorific pattern of rahasya. Both packages use one definition of a name.

Development

The notebooks in nbs/ are the source, and the Python modules are generated from them.

pip install -e .
nbdev-prepare