stapel-vocabularies

Reference vocabularies too large to inline into a category's feature schema — 14 962 phone models, 107 049 car modifications — stored as levels, terms and parent/child edges rather than as paths, so one 'black' is shared by every model that comes in black and can be translated once and faceted on. Ships the public typeahead and cascade reads a listing composer needs (ETag'd on a revision, anonymous, no cookie), the two VocabularyResolver implementations stapel-attributes' ref_select / ref_hierarchical_select types validate values through (in-process and over comm), a batched transactional loader for reviewed fixtures that spends one revision and one event per file, and Django-free converters that turn a vendor's nested XML or CSV catalogue into such a fixture without loading the document into memory.

Part of the Stapel framework — composable Django apps that deploy as a monolith or as microservices without changing module code.

Install

pip install stapel-vocabularies

At a glance

Documentation

OpenAPI · capabilities.json · llms.txt (for agents)

What this is

Some option lists are too big to be options. A phone catalogue has 14 962 models; a car catalogue has 107 049 modifications. Inlining those into a category's feature schema is not a tuning question — the schema is fetched on every form render, and it would be megabytes.

stapel-vocabularies is where they live instead.

• Levels, terms and edges — not paths. Vendor → Model → MemorySize → Color is 56 921 distinct paths in a real phone catalogue and only 15 844 distinct terms. Storing the terms and the parent/child edges between them means one Color=chernyy shared by every model that comes in black: 17 colours to translate instead of 56 921 path nodes, and a facet on a colour code that is answerable at all.
• A read surface built for a typeahead. terms/?level=Model&parent=apple&q=pro answers a page with total and has_children, prefix matches ranked first. Anonymous, ETag'd on the vocabulary's revision, Cache-Control: public, and no Set-Cookie — so the shared cache in front of it works and a crawler does not start a session per request.
• A popular band, so a dictionary does not open on the alphabet. A 529-vendor level sorted by name opens on 3Q, 4Good, 8848, A1, Aceline while the brands carrying the volume sit hundreds of rows down. Push the listing counts you already have — vocabularies.set_popularity — and the top twelve lead every page, with band on each row and popular_count on the page so a control draws its separator without guessing. Idempotent: unchanged counts write nothing and invalidate nothing.
• A match that can say no. vocabularies.match turns one free-text guess into one code with a score — 1.0 for an exact or transliterated hit, 0.9 for a unique prefix, and for a similarity hit the number the vector layer actually reported. Below MATCH_MIN_SCORE it answers {"matched": false}. That is the difference between a typeahead, where a person picks one of five rows, and a composer, which writes the answer into a listing unread.
• A level a category can BE. A node whose children are the 529 brands of its brand field is 529 rows that carry nothing the field does not already have. Point stapel-categories' children_expand_by at that feature and the tree answers with one virtual child per term instead — no rows, no slugs, each one carrying the filter the node already answers. That read is terms() on the resolver (and vocabularies.terms over the bus): the level whole, in its own order, capped at TERMS_LIMIT.
• Two resolvers, one protocol. ref_select / ref_hierarchical_select in stapel-attributes validate values through a VocabularyResolver. OrmResolver answers from these tables and is registered at startup; CommResolver answers the same questions over the bus, for a service that validates listings but holds no catalogues. Both cache describe by revision, so a re-imported catalogue stops validating against the levels it used to have the moment the import commits.
• Loading is data plumbing, not an admin screen. manage.py load_vocabulary phones.json is one transaction, one revision increment and one vocabulary.changed event for the whole file, whatever its size. The real phone catalogue — 15 844 terms, 39 749 edges — loads in ~1.2 s.
• Converters that do not read the file into memory. A vendor's nested XML or a one-path-per-row CSV becomes a reviewable fixture, streamed through iterparse, with codes assigned deterministically (Cyrillic transliterated, collisions numbered in label sort order) so re-converting an unchanged catalogue produces an unchanged diff.

Alpha. See MODULE.md for the agent-facing map of seams.

Quick start

pip install stapel-vocabularies

# settings.py
INSTALLED_APPS = [..., "stapel_vocabularies"]

# urls.py
path("vocabularies/", include("stapel_vocabularies.urls"))   # -> /vocabularies/api/v1/...

python manage.py convert_vocabulary phone_catalog.xml \
    --slug phone-models --name "Phone models" --out fixtures/phone-models.json
python manage.py load_vocabulary fixtures/phone-models.json --replace

A feature then points at it instead of carrying options:

{"type": "ref_select", "optionsRef": {"vocabulary": "phone-models",
                                      "level": "Model",
                                      "parentFeature": "vendor"}}

API

Accept-Language selects a translated label where the term carries one; the response Varys on it and the ETag covers it.

Rows lead with the popular band, then the level's own rank, then the alphabet — except under ?q=, where a prefix match still outranks everything, because that is what a typeahead is. popular_count is how many LEADING rows are in the band, so the separator goes after index popular_count - 1; 0 means this page has no band.

Asking over the bus

from stapel_core.comm import call

call("vocabularies.match", {"vocabulary": "phone-models", "level": "Vendor",
                            "text": "Самсунг"})
# -> {"matched": True, "code": "samsung", "label": "Samsung",
#     "score": 1.0, "method": "exact"}

call("vocabularies.match", {"vocabulary": "phone-models", "level": "Vendor",
                            "text": "айфон"})
# -> {"matched": False, "reason": "no_confident_match"}

call("vocabularies.set_popularity", {"vocabulary": "phone-models", "level": "Vendor",
                                     "counts": {"samsung": 41233, "apple": 38902}})
# -> {"ranked": 2, "revision": 8}

The fixture format

One file per vocabulary, byte-stable, reviewed as code (schema):

{ "slug": "phone-models", "name": "Phone models", "source": "https://…/phone_catalog.xml",
  "levels": [{"name": "Vendor"}, {"name": "Model", "parent": "Vendor"}],
  "terms": [["Vendor", "apple", "Apple", null, 0, 90], ["Model", "iphone-10", "iPhone 10", null]],
  "edges": [["Vendor", "apple", "Model", "iphone-10"]] }

A level's parent must be declared before it. That single rule is the whole acyclicity argument: a level can only point backwards, so no chain of parents can return to where it started.

A term row is [level, code, label, external_id, sort?, popularity?, extra?]. sort ranks within a band; popularity says which band. All three are optional, and a row that omits popularity leaves whatever the live term holds — so a catalogue re-import never erases a band pushed from observed counts. The columns are positional, so null states that same nothing for a writer that has to cross a column to reach a later one.

extra is a free-form object of attributes the source catalogue owns: ["Color", "chernyy", "чёрный", null, 0, 0, {"hue": "#1a1a1a"}], so a search facet over a colour level can draw a swatch. The only thing that knows «чёрный» is #1a1a1a is the catalogue the term came from, and until 0.4.0 there was nowhere to put it. It is never part of identity — that stays (vocabulary, level, code) — and never a second place to look for a label. Keys are merged into the live term one by one, so a second catalogue contributing to the same vocabulary adds what it knows without erasing what the first one knew. It reaches a reader as extra on a term of …/terms/?level=, as extras on vocabularies.terms, and as the third element of resolver.terms_with_extra().

License

MIT — see LICENSE.

_{This page is assembled by stapel-readme from docs/readme.md plus the contract artifacts in docs/. Edit the prose in docs/readme.md; the badges, facts and links above and below it are generated — do not hand-edit README.md.}