splent_feature_search

Search across a whole product, over material this feature knows nothing about.

A SPLENT product is assembled from features, and the material worth searching belongs to them: pages to courses, articles to post, people to team. This feature owns the search page, the index and the ranking, and owns no content at all. A content feature makes itself searchable by registering three callables with the framework, and a search written today keeps working with a content feature written next year. Neither imports the other.

The seam

splent_framework.search.search_registry is the whole contract. A content feature calls this once, from init_feature:

from splent_framework.search.search_registry import register_search_source

def init_feature(app):
    register_search_source(
        key="courses",
        label="Pages",
        fetch=fetch_pages,
        resolve=resolve_page,
        find=find_pages,               # optional
        scopes=course_scopes,          # optional
        current_scope=course_of_url,   # optional
        order=100,
    )

fetch() yields every document, for building an index. Each one is {"id": str, "title": str, "body": str} and may carry "url", "scope" and any extra field the feature wants stored.

resolve(doc_id, user) is the authority. Given one candidate and the reader asking, it returns a presentable result, {"title": ..., "url": ..., "snippet": ...}, or None to drop it. It is called once per candidate on every search, so it should be a primary key lookup and not a scan.

find(term, user, scope=None) is optional and is how the feature searches itself when no engine is reachable. Same return shape as resolve. A source that omits it contributes nothing while the engine is down, which is a decision the feature gets to make. When a scope arrives it has to be honoured: without an index there is nothing else between the reader and the whole site.

order sorts the groups on the results page. label names the group and is translated at render time, so pass a plain or a lazy string, never one already translated at startup.

Narrowing

A wiki holding fourteen academic years has a page called "Lab 5" in every one of them, so "everywhere" is the least useful answer the box can give. Two more optional callables turn one box into "search this course", without this feature ever learning what a course is.

scopes(user) returns the places this reader may narrow to, as [{"key": "egc-20252026", "label": "EGC 2025/2026"}, ...] in display order, already filtered to what they may be told exists. The list is rendered in a selector, so a course that has not been announced is a course whose name the selector would announce.

current_scope() returns where the reader is standing right now, read out of the request, or None. It belongs to the feature that owns those URLs. Only courses knows that the second segment of /cursos/egc-20252026/paginas/lab-5 names a course, and a search feature taking the URL apart itself would be guessing at another feature's routing, would guess wrong the day the product serves it under a different word, and would have to guess again for every content feature ever written.

A document carries "scope" with the key of the place it belongs to. It is indexed as a keyword, matched whole and never analysed, and a narrowed search filters on it. A document with no scope belongs nowhere in particular and does not appear in a narrowed search.

A scope is applied as it arrives and is never checked against the list first. A name nobody offers matches no document and answers nothing, which is what a stale bookmark or an edited address deserves; checking it and falling back would turn that same request into a search over the whole site, which is the one answer nobody asked for. Nothing leaks either way, because every candidate is still resolved by the feature that owns it.

The box in the header defaults to current_scope() and lets the reader switch to everything or to another place. On the search page, the scope the page searched wins, so the two boxes never disagree. An empty scope= means everything and is not the same as no scope= at all, which means wherever the reader is.

Why the split

An index is a cache and is never the authority on visibility.

It is a copy taken at some moment in the past, and material is withheld between one reindex and the next. If a search answered out of the index, the first reader to type "exam" after an exam was pulled would be shown its title, and a title gives away as much as a body does. So Elasticsearch is only ever allowed to propose candidates: the query asks for ids and switches _source off, and every id goes back to the feature that owns it through resolve_search_hit(key, doc_id, user), at request time, with the reader in hand. Nothing stored in the index is ever rendered, not even the title.

Deny by default, exactly like the file access registry. A source nobody registered, a resolver that raises, a result with no URL: each one drops its candidate and leaves the rest of the search alone.

The other half of the rule is what never goes into an index. Documents carry no visibility, and the registry says so. Recording who may read what would be wrong the moment somebody withheld a page, and it would be answering a question about a reader who is not there yet and will be somebody else tomorrow.

What a product gets

• GET /<SEARCH_PATH> (search.index), a server-rendered page. It is a plain GET form and it works with JavaScript switched off. A term shorter than SEARCH_MIN_CHARS renders the empty state, not an error.
• GET /<SEARCH_PATH>/results.json (search.results), the same answer as JSON, for the script that makes the page feel immediate.
• A search box in layout.nav, unless SEARCH_NAV is off. It answers over AJAX into a panel under the box, with the arrow keys moving, Enter opening, Escape closing, and a plain form submit when there is no script. Nothing in that panel is written with innerHTML.
• SearchService in the service locator, with query(), scopes(), reindex() and index_report().
• splent feature:search reindex [--source KEY] and splent feature:search status.

The JSON answer:

{
  "query": "grafo",
  "engine": "elasticsearch",
  "scope": "egc-20252026",
  "scopes": [
    {"key": "", "label": "Everything"},
    {"key": "egc-20252026", "label": "EGC 2025/2026"}
  ],
  "total": 12,
  "groups": [
    {"key": "courses", "label": "Pages",
     "results": [{"title": "...", "url": "/cursos/...", "snippet": "..."}]}
  ]
}

engine says how the answer was produced: elasticsearch from the index, fallback from the sources' own find(), none when nothing produced it. The page says so too, in a sentence, rather than letting a thin answer look like a site with nothing in it.

scope is the narrowing that was applied, or null. scopes is what the selector offers, always including the everything option under the empty key, and it travels with every answer because the box that asked the question is redrawn from it.

The engine is optional

splent_feature_elasticsearch provides the transport. A product may install search without it, and a product that installed it may be running while the node is down. Both end up on the fallback, and the answer is labelled either way. The contract names elasticsearch under requires.features because services.py imports its exception types and the contract scanner reads dependencies out of import statements; the import itself is guarded and the transport is reached by name through the service locator.

One index per source, named <ELASTICSEARCH_INDEX_PREFIX>_<source_key>. The prefix belongs to the engine feature, which keeps two products sharing a node out of each other's documents, so everything here passes the bare source key and lets the engine put it in the right namespace.

reindex drops each index and builds it again, because writing over it leaves behind every document the feature no longer has. The window in which an index is incomplete costs results and never privacy, which is the only direction that is safe to be wrong in.

Configuration

Tests

splent feature:test splent_feature_search

The sources in the tests are fakes registered through the framework registry, so nothing here depends on a content feature being installed, and the engine is a fake index that proposes exactly what a test put in it, including documents the resolver then refuses.