Ingence

Ingence is a read-only indexing and inference engine for scientific filesystems. It reads files’ names, metadata and contents where they already sit, and interprets them through two editable, versioned sources: a Dictionary, which defines what the words in names and files mean, and a Rulebook, the engine’s interpretive logic, which turns that evidence into proposals about how files relate. Every conclusion names the rule and evidence behind it, and where the rules cannot decide, the engine abstains.

The Dashboard, served locally in your browser, is where you explore the results: search for a file or experiment, see what was read from it and what was not, follow how files connect, and answer the questions the engine could not settle. A survey of what a filesystem holds is useful on its own, before any inference is confirmed.

Maturity: pre-alpha

Ingence is a pre-alpha. It runs from start to finish and is changed often; what it produces are proposals and readings, not scientific conclusions.

Send feedback to hello@ingence.org. Say what you ran and what you expected. Before you send anything, see Reporting a problem without sharing your data.

The workflow

ingence see, with no project, opens a page in your browser. The page makes a project, asks the few things the engine needs before a first run, and runs the steps below when you click, showing what it has seen so far (GETTING_STARTED.md). The same steps by hand:

ingence init PROJECT --root /path/to/data     # declare what to read; writes only PROJECT
ingence index -p PROJECT                      # the census: every file's metadata, no file opened
ingence dive  -p PROJECT --budget 900         # open files within a time budget
ingence infer -p PROJECT                      # evaluate the rules in force and publish
ingence see   -p PROJECT                      # the Dashboard, on this machine only

• init creates a project directory wherever you choose (see Where a project lives). A root is a folder, or a public archive's listing (listing:<archive>:<identifier>, fetched once with index --fetch). For someone else's folder, init --depth metadata declares once that no dive there opens a file.
• index walks each root and records what the filesystem states (names, sizes, times, links) without opening a file, then reads every name into its parts against the Dictionary. --rebind reads every stored name again under the current vocabulary and decisions without touching the root; --dry-run reports what a census would find and writes nothing.
• dive opens files within a time budget, one file type at a time. It reads headers, instrument clocks, text, tables and archives' member lists, in supervised subprocesses.
• infer evaluates the Rulebook over the current run. Each conclusion is a claim that carries its rule, the rule's version and its evidence. A rule that cannot decide abstains and says why. A rule that proposes a pair or a group asks a person and does not assert it.
• see serves the Dashboard on the loopback interface (an address that only your own computer can reach). Its pages are Explore (search and inspect), Graph, Questions, Engine and History. An answer is written to the Record, the append-only log of human decisions, and waits for Re-run with every other change. Re-run reads the affected names again and runs the rules again, and until then the pages keep showing the last published run.

rule, teach, record, log, export, map and fix read and revise the Rulebook and the Dictionary, import review sheets, show the Change Log, and diagnose a project.

index, dive and infer each end by saying what was read, what waits for you and the next command; --verbose adds the detail (each rule's counts, reuse, timings).

Domain packs and proposals

The core Dictionary reads what every laboratory holds: file formats, dates, units, stages. A domain pack adds the words and file types of one science (microscopy, biochemistry, sequencing, spectrometry, electrophysiology, clinical, computational). A pack adds vocabulary only, with no rules of its own. A new project loads the packs that init --domain names (repeat it for several), or every pack with --all-domains. With neither, it loads the core alone. programs, which is about software and not a science, always loads. Every pack ships with the engine. Where the core and a pack spell a word alike, a person's answer wins. Otherwise the core's meaning is used over a shipped pack's, unless the project has accepted that pack. The Dictionary page lists those words.

A pack of your own is one YAML file. ingence teach export-answers --out ours.yaml writes what your project was taught, and ingence teach pack install <file, folder or https://address> installs a pack. The pack's pack: block gives its name, version, author, licence, the schema it needs and a digest of its words. Ingence refuses a pack whose words do not match its digest, or that needs a newer Ingence. Your own pack stands level with the core, so a word both hold waits for your answer. Once the project accepts a shipped pack, the packs it chose, yours among them, stand above the core, and the other shipped packs stand level with it. A pack can be kept loaded without standing over the core. Every word of an installed pack is active.

The engine proposes; a person decides. After infer, the Dashboard's Questions page asks about:

• A pack to load, where a pack the project does not load would read enough of what nothing reads, or where three or more files are in a format its instruments write (ND2 for microscopy). The question shows its counts. Accepting loads the pack from the next run. Refusing stops the question.
• A pack to keep, in a project that loads every pack, where no name holds one of its words. Refusing stops loading the pack from the next run.
• Groups of experiments that share words, most general first (on Groupings).
• Pairs of files where one may have been made from the other (on Proposed pairs).
• A word nothing reads, where words of one kind stand, and what it means.

Nothing is loaded, grouped or derived until someone says yes.

Links on a share

A link is recorded and never followed into. By default (scan.resolve_symlinks: true in the project's settings.yaml) the census asks whether each link's target is there, so it can tell a link whose target is gone from one whose target is elsewhere. It follows a chain one link at a time, and only inside the root. The first link whose text points outside the root is read from that text and never followed, whatever the setting. On a slow share each of those questions waits on the share. Turn them off with

scan:
  resolve_symlinks: false

and each link is then recorded as a link, with its target unasked.

After see: the Dashboard

ingence see -p PROJECT opens the Dashboard in your browser. It has five pages:

• Explore: what is here, what could not be read, and a search box for an experiment or a file. Open any file to see what was read from it and every conclusion about it.
• Graph: how files relate (what contains what, what was made from what, what mentions what), from one folder or file you choose.
• Questions: everything waiting for you, one section per kind, each with how many are waiting: Words (teach the engine your own words: what a word means, and which spellings are the same thing), Questions (what no rule could decide, the most useful first), Groupings, Claim samples, Proposed pairs, Extensions, Programs, and Coverage (what has not been read yet, and why).
• Engine: the rules in force, each in plain words with how much of the run rests on it, the Dictionary, and the project's settings.
• History: what changed between two runs, and why.

Every answer is written to the Record under your name (the page asks who you are once) and never overwrites an earlier one. Every change waits for Re-run: an answer, a word's meaning, a pack, a committed setting. The count on the Re-run button says how many, and opens a summary by kind. Nothing runs until you press it, or run ingence index --rebind -p PROJECT and then ingence infer -p PROJECT. Opening a project where changes wait says so once, with Re-run now and Later. While a re-run works, the pages keep showing the last published run.

Export and Import carry the questions to a colleague. Export saves them as one file, and Import records each answer as its author's.

Principles

• Source data is read-only. Ingence never moves, renames, deletes or writes to indexed files, and never executes code it finds.
• Observations and interpretations never share a container. A fact and an inference cannot be written to the same field.
• Every interpretation is reproducible from four inputs: the Index, the Rulebook, the Dictionary and the Record.
• Abstention is a valid output. "Unknown", "unexamined" and "conflicting" are three different states and stay distinct.
• No model authors a claim. No language model interprets your files, and none is needed to read Ingence's output.
• More evidence never lowers a claim's standing.
• It runs on your machine.

Offline, and when the source is away

The engine runs entirely on the machine it is installed on. It makes network connections only where you ask it to: installing the package, fetching a public archive's listing with index --fetch (one request at a time, every request logged), and installing a pack from an address with teach pack install https://.... Nothing is sent anywhere else. The Dashboard binds 127.0.0.1 only, loads no remote script, font or image, and refuses writes that come from other pages in your browser.

The source can be away. A walk (index) and a dive need the data they read, and refuse a root they cannot reach before writing anything. Everything else reads the project directory alone: index --rebind, infer, the Dashboard, answering questions and applying the answers all work with a drive unplugged or a network share unmounted. The last published run stays readable at all times: a run is published atomically or not at all, and an interrupted command leaves the previous run current.

Schemas and upgrades

Every record the engine writes carries a schema_version. schemas/v1/ holds the record contract in machine-readable form: one JSON Schema per record type, the eight relation primitives with their formal properties, the specialization catalog, the publication invariants and the storage table declarations. schemas/api/v1/ holds the Dashboard API's response models. Both are generated from the code.

• A newer engine reads an older project. A minor schema version adds fields compatibly; a project written by an older engine is rebound and inferred by a newer one, and the test suite holds projects written by earlier engines to prove it.
• An older engine refuses a newer project rather than misreading it. Once a newer engine has written into a project (a new kind of decision, a new run), an older engine stops with a message naming the version. Upgrade a copy first if two engines must share a project. A project with a field the engine does not know (a name, from schema 1.23) is refused the same way, saying which build made it.
• A major schema version would change a field's meaning and require a migration; there has been none.

Versions

Two versions move independently: the package version (ingence --version) and the schema version of the record contract it writes.

The package version is SemVer-like in its pre-1.0 form, 0.MINOR.PATCH, written as Python's own versions are (PEP 440), with a .devN suffix while pre-alpha:

• MINOR moves when a change makes a project unreadable by the previous package, that is, whenever the record schema's minor version or the Dashboard API's major version moves;
• PATCH moves for every other change merged to the main branch;
• .devN marks a development build between releases. A version without it is a release, and only a tagged commit is one.

The current version is 0.3.2, for schema 1.23 and the versioned Dashboard API. Every run also records the engine's commit, which identifies a development build exactly.

Supported systems

• Python 3.12, 3.13 and 3.14.
• macOS, Linux and Windows, each tested on every change. Filesystem behavior (case sensitivity, links, birth times, rename semantics) is measured for each root and not assumed from the platform.
• A laptop is enough: the target machine has 12 GB of memory and four to eight cores. No GPU is used or required.

Install

Ingence is on the Python Package Index. There are three ways to install it, in INSTALL.md: uv (the quickest), pip in a virtual environment, and conda. With uv:

uv tool install ingence
ingence --version

To upgrade, run uv tool upgrade ingence. Then read GETTING_STARTED.md. INSTALL.md also covers a wheel file, a checkout and the optional parts.

Where a project lives

Anywhere: your own disk, a drive, a server, inside the folder it reads, or that folder itself. A project inside the folder it reads adds its own files there. Every run leaves them out, with Ingence's own installation and any other Ingence project it finds. Your own disk is best: a project is written to on every run, and a local disk is faster and is there when a drive or server is not.

Licensing and credit

Source code and bundled software documentation are MIT (LICENSE). Synthetic datasets and de-identified benchmark data are CC0-1.0, and the technical report is CC-BY-4.0. Both texts are in LICENSES/. Third-party dependencies keep their own licenses and notices, which are listed in NOTICE. Vendored web assets carry their own license beside them.

Copyright (c) 2026 Andrew Bazley. Supported by the Astera Institute through the Hidden Science Competition.