kbforge
Agent-first knowledge bases, forged from your systems of record.
kbforge reads your systems of record (a folder of markdown, a git history, a database view, any MCP server) and proposes an Open Knowledge Format (OKF) v0.2 bundle as a pull request: one markdown concept per source document, carrying the frontmatter, links and provenance an agent needs to cite it. You review and merge; kbforge never does. A re-run when nothing changed costs one fetch: no model call, no pull request.
Quickstart
pip install kbforge; then, beside a docs/ of markdown (here two files):
$ kbforge init
wrote kbforge.yaml
next: kbforge plan
keep .kbforge/ between runs: it holds the mirror that makes re-runs free
$ kbforge plan
docs 2 added, 0 modified, 0 removed, 0 drifted, 0 rebuilt → would propose 2 concepts (validation runs at publish)
docs + local_files:onboarding.md added
docs + local_files:refunds.md added
$ kbforge run
docs Published: …/.kbforge/out/sync-local_files
$ kbforge run
docs NoOp: no change detected; no MR opened.
$ kbforge status
docs last published 2026-10-29 16:32 UTC · no open request
The starter file needs no credentials: stub copies each body verbatim, and
dry-run writes the proposal (concepts plus MR_BODY.md, the review note)
under .kbforge/out/. plan is run stopped before synthesis: it fetches and
diffs, writes nothing, and needs only the sources' credentials.
| Command | Does |
|---|---|
kbforge init [PATH] |
writes a starter kbforge.yaml for the documents in PATH (default ./docs); never overwrites one |
kbforge plan [SOURCE...] |
what run would propose, and why; writes nothing, calls no model |
kbforge run [SOURCE...] |
runs the sources (default: all) in file order |
kbforge status |
per source: last publish, open review request, pending chunks |
kbforge redo SOURCE |
rolls back the last chunk so the next run proposes it again (Large first runs) |
kbforge reset NAME [--yes] |
forgets one source's local state; its next run proposes everything, as a first run would |
kbforge list |
the installed connectors, synthesizers and publishers |
Each reads ./kbforge.yaml or --project PATH. run exits 1 if any source
failed (the others still run), 2 if the project is invalid (every problem is
printed and nothing runs), 3 if another run holds the lock, and 0 otherwise.
The project file
# kbforge.yaml: one knowledge base
synthesizer: # default: stub (no model)
name: llm
model: deepseek/deepseek-v4-flash
instructions: |
Write for application managers. Prefer the product's own terms.
publisher: # default: dry-run (a local directory)
name: github
repo: acme/ev-traction-kb
token_env: GITHUB_TOKEN
links: links.yaml # a path, or the mapping inline
grounding: grounding.yaml
sources: # run in this order
products:
connector: sql # pip install kbforge-sql
config:
system: products
url_env: DENODO_URL
query: SELECT product_id, product_name, description FROM iv_product
id: [product_id]
title: product_name
text: description
handbook:
connector: local_files
config: { path: ./handbook }
chunking: { max_concepts: 40 }
synthesizer: { name: describe, model: deepseek/deepseek-v4-flash }
- Paths are relative to the file. State lives under
state_dir(default.kbforge/) and must persist between runs: in CI, a cache or a committed directory. Secrets stay in env vars that*_envkeys name, and an unknown key is an error naming its YAML path. - Per-source
synthesizerandpublisherreplace the top-level one whole.linksandgroundingexist only at the top, so every source sees the same. Inline, each takes the whole file's contents, top-level key included:links: { links: { "products:p1": [handbook:p1.md] } }, andgrounding: { grounding: {...}, rules: [...] }. - Put a source others ground on first, so their rebuilds land in the same run rather than the next.
- A system belongs to one source.
local_filesandgit_commitseach emit one fixed system, so point onelocal_filessource at a common parent; a second aborts with[source-ownership] ... system 'local_files' is already owned by source 'docs'. - One run at a time per mirror.
run,plan,redoandresetholdrun.lock; a second exits 3:Locked: another kbforge run holds …/.kbforge/run.lock; nothing was read or written. - Editing a source's config starts it on a fresh cursor, and
runprintsnote: config changed since the last run; starting from a fresh cursor (the old one stays in state/, unused).
Writing with a model needs pip install "kbforge[llm]" and
OPENROUTER_API_KEY (or api_base: for a LiteLLM gateway). llm writes the
body; editing its instructions alone re-synthesizes nothing. describe keeps
the body byte-for-byte and has the model write only description and tags
(optionally from a tags_vocabulary), cached per document.
Publishing to GitHub or GitLab
Set publisher to github or gitlab. Both take repo (required), base
(default: the repo's default branch), base_path (default: repo root),
branch (default: sync/<system>), title, api_base (GitHub Enterprise or
self-managed GitLab) and token_env (default GITHUB_TOKEN / GITLAB_TOKEN).
kbforge keeps one long-lived sync branch and one open review request per source system: a later run force-updates the branch and edits the open PR/MR rather than opening a second. Three consequences:
- A concept deleted from the source is deleted from the target repo, provided the connector emits an explicit tombstone. Absence never implies a deletion.
- Manual commits on the sync branch survive while its request is open. Once none is open, the next run rebuilds the branch from the default branch and they are gone; a hand edit to a regenerated concept is overwritten either way.
- Close a kbforge review request only by merging it. The mirror advances on
every publish, so a request's concepts are never re-proposed: closing it
unmerged discards them.
kbforge reset SOURCEundoes that, and the next run proposes everything from the source again, except a deletion whose request was closed unmerged: it is gone from the mirror and the source, so that concept stays in the bundle until you remove it by hand.
Large first runs
A cold start or a bulk upstream edit can put hundreds of concepts in one
review request. A source's chunking: {max_concepts: 40, group_by: <field>}
publishes one chunk at a time and waits for each request to be merged or closed
before the next. group_by can only name a structured field the connector
keeps (local_files drops type, which comes from synthesis); a field no
document carries groups nothing. To redo a chunk, close its request and run
kbforge redo SOURCE before the next kbforge run: closing alone discards the
chunk, and a run after closing moves on. See docs/architecture.md §7.2.
Links no source carries
Two taxonomies with no join key, or a requirement that closes a gap on another
slide: declare them in a reviewed links.yaml.
links:
planning_deck:gaps/redundant-supply:
- to: db_apps:applications/777
note: the application this gap is about
symmetric: true
- planning_deck:requirements/diagnostics # one-way, no note
Links resolve by doc_id, across systems too, and render in a ## Related
section (with the note saying why) that an OKF reader follows; connector
relations get the same section. A cross-system link lands on each side on
that side's run, and the review note names the other system so its request can
be merged first. For concepts that merely share tags or facets, use okfquery related instead. See docs/architecture.md §7.4.
Consuming a bundle
A merged bundle needs no server: OKF is markdown with frontmatter, meant to be read by agents without an SDK. Point the agent at the bundle repo's default branch (it holds only reviewed, merged concepts), through whichever access it has:
| Agent | Access |
|---|---|
| has a shell (Claude Code, an Agent SDK app) | a checkout; its own read, glob and grep tools |
| has no filesystem | the GitHub or GitLab MCP server, reading files on the default branch |
| needs aggregate answers (who owns what, what is stale, what is built on source X) | okfquery query over a checkout |
Give it a front door: run okfquery index
after each merge, so the root index.md lists every concept in one line
(* [Title](path) - description) and the agent opens only what it needs. The
okfquery README has the CI job. Then tell the agent how to read it, for example:
Answer from the knowledge base at
<path or repo>. Start atindex.md, open only the concepts you need, and follow their links. Cite each claim with the concept path and itssourcesids, and checkgenerated.at: say when a concept is old. If the knowledge base does not cover something, say so rather than guessing.
In a trial on a five-concept bundle, a small model given only that prompt read
index.md and four concepts, answered with owners and freshness, and said
plainly that the refund procedure it was asked about was not in the bundle,
which was true: that concept's review request had not merged.
Why you can trust it
- No change, no proposal. Sources are canonicalized and diffed against the mirror, so a jittering export is no change at all: the run stops before synthesis, with no tokens spent and no review request (below).
- Nothing ships unvalidated. Every publish is gated on four agent-facing artifact laws (facet well-formedness, link resolvability, anchor presence, freshness legibility) and a projection↔files coherence check (§4.4).
- A human merges. No publisher has a merge method.
- The order is fixed. fetch → normalize → mirror → diff → scope → synthesize → validate → publish is not pluggable, nor are the no-op and never-merge rules: pluggable would mean optional. Plugins extend stages, and the project file has no key that reaches them.
Where kbforge fits
OKF standardizes the artifact at rest. It says nothing about how a bundle is
produced: pulling from a wiki or a CMDB, telling a real change from an
export's jitter, keeping a claim traceable to its source, and getting an update
to main without a human losing an afternoon to review. kbforge is that half.
| Layer | Standardized by |
|---|---|
| Artifact format | OKF v0.2 |
| Production protocol: connectors, canonicalization, diff, provenance, publish | kbforge |
| Serving protocol | MCP, or any context database that ingests the bundle (see packages/okfquery for a DuckDB reader) |
Status: beta, still 0.x. Every seam only a real service can check has a
live suite (--run-live), not a mock. The largest gap is a first-party
connector for a specific system of record: kbforge-mcp gives the capability,
but nothing ships preconfigured for Confluence, ServiceNow, or their kin.
Companion distributions, versioned independently; none is required:
| Distribution | Import | What it does |
|---|---|---|
kbforge |
kbforge |
the production protocol: this README |
kbforge-mcp (src) |
kbforge_mcp |
makes any MCP server with a select tool and a read-by-id tool a kbforge source, through configuration alone |
kbforge-sql (src) |
kbforge_sql |
makes any database SQLAlchemy can reach a kbforge source, one scoped query per source |
kbforge-okfquery (src) |
okfquery |
reads a published bundle back: okfquery query "SELECT ..." over concepts, sources, links, and parse problems |
Plugins, not credentials. The core ships zero credentialed connectors and
zero CI logic. Real systems of record are plugins, discovered through the
kbforge.connectors (and kbforge.publishers) entry-point group; deployments
are separate, private repositories. A credentialed GitHub Issues connector with
a real incremental cursor is worked through in
examples/github-issues-connector/.
Running one source without a project file
kbforge run --connector NAME runs one source from flags, under the same
mirror lock. --set, --publish-set and --llm-set take connector, publisher
and synthesizer config as YAML-typed KEY=VALUE (--set max_commits=50 is an
integer), so quote text containing : or # (otherwise exit 2).
kbforge run --connector local_files --set path=./docs \
--mirror .kbforge/mirror --out .kbforge/out --state .kbforge/state
kbforge run --connector git_commits --set repo=. \
--synthesizer llm --llm-set model=deepseek/deepseek-v4-flash \
--llm-set 'instructions="Write for application managers."' \
--publisher github --publish-set repo=acme/knowledge-base \
--mirror .kbforge/mirror --out .kbforge/out --state .kbforge/state
--synthesizer, --chunking FILE, --links FILE and --grounding FILE
mirror the project file's keys. kbforge redo takes the run's --connector,
--set, --publisher, --mirror and --state. plan, status and reset
need a project file.
Moving to a project file. Write one source per invocation with its
connector config exactly as you pass it (path: ./docs is not path: docs:
config is hashed into the cursor's name), and set state_dir to the directory
holding your mirror and state as mirror/ and state/ (or move them there).
Then kbforge plan: every source that was up to date reports no change → NoOp, which is the check that it found its own cursor.
Documentation
docs/architecture.md— the spec: hookspecs, the connector protocol and its laws, the fixed pipeline, the project layer, the build sequence. Sections headed not built are specification.docs/context/knowledge-base-design.md— the system kbforge was extracted from, with its security model.docs/design/— design notes: the agentic-ingest roadmap, the data-contract bridge, the OKF v0.2 keys kbforge does not emit yet, and more.CHANGELOG.md— release history.
Related projects
kbforge is one of three contracts for agents, split by seam:
- ai-agent-contracts — the formal spine: resource, temporal, and lifecycle contracts (the seven-tuple kbforge maps onto).
- agentic-data-contracts — the consumption half for structured data: domain-driven governance enforced at query time. kbforge is the production half for unstructured knowledge; both independently converged on making freshness legible to the agent.
Not a context database
OpenViking and its kin sit in the serving row of the table above, not the production row. They ingest documents and expose them to an agent — OpenViking summarizes each one into retrieval tiers on write and serves them over a filesystem API and MCP. kbforge produces the documents such a system serves: an OKF bundle in a git repo is a valid input to one, so these compose rather than compete.
The difference shows on the second pull from a source that mostly did not change. A context database refreshes a watched resource by re-ingesting it wholesale — no diff, no changed-set, no proposal a human ever sees. kbforge canonicalizes first, so an export whose timestamps and ordering jitter reduces to no change at all: no LLM spend, no review request, no merge. Byte-level deduplication further downstream cannot substitute, because a rendered system-of-record export is rarely byte-identical across pulls even when nothing about it has meaningfully changed.
That one mechanism is why both the token bill and the review queue stay bounded on a corpus where most documents are stable — and it is what makes the human gate affordable rather than ceremonial. Reach for a context database when an agent needs to retrieve from a corpus; reach for kbforge when a corpus has to stay honest to a system of record that keeps changing, and someone has to be accountable for what it says.
Development
uv sync --all-extras --dev # create the venv and install
prek install # ruff + ty on every commit
uv run pytest
The default suite never touches the network. Tests that call a real external
service are marked live and skipped unless you pass --run-live.
The forge publishers have a live suite because their offline tests can only
assert what we meant to send — a real forge is the only thing that can say
the intent was right. It needs a throwaway repo on each forge and the two CLIs
(gh, glab) authenticated; each run writes under a fresh live/<run-id>/
prefix, so nothing accumulates and no repo is ever deleted.
That suite covers two levels. Most tests hand-build a ProposedChange and pin
the publisher adapters. One drives kbforge.pipeline.run end to end against a
real forge and pins the composition — that an unchanged source is still a
no-op, and that a later run accumulates into the same review request instead of
rebuilding the branch and dropping concepts an earlier run put there. Those three
pieces of state (mirror, cursor, open-request detection) can each be correct
while the whole is not, which is exactly how the 0.3.0 data-loss bug passed a
clean offline suite.
GITHUB_TOKEN=$(gh auth token) \
GITLAB_TOKEN=$(glab config get token --host gitlab.com) \
KBFORGE_LIVE_GITHUB_REPO=you/kbforge-live-test \
KBFORGE_LIVE_GITLAB_REPO=you/kbforge-live-test \
uv run pytest tests/test_forge_live.py --run-live
License
MIT
Metadata
Release files for kbforge 0.15.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| kbforge-0.15.0.tar.gz | 431.2 kB | With provenance | Details for kbforge-0.15.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| kbforge-0.15.0-py3-none-any.whl | Python 3 | none | any | Details for kbforge-0.15.0-py3-none-any.whl |
Total release size: 545.9 kB
Release files / kbforge-0.15.0.tar.gz
| Download URL | kbforge-0.15.0.tar.gz |
|---|---|
| Size | 431.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
60dd3dd3a712609fb3fcf070c1881c49d3dbdaecbda64a51f224dd42da0bc6d4
|
|
BLAKE2b-256 checksum How to use checksums |
a4b62d03f046eaafe6749f23bffb699cb7329a4395d43aafbff2ec66bcc46067
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 29, 2026.
Transparency logRelease files / kbforge-0.15.0-py3-none-any.whl
| Download URL | kbforge-0.15.0-py3-none-any.whl |
|---|---|
| Size | 114.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
fcd14fbd095ac0c1a1a54da6d608b2284e8c0eaa94500fa0bc8a35fc1d09976e
|
|
BLAKE2b-256 checksum How to use checksums |
6ea74bf3cd4934a53dea42edc22ebdfcb6afc7982dd0c1dcbd5851f50d629cbb
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 29, 2026.
Transparency logSecurity for kbforge 0.15.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for kbforge-0.15.0.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 29, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.