kaos-content
Part of Kelvin Agentic OS (KAOS) — open agentic infrastructure for legal work, built by 273 Ventures. See the full KAOS package map for the rest of the stack.
kaos-content is the canonical document model for KAOS — a typed
Block/Inline AST with provenance, annotations, views, and round-trip
serializers.
Every KAOS document processor (kaos-pdf, kaos-office, kaos-web)
produces a ContentDocument; every downstream consumer (search,
chunking, LLM programs, MCP resources) reads one. The shape is inspired
by Pandoc's Block/Inline discipline and Docling's provenance model.
kaos-content does not parse PDFs, fetch URLs, or call LLMs —
companion packages do. This package is what makes them interoperable.
To expose kaos-content parsers and serializers over MCP, add the
companion package kaos-mcp.
Install
uv add kaos-content
# or
pip install kaos-content
kaos-content requires Python 3.13 or newer. Optional extras unlock
specific capabilities:
| Extra | Pulls in | Unlocks |
|---|---|---|
[markdown] |
markdown-it-py[plugins] |
parse_markdown round-trip |
[html] |
lxml>=6.1 |
HTML parser to AST |
[images] |
Pillow>=12.2, numpy |
KaosImage wrapper (PIL + DPI + provenance) |
[layout] |
numpy only |
Layout primitives (X-Y cut, projection profiles, clustering, valley detection). Operates on numeric coordinate arrays — no PIL, no raster IO. Install [images] separately if you also need to load source images. |
[polars] |
polars>=1.5.0 |
TabularDocument ↔ Polars DataFrame |
[duckdb] |
duckdb>=1.1.1 |
DuckDB SQL bridge for tabular |
[nlp] |
kaos-nlp-core |
BM25 search + sentence-level units + fuzzy_binary / minhash dedup levels + the cosine similarity-graph kernels (knn_graph / near_duplicates) behind SemanticGraphDedupLevel |
[graph] |
kaos-graph>=0.1.4 |
The connected_components_from_edges union-find kernel that turns the cosine similarity graph into duplicate groups for SemanticGraphDedupLevel / dedup(embedder=...). Pair with [nlp] and a caller-supplied embedder. |
[transformers] |
kaos-nlp-transformers>=0.2.0a2 |
Dense embedding + cross-encoder retrieval (`SearchableDocument(retrieval='embeddings' |
[clustering] |
scipy>=1.14.1 |
The clustering step of SemanticDedupLevel (hierarchical agglomerative on cosine distance). Pair with [transformers] to actually run the level — both extras are required at find_clusters time. Without them the COMPREHENSIVE and OCR_AWARE presets gracefully degrade to lexical-only. |
[dedup-perceptual] |
imagehash |
Perceptual page-image dedup (PerceptualHashLevel). For embedding-based semantic dedup, install kaos-content[transformers,clustering] to enable SemanticDedupLevel (KNT-602 0.1.0a3 — the level lives natively in this package now, no longer registered by an external plugin). |
Quick start
Build a small document, serialize it, search it, and walk its sections:
from kaos_content.model.document import ContentDocument
from kaos_content.model.metadata import DocumentMetadata
from kaos_content.shortcuts import bold, heading, link, paragraph, text
from kaos_content.search import search_document
from kaos_content.serializers.markdown import serialize_markdown
from kaos_content.serializers.html import serialize_html
from kaos_content.views.document_view import DocumentView
doc = ContentDocument(
metadata=DocumentMetadata(title="Hello"),
body=(
heading(1, "Hello, KAOS"),
paragraph(
text("Built on a "),
bold("typed AST"),
text(" with "),
link("https://kelvin.legal", "provenance"),
text("."),
),
),
)
print(serialize_markdown(doc))
# # Hello, KAOS
#
# Built on a **typed AST** with [provenance](https://kelvin.legal).
print(serialize_html(doc, allow_raw_html=False))
# <h1>Hello, KAOS</h1><p>Built on a <strong>typed AST</strong> with
# <a href="https://kelvin.legal">provenance</a>.</p>
results = search_document(doc, "typed AST", top_k=5)
for r in results.results:
print(r.score, r.block_ref, r.text[:80])
for section in DocumentView(doc).flat_sections:
print(section.heading_text, section.heading_ref)
The AST is constructed entirely from frozen Pydantic models — every
field is type-validated at construction time, content-model
constraints (blocks contain blocks or inlines, never mixed) are
enforced before a node ever reaches a serializer, and the same
ContentDocument round-trips losslessly through JSON via
model_dump_json() / model_validate_json().
Concepts
The package is built around nine composable primitives.
| Concept | What it is |
|---|---|
ContentDocument |
Frozen Pydantic AST: metadata + body of Block nodes |
Block / Inline |
Strictly separated — blocks contain blocks or inlines, never mixed; enforced at construction |
Provenance |
Source file, page, bounding box, char span, confidence — on any node |
Attr |
Pandoc-style (id, classes, key-value) triple — universal extension mechanism |
Annotation |
Standoff layer for overlapping marks (redactions, defined terms, citations, NLP entities) |
node_ref |
JSON pointer addressing (e.g. #/body/5) — stable target for MCP resources |
DocumentView |
Dynamic hierarchical views (pages, sections, paragraphs, sentences) computed from the flat AST |
TabularDocument |
Universal tabular AST — peer to ContentDocument, 17-type column system, Polars/DuckDB bridges |
KaosImage |
PIL wrapper carrying DPI + provenance, with bomb-resistant load (100 MP cap) |
Compatibility & status
| Aspect | |
|---|---|
| Python | 3.13, 3.14, 3.15 pre-release (CI runs all three on linux-x64; 3.13 on macOS-arm64 and Windows-x64). 3.14t free-threaded is intentionally skipped until the dependency ecosystem ships wheels for it. |
| OS | Linux, macOS, Windows (pure-Python wheel; no native code) |
| Maturity | Alpha (0.1.0a1). The Block/Inline grammar is near-stable; serializer flags, traversal helpers, and the [dedup-perceptual] / [layout] APIs are subject to refinement during the alpha cycle. |
| Stability policy | Pre-1.0: minor bumps (0.x → 0.(x+1)) may break behaviour; patch bumps are additive only. Every change is documented in CHANGELOG.md. The MCP tool surface and the safe-by-default serializer/parser/SQL contracts are public API. |
| Test coverage | 2,068 unit tests + 42 Hypothesis property tests pass on Python 3.13. |
| Type checker | Validated with ty, Astral's Python type checker. |
Security model
kaos-content ships safe-by-default serializers and bridges. The
contract:
| Surface | Default | Override |
|---|---|---|
serialize_html(allow_raw_html=False) |
strips raw HTML blocks; neuters javascript: / data: / vbscript: / file: URLs to # with a data-unsafe-url forensic attribute |
allow_raw_html=True for trusted content |
serialize_markdown(allow_raw_html=False) |
same URL-neutering; <!-- raw {format} stripped --> placeholder for raw blocks |
allow_raw_html=True |
parsers.html |
URLs canonicalised through HTML-entity decode → percent-decode → whitespace removal before scheme checks; defeats jav	ascript: and javascript%3A style bypasses |
— |
bridges.duckdb.execute_query(untrusted_sql=True) |
application-level deny-list rejects read_csv, read_parquet, attach, copy, install, load, pragma (and SQL-comment evasions); strips line/block comments before matching |
untrusted_sql=False for fully trusted SQL |
bridges.duckdb.create_safe_connection() |
engine-level sandbox: enable_external_access=false, unsigned-extension loads disabled |
use a raw duckdb.connect() if you need filesystem access |
KaosImage.from_bytes / from_path |
rejects images > 100 MP via ImageDecompressionBombError (PIL's warning promoted to a hard error) |
kaos_content.images.model.MAX_IMAGE_PIXELS = N |
images.artifacts.load_image(max_bytes=50_000_000) |
rejects artifact bodies > 50 MB before decoding | max_bytes=None for trusted artifacts |
BoundingBox / Provenance / Cell / Image |
Pydantic Field constraints reject inverted boxes, page=0, confidence>1, zero/negative spans, zero/negative dimensions at construction time |
— |
See SECURITY.md for the disclosure policy and threat model.
MCP tools
kaos-content registers 17 MCP tools through
kaos_content.tools.register_content_tools(runtime) (count enforced by
tests/unit/test_tools.py::TestRegistration
to prevent README drift):
| Tool | What it does |
|---|---|
kaos-content-parse-markdown |
Parse markdown text into a ContentDocument artifact |
kaos-content-serialize |
Load an artifact and serialize to markdown / HTML / text |
kaos-content-chunk-document |
Split a document at heading boundaries, store chunks as artifacts |
kaos-content-search-document |
BM25 / term-frequency search with AST block_ref results |
kaos-content-search-table |
Case-insensitive substring search inside a TabularDocument |
kaos-content-extract-section |
Pull a section by heading ref into a standalone document |
kaos-content-extract-page |
Pull a single page (requires page provenance) |
kaos-content-context-window |
Expand a node ref into a structural context window |
kaos-content-dedup-semantic |
Lexical / semantic dedup over a corpus of artifacts |
kaos-content-stats |
Per-document statistical summary (counts, structure, pages) |
kaos-content-sentences-with-dates |
Yield sentences containing typed date entities |
kaos-content-sentences-with-money |
Yield sentences containing typed money entities |
kaos-content-sentences-with-percents |
Yield sentences containing typed percent entities |
kaos-content-sentences-with-durations |
Yield sentences containing typed duration entities |
kaos-content-sentences-with-numbers |
Yield sentences containing typed numeric entities |
kaos-content-corpus-summarize |
Build / refresh DocumentSummary for each artifact in a corpus |
kaos-content-corpus-narrow |
BM25-rank a corpus by query against per-document summaries |
Configuration
kaos-content has no module-level environment variables — its public
APIs are all in-process. Settings that affect behaviour are documented
inline at the call site (MAX_IMAGE_PIXELS, DEFAULT_LOAD_IMAGE_MAX_BYTES,
allow_raw_html, untrusted_sql).
Companion packages
kaos-content is one of the packages in the
Kelvin Agentic OS. The broader stack:
| Package | Layer | What it does |
|---|---|---|
kaos-core |
Core | Foundational runtime, MCP-native types, registries, execution engine, VFS |
kaos-content |
Core | Typed document AST: Block/Inline, provenance, views |
kaos-mcp |
Bridge | FastMCP server, kaos management CLI, MCP resource templates |
kaos-pdf |
Extraction | PDF → AST with provenance |
kaos-web |
Extraction | Web extraction, browser automation, search, domain intelligence |
kaos-office |
Extraction | DOCX / PPTX / XLSX readers + writers to AST |
kaos-tabular |
Extraction | DuckDB-powered SQL analytics |
kaos-source |
Data | Government + financial data connectors (Federal Register, eCFR, EDGAR, GovInfo, PACER, GLEIF) |
kaos-llm-client |
LLM | Multi-provider LLM transport |
kaos-llm-core |
LLM | Typed LLM programming (Signatures, Programs, Optimizers) |
kaos-nlp-core |
Primitives (Rust) | High-performance NLP primitives |
kaos-nlp-transformers |
ML (Rust) | Dense embeddings + cross-encoder reranking via Rust ort cdylib (libonnxruntime statically linked) |
kaos-graph |
Primitives (Rust) | Graph algorithms + RDF/SPARQL |
kaos-ml-core |
Primitives (Rust) | Classical ML on the document AST |
kaos-citations |
Legal | Legal citation extraction, resolution, verification |
kaos-agents |
Agentic | Agent runtime, memory, recipes |
kaos-reference |
Sample | Reference module for module authors |
Packages depend on kaos-core; everything else is opt-in. Mix and match
the ones you need.
Development
git clone https://github.com/273v/kaos-content
cd kaos-content
uv sync --group dev
Install pre-commit hooks (recommended — they run the same checks as CI on every commit, scoped to staged files):
uvx pre-commit install
uvx pre-commit run --all-files # one-time full sweep
Manual QA commands (the same set CI runs):
uv run ruff format --check kaos_content tests
uv run ruff check kaos_content tests
uv run ty check kaos_content tests
uv run pytest -m "not live and not network and not slow"
Build from source
uv build
uv pip install dist/*.whl
Contributing
Issues and pull requests are welcome. See CONTRIBUTING.md
for setup, quality gates, pull request expectations, and engineering
standards. By contributing you agree to follow the
project conduct expectations and certify the
Developer Certificate of Origin v1.1 —
sign every commit with git commit -s. Please open an issue before starting
on a non-trivial change so we can align on scope.
Security
For security issues, please do not file a public issue. Report privately via GitHub Private Vulnerability Reporting or email security@273ventures.com. See SECURITY.md for the full disclosure policy.
License
Apache License 2.0 — see LICENSE and NOTICE.
Copyright 2026 273 Ventures LLC. Built for kelvin.legal.
Release files for kaos-content 0.1.7
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| kaos_content-0.1.7.tar.gz | 1.1 MB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| kaos_content-0.1.7-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 1.4 MB
Release files / kaos_content-0.1.7.tar.gz
| Download URL | kaos_content-0.1.7.tar.gz |
|---|---|
| Size | 1.1 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
f19bd05b17e0727c2ed73aea1ac2c30d84b3e7ed31583b69a4ab5be9a1c90ae2
|
|
BLAKE2b-256 checksum How to use checksums |
17e6bf2ad25df1e0fb79d909f1b979ebb83f7de6ae97b6722470b16c82e66639
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 23, 2026.
Transparency logRelease files / kaos_content-0.1.7-py3-none-any.whl
| Download URL | kaos_content-0.1.7-py3-none-any.whl |
|---|---|
| Size | 289.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
2e42ac887d015075fe0bab593d54f25d970a58153a8eb18373b52ef74fbd43e7
|
|
BLAKE2b-256 checksum How to use checksums |
d19986ab1a715ed65b3a16b20662533c56a52540da849da4e2fc89c47c0da681
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 23, 2026.
Transparency log