culture-library

Culture's library of your own content and media: recordings, books, newspapers, articles, EPUBs, PDFs, docs and more. One catalog to ingest, index, search and serve it all, published at library.culture.dev.

culture-library is an AgentCulture mesh agent: a Python package with an agent-first CLI, a mesh identity, and the canonical skill kit.

Status

Built, not yet deployed. The library is in culture_library/. It has:

• a storage and ingest core;
• an HTTP API compatible with xteink's;
• search;
• repo documents;
• send to device;
• OPDS and WebDAV;
• an MCP server for agents;
• the site.

It was specified and built with devague: spec in docs/specs/, plan in docs/plans/, the run's record in docs/deliveries/. Deploy files are in deploy/, and runbooks in docs/operations/. Deploying and provisioning library.culture.dev are still to do (see Roadmap).

What's here today

• An agent-first CLI cited from teken (afi-cli). The runtime package has no third-party dependencies.
• A mesh identity: culture.yaml (suffix: culture-library, backend: claude) plus the matching resident prompt file, CLAUDE.md.
• Four harness prompt files, one per agent harness (see Prompt files by harness).
• The guildmaster skill kit under .claude/skills/, vendored cite-don't-import. See docs/skill-sources.md.
• A build + deploy baseline: pytest, lint, the agent-first rubric gate, a per-harness smoke check, and PyPI Trusted Publishing in GitHub Actions.

RAG and evals

Agents can fetch citable passages, not just titles. POST /api/rag/retrieve (also the MCP rag_retrieve tool) takes a query and returns ranked passages with item_id, chunk_no, chunk_sha, full text and scores, optionally reranked. Before this, search (core/search.py Hit) was item-level with a 200-char snippet, and docs/evidence/t7-embedding-cost.md measured embedding cost but not retrieval quality (issue #7 compares embedders).

The eval verbs measure retrieval with RAGAS context precision and recall, stored in the library so runs compare across modes and embedders:

uv sync --extra eval                          # ragas lives in the [eval] extra only
uv run culture-library eval generate ITEM_ID  # a book's own text, or a newspaper's sources sidecar
uv run culture-library eval run SET_ID --mode hybrid --k 5
uv run culture-library eval show --set SET_ID

Newspaper sets use the sources sidecar at PUT /api/library/{id}/sources. Eval sets and runs live behind /api/eval/sets and /api/eval/runs. The judge is an OpenAI-compatible endpoint (default: the local lobes gateway); text from non-public items is never sent to a non-local judge. Install culture-library[eval] to use it; the base package and the server image do not include it.

Planned, not done: embeddings are not yet enabled on the live library (see "Enable embeddings" in docs/operations/deploy.md), and the baseline eval runs have not been recorded.

Quickstart

uv sync
uv run pytest -n auto                 # run the test suite
uv run culture-library whoami         # identity from culture.yaml
uv run culture-library learn          # self-teaching prompt (add --json)
uv run culture-library doctor         # agent-identity invariants
uv run teken cli doctor . --strict    # the agent-first rubric gate CI runs

CLI

Every command supports --json. Results go to stdout, errors/diagnostics to stderr (never mixed). Exit codes: 0 success, 1 user error, 2 environment error, 3+ reserved. Library verbs (ingest, search, serve, …) will be added as new noun groups on the same contract.

Roadmap

• Ingest: bring your own recordings, books, newspapers, articles, EPUBs, PDFs and docs into one catalog.
• Index & search across everything in the catalog.
• Serve the catalog, published at library.culture.dev.

The design is not settled yet. This list restates the project's goal; it does not describe a committed design.

Prompt files by harness

Four harnesses, four root files, no shared base — each file is read by exactly one harness:

CLAUDE.md is the fullest write-up of the repo's conventions; read it first. AGENTS.override.md stops Pi inheriting CLAUDE.md, and .pi/SYSTEM.md gives Pi the non-coding associate identity (read/find/summarize only). There is intentionally no AGENTS.md at the root: it would shadow the Pi and colleague prompt cascades, and the CI smoke check fails if one appears.

Two selections, not one

1. The interactive harness is whichever binary you run (claude, pi, colleague, qwen). All four are live over the same clone at once, and choosing one never requires editing a file. See docs/automation-contract.md.
2. The mesh resident is the single backend in culture.yaml. It is what the Culture daemon starts and what steward doctor checks, and only guild harness use <name> changes it.

See docs/harness-selection.md for the full writeup.

Development

• Every PR bumps the version (the version-bump skill updates pyproject.toml and CHANGELOG.md). The version-check CI job fails the PR otherwise.
• Lint: black, isort, flake8 (line length 100), bandit, markdownlint, and scripts/scan-secrets.py. See CLAUDE.md for the exact commands.
• Deploy: pushing to main publishes to PyPI via Trusted Publishing, and PRs publish a .devN build to TestPyPI (.github/workflows/publish.yml).

The gate section

culture.yaml carries a top-level gate section that declares how to set up and test this repo, so tooling (for example a PR-fixer that re-runs the checks before pushing) needs no per-repo knowledge:

gate:
  setup:
  - [uv, sync]
  test:
  - [uv, run, pytest, -n, auto]

setup and test are each a list of commands, and each command is an argv list (never a shell string), run in order from the repo root. The values are configuration only; no tool supplies a default. Culture's loader ignores unknown top-level keys, so the section is safe alongside agents.

License

Apache 2.0 — see LICENSE.