zotkit

Headless Zotero library management — no desktop app required. Built for AI agents and scripts: ships with an agent skill and AGENTS.md.

English | 简体中文

"Headless" simply means zotkit never needs the Zotero app (or any window) open: it is a Python library + CLI that talks straight to the Zotero Web API, so you can search, create, tag, and organize items from any terminal — macOS, Windows, or Linux, your laptop or a remote server. If your attachments sync to a personal WebDAV server, zotkit can upload and download the files themselves by speaking Zotero's WebDAV storage format directly — a capability the Web API itself does not provide. The format is documented in docs/webdav-format.md.

Built for servers, scripts, and LLM agents: bulk writes are dry-run by default, batched, and version-checked, and you can define a tag taxonomy that is enforced in code so an agent (or a tired human) can't pollute your library with inconsistent tags.

Why zotkit

The zotkit family

Two independent, complementary projects — use either on its own, or both together:

They share a name and a philosophy, not a codebase: zotkit-reader can call zotkit over MCP for library-wide operations, but neither requires the other to be installed.

Install

Pure Python (3.11+), no platform-specific bits — the same package works on macOS, Windows, and Linux:

pipx install zotkit        # or: uv tool install zotkit / pip install zotkit
uvx zotkit --help          # …or try it without installing anything

Configure

Copy .env.example to ./.env, ~/.config/zotkit/env, or any path in $ZOTKIT_ENV, and fill in:

• Zotero Web API: create a key (with write access) at https://www.zotero.org/settings/keys — your numeric ZOTERO_LIBRARY_ID is shown on the same page.
• WebDAV (only for attach/fetch): copy the exact values from the Zotero desktop app on any of your machines — Settings → Sync → File Syncing — and append /zotero/ to the URL (the desktop does this implicitly).
• Using Zotero Storage instead of WebDAV? Just leave the WEBDAV_* lines out — attach/fetch automatically use Zotero Storage through the Web API's upload/download endpoints instead. The storage mode is detected from your .env, nothing to configure.

After filling it in, run zotkit doctor — it validates the config file, API access, and attachment storage, and tells you exactly what to fix if anything fails.

Optionally, copy conventions.example.toml to conventions.toml next to your .env to define a namespaced tag taxonomy (field:physics, status:to-read, …). With it in place, zotkit create / zotkit tag reject violations; without it, tags are unrestricted. Searched in order: $ZOTKIT_CONVENTIONS (a full path to the file), then next to the .env in use, then ~/.config/zotkit/conventions.toml.

Quickstart

zotkit find --title "boson sampling"        # search by title/tag/collection
zotkit find --tag status:to-read
zotkit find --any vaswani                   # metadata-wide: title, abstract,
                                            #   creators, tags, extra ("All
                                            #   Fields & Tags"); --abstract
                                            #   for the abstract field only
zotkit show AB12CD34 EF56GH78               # one line per key (read-only);
                                            #   --json for the full item data

zotkit create --arxiv 2401.12345            # fetch arXiv metadata, dry-run preview
zotkit create --arxiv 2401.12345 --apply --tags field:ai   # create + download & attach the PDF
zotkit create --arxiv 2401.12345 1706.03762 math/0211159 --apply   # batch: one metadata
                                            #   request, PDFs politely spaced ≥3 s apart
zotkit create --doi 10.1038/nature14539     # fetch CrossRef metadata by DOI (no PDF —
                                            #   usually paywalled; attach manually after)

zotkit create --file papers.json            # batch from JSON: dry-run preview —
                                            #   flags items already in the library
zotkit create --file papers.json --apply    # create (dedups by DOI/title)
zotkit attach --from papers.created.json --all   # upload the PDFs (WebDAV or Zotero
                                                 #   Storage — auto-detected from .env)

zotkit enrich --key AB12CD34 EF56GH78       # fill missing fields on existing items
                                            #   in place (dry-run; --apply to write)
zotkit enrich --missing abstract --apply    # …or batch: every item missing an
                                            #   abstract (also --missing doi, --all)
zotkit audit                                # read-only health report: what's missing
                                            #   where, stamp hygiene (--json for agents)
zotkit abstract --key AB12CD34 --source cnki --file abs.txt   # paste an abstract
                                            #   with provenance (owner's tool)

zotkit attach --key AB12CD34 --pdf paper.pdf     # single attach
zotkit fetch --key AB12CD34 --out downloads      # download attachments (same auto-detection)

zotkit export --key AB12CD34 EF56GH78 -o refs.bib   # BibTeX; cite key = item key

zotkit tag AB12CD34 topic:qaoa prio:high    # validated against conventions.toml
zotkit status AB12CD34 read                 # replaces the status: tag
zotkit move AB12CD34 "Algorithms"           # or "Parent :: Child"; --add keeps old home

zotkit backup                               # full JSON snapshot -> backups/
zotkit lint field:physics topic:new-idea    # offline tag check

--arxiv takes ids or abs/pdf URLs (several, space- or comma-separated) and maps the full record (all authors, abstract, date, DOI); --doi maps CrossRef records (journal articles, conference papers, books, chapters, …) and refuses to guess on CrossRef types it doesn't know. Both accept --collection and --tags, and --no-pdf skips the arXiv PDFs. Rate limiting is built into the request layer — batches use one arXiv metadata request and space PDF downloads per arXiv's terms of use, so callers (humans or agents) never pace themselves. A bad id fails alone, not the batch; the exit code is non-zero only if something failed. Fetching metadata from arbitrary web pages is out of scope (zotkit stays a daemon-free CLI — no translation-server). CrossRef requests identify themselves to the polite pool with a contact address; that should be reachable, so if you distribute a tool built on zotkit, set your own via ZOTKIT_MAILTO.

Duplicates: with arXiv ids or DOIs in hand, the existence check is the create --arxiv/--doi dry run. It checks the whole batch in one call, by exact DOI and then by normalized title, and prints !! already in library as <KEY> — --apply will skip it (use --no-dedup to force) for each item already there. No separate find is needed, and a loop of one find per candidate only repeats work the dry run already does. --apply runs the same check and names the existing key in its skip messages. --no-dedup turns the check off and creates the item regardless. Without an identifier (a local PDF, hand-built --file JSON), or to answer "do we have anything by X?", use zotkit find --any <first-author-lastname>. That is one call across title, abstract, creators, tags, and extra, and it beats guessing at title wording: title-substring searches routinely take several attempts before a zero hit can be trusted.

Version of record: when arXiv reports a journal DOI (the paper was formally published), --arxiv builds the journal record from CrossRef instead of a preprint: proper item type, venue, volume/pages, formal date. The arXiv identity is kept — arXiv: <id> goes in Extra, the url stays the open-access abs page (the journal link lives in the DOI field), the arXiv abstract fills in when CrossRef has none, and the PDF still comes from arXiv. If the CrossRef lookup fails, the item falls back to the preprint record with a warning rather than failing. A repository DOI is never mistaken for a journal one: DOIs under the preprint servers' own prefixes (arXiv 10.48550, SSRN 10.2139, bioRxiv/medRxiv 10.1101, Research Square, OSF, ChemRxiv, TechRxiv, Preprints.org) don't trigger the upgrade, and neither does a CrossRef record that turns out to be a repository posting in disguise (SSRN registers working papers as articles in a fake "SSRN Electronic Journal") — those stay preprints, with the reason stated. The same predicate gates enrich --rebuild-record. Items whose abstract zotkit wrote also carry an abstract-source: arxiv|crossref line in Extra, naming where it actually came from.

Looking items up: find, show and export

zotkit find --any TEXT matches a case-insensitive substring across title, abstract, creator names, tags, and extra — the Zotero client's "All Fields & Tags" mode — and composes (AND) with --title/--tag/--collection. --abstract TEXT is the same match restricted to the abstract, for when --any hits too much via creators or extra. When a match isn't visible in the one-line result (abstract, extra, or a creator name), an indented hit: <field> "..." line names the field, with ~±60 chars of context for abstract/extra — so you never have to show --json a result just to learn why it matched. Deliberate non-goals: no PDF full-text search (once you need the full text you've already identified the paper — fetch it, don't search it) and no server-side q= search (its index coverage is unreliable — measured on this library it returned 26 hits where 37 abstract-level matches exist — and a zero-hit find answer is used to conclude "not in the library", so completeness is a hard requirement; find stays a local filter over the full library). One caveat: a zero hit on a concept term does not mean the library lacks a paper on that concept — abstract matching is only as good as the stored abstract, and some items carry short publisher/CrossRef abstracts that omit the paper's key terms. The author's last name remains the reliable existence key.

zotkit show K [K …] is the read-only lookup by item key: one line per key — KEY · itemType · FirstAuthor Year · Title · DOI-or-arXiv-id — or --json for the full item data. An unknown key is reported on stderr as a single sanitized line (error: KEY: not found (404) — no request URL, so nothing private lands in CI logs; --verbose restores the full API exception) and the exit code is 1, while the remaining keys still print. This deliberately supersedes requests for a find --key flag and a verify --manifest subcommand: verifying a manifest is a loop over show on the project side, and manifest-file parsing is too project-specific to belong in zotkit.

zotkit export --key K [K …] emits BibTeX for those items, to stdout or to -o refs.bib; keys are also accepted on stdin, one per line, so it composes with whatever produced them. Every entry's cite key is always rewritten to the Zotero item key — that is the point of the command: item keys are the identity in the manifest model, and author-year cite keys don't join against them. An unknown key errors on stderr (exit 1) without stopping the other entries from exporting. The output starts with a fixed provenance comment (% Generated by zotkit <version> -- do not hand-edit. …) so a regenerated file is never mistaken for a hand-kept one. Entries come raw off the Zotero Web API, preserving Zotero's field order, tab indentation, and month = jan abbreviations — so the first textual diff against a file that went through bibtexparser (or any other reflowing tool) is large even when the entries are semantically identical. That's expected, not a bug.

zotkit fetch still defaults to ./downloads and prints the absolute path of every file it saves. If you omit --out while the current directory sits inside a git repository, it warns on stderr and suggests an explicit destination: loose PDF copies committed into a repo break a library-of-record setup, where the library is the one place a file lives.

Enriching existing items

zotkit enrich --key K [K …] completes incomplete items — missing abstracts, DOIs, truncated author lists — from the same arXiv/CrossRef sources, in place. The item key never changes (keys are the stable handle downstream tooling references; delete-and-recreate is not an option), and writes carry the item version, so a concurrent edit fails that item loudly instead of clobbering.

The merge is deliberately conservative: only empty fields are filled; creators are extended only when the current list is a same-order prefix of the authoritative one (the classic truncated-list case — any other difference is reported, not touched); tags, collections, relations, and attachments are never modified; Extra only gains lines. Abstracts zotkit writes are stamped abstract-source: … in Extra — a stamp with no abstract means someone deliberately removed it, and enrich will not re-add it (reported as NEEDS OWNER). Items with no DOI or arXiv id are reported as needs-identifier. When an item still has no abstract only because the upstream source has none, enrich says so with a NOTE abstract still missing: <source> has none … line — "missing, but this source had nothing to fill it with" is a different answer from "nothing missing", and only the note distinguishes them (the item stays up-to-date, since nothing was writable, and audit keeps listing it).

--rebuild-record additionally upgrades a preprint whose paper has since been published (journal DOI present) to the journal record in the same item: itemType, published title/date/venue/volume/pages — with arXiv: <id> appended to Extra and attachments untouched.

Instead of --key, --all runs over every top-level scholarly item and --missing abstract / --missing doi (repeatable) over just the gaps — batching and arXiv/CrossRef rate limiting stay internal, so no pacing or batch-splitting on the caller's side. zotkit audit is the read-only companion: counts and key lists for missing abstracts, missing DOI/arXiv id, missing PDF attachments, and abstract-source stamp hygiene (--json for agents).

For text enrich can't fetch — CNKI, publisher pages, your own summary — zotkit abstract --key K --source <slug> stores a pasted abstract (stdin or --file) with provenance: the text is cleaned (hidden tags,  , soft hyphens, hard-wrapped lines) and an abstract-source: <slug> line records where it came from (cnki, ssrn, publisher, manual = your own words — any lowercase slug). It is the owner's tool, so unlike enrich it may replace an existing abstract — but only with --force, which also rewrites the stamp line in place; all other Extra lines stay untouched.

Item JSON for zotkit create --file (a list, one object per reference):

[{"itemType": "journalArticle", "title": "…",
  "creators": [{"creatorType": "author", "firstName": "A", "lastName": "B"}],
  "date": "2024", "publicationTitle": "…", "DOI": "10.x/y",
  "tags": ["field:physics", "status:to-read"],
  "collection": "Algorithms", "file_path": "/abs/path/paper.pdf"}]

From Python

from zotkit import Zot

z = Zot()                                   # reads .env automatically
z.find(tag="status:to-read")
z.create_items([...])                       # dedup + convention checks
z.attach("AB12CD34", "paper.pdf")           # PDF -> WebDAV / Zotero Storage
z.fetch("AB12CD34", "downloads")
z.set_status("AB12CD34", "read")
z.backup()

z.z is the underlying pyzotero client for anything not wrapped.

Using zotkit with AI agents

zotkit is designed to be driven by coding agents (Claude Code and similar): dry-run defaults, code-enforced tag conventions, and a ready-made Claude Code skill in skills/zotkit/ — copy it to ~/.claude/skills/zotkit/ and any Claude session can search, file, and attach papers for you while respecting your taxonomy.

Want to clean up a messy library, not just maintain one? The battle-tested method — taxonomy design, parallel read-only analysis, serial reviewed writes — is written up in docs/organizing-with-agents.md.

mkdir -p ~/.claude/skills && cp -r skills/zotkit ~/.claude/skills/

Safety model

• create is dry-run by default; --apply to execute.
• Writes go through fetch→modify→update (carries the item version, so concurrent edits fail loudly with 412 instead of clobbering), in batches of ≤ 50.
• zotkit backup snapshots every item, collection, tag, and membership to one JSON file — run it before bulk operations.
• Remember: writes propagate to zotero.org and all your synced devices.

How WebDAV attachments work

(With Zotero Storage, zotkit simply uses the Web API's official file endpoints — this section is about the WebDAV mode.) Zotero's WebDAV storage format is undocumented but simple: each attachment item K is stored as K.zip (the file, zipped) plus K.prop (its md5 + mtime). zotkit creates the attachment item via the Web API and PUTs both objects directly — after which every desktop client syncs the file down normally. Details in docs/webdav-format.md.

The format was determined by interoperability inspection of the author's own library. This project is not affiliated with or endorsed by Zotero.

Limits & roadmap

• find filters the whole library client-side. That's by design: server-side q= search has unreliable index coverage, and find's zero-hit answers must be trustworthy. The listing is cached locally and reconciled against the server's version map on every call (ADR-0004), so only changed items are downloaded. A cached answer is as complete as a fresh one. The first call downloads everything, about 0.5 MB of item JSON per 100 items, which can take 10–20 s for a few hundred items on a slow link. After that, a call on an unchanged library takes a couple of seconds. The same cache serves audit and create's dedup. It lives in ~/.cache/zotkit (%LOCALAPPDATA%\zotkit on Windows; honors $XDG_CACHE_HOME) and holds your items' metadata, abstracts included. Set ZOTKIT_CACHE=0 to turn it off.
• Group libraries should work for item operations (untested); WebDAV file sync is personal-libraries-only (a Zotero limitation).
• --doi/--arxiv import covers arXiv + CrossRef; DataCite-only DOIs are not yet supported (#4).
• Not planned: arbitrary-URL scraping (translation-server territory), SSRN/CNKI scraping, and an overwrite mode for enrich — fill-empty-only is the contract.
• Planned work is tracked as GitHub issues, including the MCP server wrapper (#8).

License

MIT. If you build on the WebDAV implementation, a link back is appreciated.