zotkit

Headless Zotero library management — no desktop app required. Built for AI agents and scripts: ships with an agent skill and AGENTS.md.

English | 简体中文

"Headless" simply means zotkit never needs the Zotero app (or any window) open: it is a Python library + CLI that talks straight to the Zotero Web API, so you can search, create, tag, and organize items from any terminal — macOS, Windows, or Linux, your laptop or a remote server. If your attachments sync to a personal WebDAV server, zotkit can upload and download the files themselves by speaking Zotero's WebDAV storage format directly — a capability the Web API itself does not provide. The format is documented in docs/webdav-format.md.

Built for servers, scripts, and LLM agents: bulk writes are dry-run by default, batched, and version-checked, and you can define a tag taxonomy that is enforced in code so an agent (or a tired human) can't pollute your library with inconsistent tags.

Why zotkit

Install

Pure Python (3.11+), no platform-specific bits — the same package works on macOS, Windows, and Linux:

pipx install zotkit        # or: uv tool install zotkit / pip install zotkit
uvx zotkit --help          # …or try it without installing anything

Configure

Copy .env.example to ./.env, ~/.config/zotkit/env, or any path in $ZOTKIT_ENV, and fill in:

• Zotero Web API: create a key (with write access) at https://www.zotero.org/settings/keys — your numeric ZOTERO_LIBRARY_ID is shown on the same page.
• WebDAV (only for attach/fetch): copy the exact values from the Zotero desktop app on any of your machines — Settings → Sync → File Syncing — and append /zotero/ to the URL (the desktop does this implicitly).
• Using Zotero Storage instead of WebDAV? Just leave the WEBDAV_* lines out — attach/fetch automatically use Zotero Storage through the Web API's upload/download endpoints instead. The storage mode is detected from your .env, nothing to configure.

After filling it in, run zotkit doctor — it validates the config file, API access, and attachment storage, and tells you exactly what to fix if anything fails. It also warns, without failing, when Zotero's global schema (item types, fields, creator types) has moved past the version this zotkit release was verified against.

Optionally, copy conventions.example.toml to conventions.toml next to your .env to define a namespaced tag taxonomy (field:physics, status:to-read, …). With it in place, zotkit create / zotkit tag reject violations; without it, tags are unrestricted. Searched in order: $ZOTKIT_CONVENTIONS (a full path to the file), then next to the .env in use, then ~/.config/zotkit/conventions.toml.

Quickstart

zotkit find --title "boson sampling"        # search by title/tag/collection
zotkit find --tag status:to-read
zotkit find --any vaswani                   # metadata-wide: title, abstract,
                                            #   creators, tags, extra ("All
                                            #   Fields & Tags"); --abstract
                                            #   for the abstract field only
zotkit find --id 2401.12345 10.1038/nature14539   # already in the library? one line
                                            #   per id: KEY or "absent" (local, fast)
zotkit show AB12CD34 EF56GH78               # one line per key (read-only); --json for
                                            #   the full item data; --attachments adds
                                            #   each attachment's md5 (no download)
zotkit cite AB12CD34 EF56GH78               # short citations: "Laughlin, Phys. Rev.
                                            #   Lett. 50, 1395 (1983)"

zotkit create --arxiv 2401.12345            # fetch arXiv metadata, dry-run preview
zotkit create --arxiv 2401.12345 --apply --tags field:ai   # create + download & attach the PDF
zotkit create --arxiv 2401.12345 1706.03762 math/0211159 --apply   # batch: one metadata
                                            #   request, PDFs politely spaced ≥3 s apart
zotkit create --doi 10.1038/nature14539     # fetch CrossRef/DataCite metadata (no PDF —
                                            #   usually paywalled; attach manually after)
zotkit create --doi 10.21468/SciPostPhys.2.1.003 --oa-pdf --apply   # …and attach the
                                            #   legal open-access PDF, if there is one

zotkit create --file papers.json            # batch from JSON: dry-run preview —
                                            #   flags items already in the library
zotkit create --file papers.json --apply    # create (dedups by DOI/title)
zotkit attach --from papers.created.json --all   # upload the PDFs (WebDAV or Zotero
                                                 #   Storage — auto-detected from .env)

zotkit enrich --key AB12CD34 EF56GH78       # fill missing fields on existing items
                                            #   in place (dry-run; --apply to write)
zotkit enrich --missing abstract --apply    # …or batch: every item missing an
                                            #   abstract (also --missing doi, --all)
zotkit audit                                # read-only health report: what's missing
                                            #   where, stamp hygiene (--json for agents)
zotkit abstract --key AB12CD34 --source cnki --file abs.txt   # paste an abstract
                                            #   with provenance (owner's tool)

zotkit attach --key AB12CD34 --file paper.pdf    # single attach (any file type; --pdf
                                                 #   is an alias); --title names it,
                                                 #   --verify downloads it back to check
zotkit fetch --key AB12CD34 --out downloads      # download attachments (same auto-detection)

zotkit export --key AB12CD34 EF56GH78 -o refs.bib   # BibTeX; cite key = item key

zotkit tag AB12CD34 topic:qaoa prio:high    # validated against conventions.toml
zotkit status AB12CD34 read                 # replaces the status: tag
zotkit move AB12CD34 "Algorithms"           # or "Parent :: Child"; --add keeps old home

zotkit backup                               # full JSON snapshot -> backups/
zotkit lint field:physics topic:new-idea    # offline tag check

--arxiv takes ids or abs/pdf URLs (several, space- or comma-separated) and maps the full record (all authors, abstract, date, DOI); --doi maps CrossRef records (journal articles, conference papers, books, chapters, …) and refuses to guess on CrossRef types it doesn't know. A DOI CrossRef has no record for is looked up at DataCite (theses, datasets, software, repository records), under the same rule: only types with a faithful Zotero equivalent, anything else errors with a pointer to --file. A DOI in neither registry (e.g. one registered with CNKI) says so. arXiv's own DOIs (10.48550/arXiv.*) are refused with a pointer to --arxiv, which builds the better record. Both accept --collection and --tags, and --no-pdf skips the arXiv PDFs. Rate limiting is built into the request layer — batches use one arXiv metadata request and space PDF downloads per arXiv's terms of use, so callers (humans or agents) never pace themselves. A bad id fails alone, not the batch; the exit code is non-zero only if something failed. Fetching metadata from arbitrary web pages is out of scope (zotkit stays a daemon-free CLI — no translation-server). CrossRef and Unpaywall requests identify themselves with a contact address; that should be reachable, so if you distribute a tool built on zotkit, set your own via ZOTKIT_MAILTO.

Open-access PDFs (create --doi … --oa-pdf, opt-in): zotkit asks Unpaywall for the DOI's best open-access location and, only when that location has a direct PDF link, downloads it and attaches it as "Full Text PDF". The parent gets an append-only Extra line recording the source: oa-pdf: <license> · <host type> · <version> · <url>. The dry run looks up Unpaywall only and shows what it would download and from where; it fetches no PDF. Nothing else is ever tried: no subscription route, proxy or login, no other OA location, no scraping of a landing page. A download that is refused (a 403, a rate limit, a bot wall or login page instead of a PDF) is reported, with no retry, and that host gets no further download attempts in the run; downloads from other hosts go on. The items are still created, and the exit code is 1. "No open-access copy" is reported, not an error.

Duplicates: with arXiv ids or DOIs in hand, the existence check is zotkit find --id <id…> (or the ids on stdin, one per line). It answers locally, with no arXiv/CrossRef lookup, and prints exactly one line per input, in input order: <id> · <KEY> · <itemType> or <id> · absent (plus · also <KEY> when the library holds duplicates). arXiv ids match with the version stripped, via archiveID, an Extra arXiv: line, the arxiv.org url, or arXiv's own 10.48550/arXiv.* DOI; DOIs match case-insensitively, including a DOI: line in Extra. Absent is an answer, not an error: the exit code is 1 only for an input that is neither an arXiv id nor a DOI. create --arxiv/--doi checks again before writing: its dry run and --apply check the whole batch by exact DOI and then by normalized title, and print !! already in library as <KEY> — --apply will skip it (use --no-dedup to force) for each item already there. The title match also catches a copy that was filed without its identifier, which find --id cannot see. --no-dedup turns that check off and creates the item regardless. Without an identifier (a local PDF, hand-built --file JSON), or to answer "do we have anything by X?", use zotkit find --any <first-author-lastname>. That is one call across title, abstract, creators, tags, and extra, and it beats guessing at title wording: title-substring searches routinely take several attempts before a zero hit can be trusted.

Version of record: when arXiv reports a journal DOI (the paper was formally published), --arxiv builds the journal record from CrossRef instead of a preprint: proper item type, venue, volume/pages, formal date. The arXiv identity is kept — arXiv: <id> goes in Extra, the url stays the open-access abs page (the journal link lives in the DOI field), the arXiv abstract fills in when CrossRef has none, and the PDF still comes from arXiv. If the CrossRef lookup fails, the item falls back to the preprint record with a warning rather than failing. A repository DOI is never mistaken for a journal one: DOIs under the preprint servers' own prefixes (arXiv 10.48550, SSRN 10.2139, bioRxiv/medRxiv 10.1101, Research Square, OSF, ChemRxiv, TechRxiv, Preprints.org) don't trigger the upgrade, and neither does a CrossRef record that turns out to be a repository posting in disguise (SSRN registers working papers as articles in a fake "SSRN Electronic Journal"), nor a DOI that names a dataset or software deposit — those stay preprints, with the reason stated. The same predicate gates enrich --rebuild-record. Items whose abstract zotkit wrote also carry an abstract-source: arxiv|crossref|datacite line in Extra, naming where it actually came from.

Journal abbreviations: journalAbbreviation is stored as dotted ISO 4 ("Phys. Rev. Lett."). Periods can be stripped when rendering but not reliably added back, and desktop citation styles, word-processor plugins and scripts read the stored value. CrossRef's own short title is unreliable: undotted for some publishers, the full title or empty for others. So zotkit uses one normaliser for create, enrich, audit and cite, with three sources in order of authority:

1. Registered: the abbreviation registered by the ISSN network (the "abbreviated key-title"), from a table shipped with zotkit. You can add rows you have verified yourself in a local file of the same format (title<TAB>abbreviation, one per line) named by ZOTKIT_JOURNALS; your rows win. Only a registered form may replace a stored value.
2. Title: a journal title that is visibly abbreviated already ("Appl. Phys. Lett.") is taken as written.
3. Rule-derived: ISO 4 word rules over the LTWA (List of Title Word Abbreviations). zotkit does not ship the LTWA. The ISSN International Centre maintains it and reserves its database rights. To enable this source, download the list yourself from the LTWA page on issn.org and point ZOTKIT_LTWA at the file. A rule-derived value is a candidate, not a registered form. Word-by-word output sometimes differs from the registered abbreviation ("Physica A" is registered, the rules give "Phys. A: Stat. Mech. Appl."). Only LTWA entries tagged English or "Multiple languages" apply. Without that restriction, the French financ- would abbreviate "Finance". A word with no such entry stays whole and is reported, so titles in other languages ("Comptes Rendus. Physique") need the registered table.

Title and rule-derived values are both candidates. A candidate only fills an empty field, never replaces a stored value, and create prints a candidate (rule-derived) note when it stores one. A non-Latin-script title gets no abbreviation.

Issue year: date often holds the online-first date, which can precede the volume's year (online December 2022, January 2023 issue). When CrossRef's print issue date has a different year, create and enrich append issue-date: YYYY[-MM[-DD]] to Extra, on journal articles and conference papers only, never preprints. cite uses that year. The key is deliberately not a CSL variable (issued: would change how the desktop client cites the item).

Looking items up: find, show and export

zotkit find --any TEXT matches a case-insensitive substring across title, abstract, creator names, tags, and extra — the Zotero client's "All Fields & Tags" mode — and composes (AND) with --title/--tag/--collection. --abstract TEXT is the same match restricted to the abstract, for when --any hits too much via creators or extra. When a match isn't visible in the one-line result (abstract, extra, or a creator name), an indented hit: <field> "..." line names the field, with ~±60 chars of context for abstract/extra — so you never have to show --json a result just to learn why it matched. Deliberate non-goals: no PDF full-text search (once you need the full text you've already identified the paper — fetch it, don't search it) and no server-side q= search (its index coverage is unreliable — measured on this library it returned 26 hits where 37 abstract-level matches exist — and a zero-hit find answer is used to conclude "not in the library", so completeness is a hard requirement; find stays a local filter over the full library). One caveat: a zero hit on a concept term does not mean the library lacks a paper on that concept — abstract matching is only as good as the stored abstract, and some items carry short publisher/CrossRef abstracts that omit the paper's key terms. The author's last name remains the reliable existence key.

zotkit show K [K …] is the read-only lookup by item key: one line per key — KEY · itemType · FirstAuthor Year · Title · DOI-or-arXiv-id — or --json for the full item data. With --attachments, one indented line per attachment follows each item:

  attachment ATTKEY01 · paper.pdf · application/pdf · md5 9e107d9d372bb6826bd81d3542a419d6 · stored

The md5 is the attachment item's own, so "is this local file already in the library?" is a comparison against md5sum, with no download. The last field says whether the file is on the server (stored / missing), linked for linked files and URLs, or unknown (…) when the check couldn't tell. The check is one WebDAV HEAD per attachment, or, on Zotero Storage, one request that stops at the download redirect, which is why it is opt-in: plain show stays one request per key for whole reference lists. Given an attachment's own key, show --attachments prints its line too. With --json, the same facts come as an attachments list on each item. An unknown key is reported on stderr as a single sanitized line (error: KEY: not found (404) — no request URL, so nothing private lands in CI logs; --verbose restores the full API exception) and the exit code is 1, while the remaining keys still print. This deliberately supersedes requests for a find --key flag and a verify --manifest subcommand: verifying a manifest is a loop over show on the project side, and manifest-file parsing is too project-specific to belong in zotkit.

zotkit cite K [K …] prints one short citation per key, e.g. Laughlin, Phys. Rev. Lett. 50, 1395 (1983). It shows the first author's surname (A & B for two authors, A et al. beyond), the ISO 4 journal from the normaliser above (else the stored abbreviation), volume, first page or article number, and the issue year when an issue-date: line records one. Preprints cite as Author, arXiv:<id> (year). Keys can also come on stdin. It is read-only and local (the cached listing, no CrossRef). An unknown key errors on stderr and exits 1, while the other keys still print.

zotkit export --key K [K …] emits BibTeX for those items, to stdout or to -o refs.bib; keys are also accepted on stdin, one per line, so it composes with whatever produced them. Every entry's cite key is always rewritten to the Zotero item key — that is the point of the command: item keys are the identity in the manifest model, and author-year cite keys don't join against them. An unknown key errors on stderr (exit 1) without stopping the other entries from exporting. The output starts with a fixed provenance comment (% Generated by zotkit <version> -- do not hand-edit. …) so a regenerated file is never mistaken for a hand-kept one. Entries come raw off the Zotero Web API, preserving Zotero's field order, tab indentation, and month = jan abbreviations — so the first textual diff against a file that went through bibtexparser (or any other reflowing tool) is large even when the entries are semantically identical. That's expected, not a bug.

zotkit fetch still defaults to ./downloads and prints the absolute path of every file it saves. If you omit --out while the current directory sits inside a git repository, it warns on stderr and suggests an explicit destination: loose PDF copies committed into a repo break a library-of-record setup, where the library is the one place a file lives.

--key takes an item key or an attachment's own key (as seen in WebDAV filenames, attach output, or Zotero links). Given an attachment key, fetch downloads that one file and notes its parent on stderr. When nothing is saved, it says why and exits 1: the item has no stored file attachments, the downloads failed, or the key doesn't exist. Scripts can rely on the exit code.

attach --key K --file F uploads any file type (--pdf is kept as an alias). The attachment's title defaults to the filename; --title "Talk slides" sets it instead. --verify downloads each upload back and compares md5 with the local file. On a mismatch or a failed download it names the attachment key on stderr and exits 1, and deletes nothing: delete the attachment or retry. On Zotero Storage, --title and --verify have not been tested against a real storage library yet.

Enriching existing items

zotkit enrich --key K [K …] completes incomplete items — missing abstracts, DOIs, truncated author lists — from the same arXiv/CrossRef/DataCite sources, in place. The item key never changes (keys are the stable handle downstream tooling references; delete-and-recreate is not an option), and writes carry the item version, so a concurrent edit fails that item loudly instead of clobbering.

The merge is deliberately conservative: only empty fields are filled; creators are extended only when the current list is a same-order prefix of the authoritative one (the classic truncated-list case — any other difference is reported, not touched); tags, collections, relations, and attachments are never modified; Extra only gains lines. Abstracts zotkit writes are stamped abstract-source: … in Extra — a stamp with no abstract means someone deliberately removed it, and enrich will not re-add it (reported as NEEDS OWNER). Items with no DOI or arXiv id are reported as needs-identifier. When an item still has no abstract only because the upstream source has none, enrich says so with a NOTE abstract still missing: <source> has none … line — "missing, but this source had nothing to fill it with" is a different answer from "nothing missing", and only the note distinguishes them (the item stays up-to-date, since nothing was writable, and audit keeps listing it).

--rebuild-record additionally upgrades a preprint whose paper has since been published (journal DOI present) to the journal record in the same item: itemType, published title/date/venue/volume/pages — with arXiv: <id> appended to Extra and attachments untouched. When arXiv doesn't name the journal DOI, supply it: zotkit enrich --key K --doi <DOI> --rebuild-record (one key, dry by default). Nothing vouches for a hand-supplied DOI the way the item's own arXiv record would, so it has to pass a same-paper check: the titles agree up to small edits (or the journal adds a subtitle) with the same numbering ("Part II" is another paper), and the first authors' surnames are equal. Otherwise enrich refuses with a NEEDS OWNER line and writes nothing, and there is no override, because a wrong DOI would rewrite the record in place. It also refuses a repository DOI, an item that isn't a preprint, and an item that already carries a different journal DOI.

journalAbbreviation is the one field enrich may correct rather than only fill. It replaces a stored value only with a registered form (see "Journal abbreviations" above). A candidate fills an empty field only when CrossRef has nothing for it. This part needs no DOI or arXiv id, since it reads the journal title. Enrich also appends a missing issue-date: line.

Instead of --key, --all runs over every top-level scholarly item and --missing abstract / --missing doi / --missing abbreviation (repeatable) over just the gaps — batching and arXiv/CrossRef rate limiting stay internal, so no pacing or batch-splitting on the caller's side. zotkit audit is the read-only companion: counts and key lists for missing abstracts, missing identifiers, missing PDF attachments, abstract-source stamp hygiene, and journal abbreviations (--json for agents). Only paper types are checked (journal articles, conference papers, preprints, books and chapters, theses, reports, manuscripts, patents, standards). Webpages, blog posts, documents, presentations and other types are listed once, under "non-paper items", instead of sitting in the missing-abstract/identifier/PDF lists for good, and enrich --all skips them. An identifier is whatever fits the type: a DOI, an arXiv id, an ISBN, or a patent number. An item whose only PDFs are supplementary material (Supplementary Information, a Peer Review File, Nature's …_MOESM1_ESM, Science's …_sm.pdf) is listed separately as "only supplementary PDFs", since its article PDF is still missing. The check reads the attachment's title, or its filename when the title is just the filename. If it misjudges an article PDF, give that attachment a plain title such as "Full Text PDF". The journal check has two parts:

• To fix: what enrich would change, each labelled registered or candidate (rule-derived).
• Unverified: every journal without a registered form, whatever is stored. This holds even when the stored value equals the rule-derived candidate, because agreeing with a guess proves nothing. That list is the worklist for checking abbreviations on the ISSN Portal (then add them to ZOTKIT_JOURNALS). Items saved by the desktop client or the browser connector bypass zotkit, so a periodic audit is how they get caught.

For text enrich can't fetch — CNKI, publisher pages, your own summary — zotkit abstract --key K --source <slug> stores a pasted abstract (stdin or --file) with provenance: the text is cleaned (hidden tags,  , soft hyphens, hard-wrapped lines) and an abstract-source: <slug> line records where it came from (cnki, ssrn, publisher, manual = your own words — any lowercase slug). It is the owner's tool, so unlike enrich it may replace an existing abstract — but only with --force, which also rewrites the stamp line in place; all other Extra lines stay untouched.

Item JSON for zotkit create --file (a list, one object per reference):

[{"itemType": "journalArticle", "title": "…",
  "creators": [{"creatorType": "author", "firstName": "A", "lastName": "B"}],
  "date": "2024", "publicationTitle": "…", "DOI": "10.x/y",
  "tags": ["field:physics", "status:to-read"],
  "collection": "Algorithms", "file_path": "/abs/path/paper.pdf"}]

From Python

from zotkit import Zot

z = Zot()                                   # reads .env automatically
z.find(tag="status:to-read")
z.create_items([...])                       # dedup + convention checks
z.attach("AB12CD34", "paper.pdf")           # PDF -> WebDAV / Zotero Storage
z.fetch("AB12CD34", "downloads")
z.set_status("AB12CD34", "read")
z.backup()

z.z is the underlying pyzotero client for anything not wrapped.

Using zotkit with AI agents

zotkit is designed to be driven by coding agents (Claude Code and similar): dry-run defaults, code-enforced tag conventions, and a ready-made Claude Code skill in skills/zotkit/ — copy it to ~/.claude/skills/zotkit/ and any Claude session can search, file, and attach papers for you while respecting your taxonomy.

Want to clean up a messy library, not just maintain one? The battle-tested method — taxonomy design, parallel read-only analysis, serial reviewed writes — is written up in docs/organizing-with-agents.md.

mkdir -p ~/.claude/skills && cp -r skills/zotkit ~/.claude/skills/

Safety model

• create is dry-run by default; --apply to execute.
• Writes go through fetch→modify→update (carries the item version, so concurrent edits fail loudly with 412 instead of clobbering), in batches of ≤ 50.
• zotkit backup snapshots every item, collection, tag, and membership to one JSON file — run it before bulk operations.
• Remember: writes propagate to zotero.org and all your synced devices.

How WebDAV attachments work

(With Zotero Storage, zotkit simply uses the Web API's official file endpoints — this section is about the WebDAV mode.) Zotero's WebDAV storage format is undocumented but simple: each attachment item K is stored as K.zip (the file, zipped) plus K.prop (its md5 + mtime). zotkit creates the attachment item via the Web API and PUTs both objects directly — after which every desktop client syncs the file down normally. Details in docs/webdav-format.md.

The format was determined by interoperability inspection of the author's own library. This project is not affiliated with or endorsed by Zotero.

Limits & roadmap

• find filters the whole library client-side. That's by design: server-side q= search has unreliable index coverage, and find's zero-hit answers must be trustworthy. The listing is cached locally and reconciled against the server's version map on every call (ADR-0004), so only changed items are downloaded. A cached answer is as complete as a fresh one. The first call downloads everything, about 0.5 MB of item JSON per 100 items, which can take 10–20 s for a few hundred items on a slow link. After that, a call on an unchanged library takes a couple of seconds. The same cache serves audit and create's dedup. It lives in ~/.cache/zotkit (%LOCALAPPDATA%\zotkit on Windows; honors $XDG_CACHE_HOME) and holds your items' metadata, abstracts included. Set ZOTKIT_CACHE=0 to turn it off.
• Group libraries should work for item operations (untested); WebDAV file sync is personal-libraries-only (a Zotero limitation).
• --doi/--arxiv import covers arXiv, CrossRef and DataCite; DOIs from other registration agencies (CNKI, mEDRA, …) need --file.
• Not planned: arbitrary-URL scraping (translation-server territory), SSRN/CNKI scraping, and an overwrite mode for enrich — fill-empty-only is the contract.
• Planned work is tracked as GitHub issues.

License

MIT. If you build on the WebDAV implementation, a link back is appreciated.