zotero-project-manager 0.6.0

Export Zotero collections into clean, incremental research workspaces.

Zotero Project Manager (zpm)

zpm creates clean, ordinary project folders from Zotero collections. Zotero remains the source of truth; zpm only reads its database and attachments and writes into a separate output directory.

The current release supports recursive collection export, multiple collections, configurable portable filenames, incremental updates, SHA-256 verification, safe pruning, DOI/tag metadata, opt-in annotation and child-note Markdown, research indexes, dry runs, diagnostics, and a versioned JSON manifest.

Quick start: export My-AI

After installation, zpm can be run from any directory. Export My-AI with:

zpm export "My-AI" --output ~/ResearchProjects

The files are written to:

~/ResearchProjects/My-AI/

Preview the export without writing anything:

zpm export "My-AI" --output ~/ResearchProjects --dry-run --verbose

Run the same zpm export command whenever the Zotero collection changes. The manifest makes the update incremental: new or changed PDFs are copied and unchanged PDFs are left alone.

Include PDF annotations and Zotero child notes as Markdown:

zpm export "My-AI" --output ~/ResearchProjects --annotations

Inspect changes without writing anything:

zpm status "My-AI" --output ~/ResearchProjects

Preview files removed from Zotero that can be safely pruned:

zpm status "My-AI" --output ~/ResearchProjects --prune

After reviewing the preview, apply safe pruning:

zpm export "My-AI" --output ~/ResearchProjects --prune

Only manifest-owned files whose SHA-256 still matches are deleted. Files changed outside zpm, files without a trusted hash, and unsafe paths are protected.

Other useful global commands:

zpm list
zpm doctor --output ~/ResearchProjects
zpm export --help
zpm --version

Configuration and named projects

Set global defaults once:

zpm config set --zotero-dir ~/Zotero --output ~/ResearchProjects
zpm config show

The default configuration file is ~/.config/zpm/config.toml. Override it with the global --config PATH option or the ZPM_CONFIG environment variable.

Create a named project from one or more collections:

zpm project add ai "My-AI" "Claude"
zpm project list
zpm project show ai

Synchronize it from any directory:

zpm sync ai
zpm sync ai --dry-run

A named project can save recursive traversal, non-PDF inclusion, pruning, full hash verification, metadata and annotation generation, filename ordering, and its own output directory. Use --force with zpm project add to replace an existing project definition.

For example:

zpm project add ai "My-AI" --annotations --filename-template year_author_title --force
zpm sync ai

Metadata and research index

Exports generate two additional workspace files by default:

• metadata.json contains collection keys, Zotero item and attachment keys, titles, dates, creators, DOI, tags, source and destination paths, state, and SHA-256.
• INDEX.md provides a human-readable table with DOI links and relative PDF links.

Disable these generated artifacts when needed:

zpm export "My-AI" --output ~/ResearchProjects --no-metadata

For safety, zpm refuses to overwrite an existing metadata.json or INDEX.md unless that file was previously generated by zpm.

Annotations and child notes

Annotation export is opt-in because comments and notes may contain private research material:

zpm export "My-AI" --annotations

For each attachment, zpm creates a hierarchy-preserving Markdown document under Annotations/. It includes highlights, underlines, comments, colors, page labels, annotation tags, modification dates, image/ink annotation metadata, and child notes. For items in the personal Zotero library, links open the PDF or annotation in Zotero.

Generated annotation files carry a safety marker. zpm refuses to replace an unmanaged Markdown file at the same path and avoids rewriting unchanged generated files. Zotero and its PDFs remain read-only.

Filename preferences

The default remains author_year_title, producing names such as:

Curie - 2024 - Paper title.pdf

Choose another preset for a direct export:

zpm export "My-AI" --filename-template year_author_title

Available presets are:

• author_year_title
• author_title_year
• year_author_title
• year_title_author
• title_author_year
• title_year_author
• author_title
• year_title
• title_author
• title_year
• title

The six three-part presets cover every possible ordering of author, year, and title. The shorter presets always retain the title while allowing the author or year to be omitted. Missing Zotero metadata is skipped gracefully in every preset.

Set a global default:

zpm config set --filename-template year_author_title

Or save it in a named project:

zpm project add ai "My-AI" --filename-template year_author_title --force

The selected template is recorded in the workspace manifest. To avoid unsafe or surprising renames, changing the template for an existing workspace is rejected; export to a new output directory when reorganizing existing filenames.

Safety model

• The Zotero database is opened with SQLite mode=ro and PRAGMA query_only.
• Queries run in a consistent read-only SQLite transaction. Optional --snapshot mode uses the SQLite backup API when an isolated copy is useful.
• zpm never renames, moves, or edits Zotero files.
• Output paths inside the Zotero data directory are rejected.
• Existing directories without a zpm manifest are rejected unless explicitly adopted with --overwrite.

Always keep independent backups of important research data.

Requirements and installation

• macOS with Homebrew, or Python 3.11 or newer

The recommended macOS installation is Homebrew:

brew install sbilmis/tap/zpm

Upgrade a Homebrew installation with:

brew update
brew upgrade zpm

Alternatively, install the published Python package globally with pipx:

brew install pipx
pipx install zotero-project-manager

Upgrade an existing pipx installation with:

pipx upgrade zotero-project-manager

Install from a checkout:

python -m pip install .

For development:

python -m pip install -e '.[dev]'
pytest

Usage

Optional shell helpers

For a checkout at ~/zotero-collection-mirror, create its environment once:

cd ~/zotero-collection-mirror
python -m venv .venv
.venv/bin/pip install -e .

Load the convenience functions in the current shell:

source ~/zotero-collection-mirror/scripts/zpm-helper.sh

Then use:

zpm_collections
zpm_my_ai
zpm_export "My-AI" "Claude"
zpm_export "My-AI" --dry-run --verbose

The helpers default to ~/Zotero and ~/ResearchProjects. Override these locations before sourcing the file when needed:

export ZPM_ZOTERO_DIR="/path/to/Zotero"
export ZPM_OUTPUT_DIR="/path/to/ResearchProjects"
source ~/zotero-collection-mirror/scripts/zpm-helper.sh

List collections, including the keys needed to disambiguate duplicate names:

zpm list
zpm list --zotero-dir ~/Zotero

Export one collection to the current directory:

zpm export "My-AI"

Export several collections beneath a chosen parent directory:

zpm export "My-AI" "Claude" "Image_Processing" --output ~/ResearchProjects

Useful options:

--recursive / --no-recursive       Include descendants (default: recursive)
--pdf-only / --include-non-pdf     Attachment filter (default: PDF only)
--dry-run                          Show counts without writing
--prune                            Safely remove hash-verified stale exports
--verify                           Fully recompute source and destination hashes
--metadata / --no-metadata        Enable or disable metadata and index generation
--annotations / --no-annotations  Export annotations and child notes as Markdown
--filename-template PRESET        Select metadata component ordering
--overwrite                        Adopt an unmanaged destination
--verbose                          Enable diagnostic messages
--linked-attachment-base-dir PATH  Resolve Zotero relative linked files
--snapshot                         Query an isolated temporary database copy

Collection selectors are matched first by exact Zotero key, then by case-insensitive full collection name. Ambiguous names produce an error listing the usable keys.

Incremental behavior

Each workspace contains manifest.json and README.md. On later runs, zpm uses recorded source and destination metadata plus SHA-256 digests to:

• leave unchanged files alone;
• replace changed managed files;
• copy newly discovered files;
• report missing source attachments without deleting an earlier export.
• reconcile identical files re-added under a new Zotero attachment key;
• track files removed from the collection until they are safely pruned.

--overwrite is not needed for normal updates. It is the explicit opt-in for adopting an existing directory or replacing an unmanaged filename conflict. Use --verify for a full content audit even when file metadata appears unchanged.

Manifest v1 workspaces are read automatically. The next successful export adds hashes and upgrades the manifest to v2; status and --dry-run never rewrite it.

Troubleshooting

Zotero database is locked

Zotero can normally remain open because zpm uses read-only SQLite access and waits briefly for transient locks. An export may still be blocked while Zotero is syncing, upgrading its database, or performing maintenance.

If zpm reports that the database is locked:

1. Wait for Zotero synchronization or maintenance to finish, then retry.
2. If it remains locked, close Zotero completely and run the command again.
3. Use zpm doctor --output ~/ResearchProjects to check the configuration.

zpm does not modify Zotero when a lock occurs. A lock error happens before the collection can be read or any export changes are applied.

Attachment paths

Stored Zotero attachments (storage:...), absolute linked paths, and file:// paths are resolved automatically. Zotero's attachments:... relative paths require --linked-attachment-base-dir because that preference is not stored in zotero.sqlite.

Development layout

src/zotero_project_manager/
    cli.py          CLI parsing and presentation
    zotero.py       read-only SQLite and attachment path resolution
    collections.py hierarchy, lookup, and traversal
    exporter.py     safe incremental export orchestration
    filenames.py    portable filename generation and collisions
    manifest.py     versioned manifest serialization
    metadata.py     JSON metadata and Markdown research indexes
    annotations.py  annotation and child-note Markdown generation
    models.py       shared domain dataclasses
    diagnostics.py  read-only doctor checks
    config.py       TOML defaults and named projects
    utils.py        logging and path safety helpers

Current non-goals and roadmap

Better BibTeX, DEVONthink or automatic personal NotebookLM uploads, watch mode, a GUI, and symlink mode are not implemented in v0.6.0. They remain possible future additions; Zotero continues to be the source of truth.