cquarry

Canonical Calibre database layer and search grammar engine for Calibre libraries.

This library powers CalibreQuarry (CLI/TUI), Hermitage (GTK4 gallery), Carrel-calibre-web (web reader), and Bindery (EPUB repair & audit). By centralizing the search grammar parser and metadata access, cquarry evaluates virtual library definitions and search queries consistently across frontends.

Features

• Direct SQLite access. No calibredb binary required, no Calibre Python initialization overhead.
• Coherence on demand. Caches are lazy and cheap, and CalibreDB.refresh() clears them all in one call, so a connection held open across an external Calibre write never contradicts itself.
• Lock-safe snapshots. Automatically detects if Calibre holds an exclusive write lock on metadata.db and routes queries through a temporary WAL-consistent copy.
• Full search grammar parity. A recursive-descent parser implementing Calibre's native search capabilities: boolean logic, field prefixes, date math (hyphen and slash separators), hierarchical tags with ./.. component modifiers on every text field, custom columns, identifiers, saved-search interpolation (search:"Name"), multi-valued count operators (tags:#>3), language canonicalization (languages:English → eng), and nested virtual library cross-references. Adversarial grammar-valid queries surface as ParseException, never a raw RecursionError; an empty query after any location matches nothing; invalid boolean keywords raise; two-letter language codes canonicalize (languages:ja matches jpn); super-quotes """...""" shield quote- and paren-heavy queries from the lexer; and the vocabulary is upstream-strict (dates and numerics take exactly true/false as presence words), all verified against upstream.
• Native page counts. The pages: location reads Calibre's own books_pages_link table first (maintained by upstream's CountPages integration) and falls back to an int custom column labelled pages; counts also ride along in every book row.
• Entity secondary columns & display config. Book rows carry author_sorts/author_links parallel to authors; get_entities(kind) exposes {id, name, sort, link, count} for authors/series/publishers/tags/languages; custom columns report editable, normalized and their decoded display JSON (enum_values, enum_colors, …); and a typed preferences accessor covers everything else (get_preference, get_field_metadata, get_user_categories, get_tag_browser_state).
• Metadata portability. Read e-reader annotations, per-device reading progress, third-party plugin data, and conversion profiles; sanitize comments HTML for display.
• Book-content search. Calibre's full-text-search.db sidecar is a sanctioned read: get_book_text() returns a format's extracted plain text, search_book_text() runs a folded content search over it, and get_text_extractions() feeds integrity audits (find_failed_text_extraction reports the formats whose extraction failed: scans, DRM, corrupt files). No FTS5 machinery required.
• Opt-in write path. cquarry.write.WritableCalibreDB offers trigger-safe mutations in a separate module the read-only API can never touch: title, authors (with author_sort recomputation), series (+index), publisher, rating (UNIQUE-deduped, with clear_rating), languages (canonicalized to ISO codes), tags (add/remove/clear_tags for all at once), identifiers, comments, generic custom-column writes (layout auto-detected, enumerations validated against display.enum_values, non-editable columns refused, plus add_custom_column_values append semantics for multi-valued columns), format registration/removal, has_cover, full book removal with orphan pruning, and the creation path (add_book: Calibre-parity row insert with trigger-filled sort/uuid, Author/Title (id) layout, atomic format/cover placement with truthful data rows, dry-run plan, copy-only); every mutation queued in metadata_dirtied for OPF resync; entity-wide renames (rename_entity: fix a misspelled author everywhere, merge case variants), cover replacement (set_cover/remove_cover), verbatim sort corrections, and undo-able format repair (save_original_format/restore_original_format); trash management (list_trash/empty_trash/expire_trash) and custom-column schema management (create_custom_column/delete_custom_column). Renames keep the on-disk layout truthful (update_title/set_authors move the book directory and rename format files to the new stems), format writes keep Calibre's FTS index and page counts honest (re-extraction queued in the sidecar, pages rescans flagged), and add_book creates books complete with trigger-filled sort/uuid, atomic format/cover placement, and a dry-run plan.
• Context manager. CalibreDB supports with statements for automatic cleanup of snapshot files.
• Zero dependencies. Pure Python 3.14+ stdlib (sqlite3, re, json, unicodedata).

Usage

from cquarry.db import CalibreDB

# Open a library (creates a snapshot if Calibre has the lock)
with CalibreDB("~/Calibre Library/metadata.db") as db:
    # Fetch all books with pre-joined metadata
    books = db.get_all_books()

    # Search using Calibre's native grammar
    sci_fi = db.search("tags:Fic.SciFi and rating:>=4")
    print(f"Found {len(sci_fi)} highly rated Sci-Fi books.")

    # Resolve a virtual library to a set of book IDs
    wing = db.resolve_vl("To Read")

    # Interpolate a saved search straight from Calibre's preferences
    award_winners = db.search('search:"Award Winners"')

    # Inspect custom columns (#label, bare label, or display name all work)
    cols = db.get_custom_columns()
    status = db.load_custom_column("#reading_status")

    # Single-entity helpers (no whole-library scan)
    book = db.get_book(42)
    epub = db.get_format_path(42, "EPUB")

    # Metadata portability
    highlights = db.get_annotations(42)
    progress = db.get_last_read_positions(42)
    wordcounts = db.get_plugin_data(name="wordcount")

The composed deep fetch combines metadata, formats, custom columns, and annotations:

dossier = db.get_book_dossier(42, include_comments=True)
print(dossier["formats"], dossier["custom_columns"])
print(dossier["comments"]["plain"])  # comments HTML, already stripped

Writes live behind an explicit opt-in import:

from cquarry.write import WritableCalibreDB

with WritableCalibreDB("~/Calibre Library/metadata.db") as wdb:
    wdb.add_tag(42, "Audited")
    wdb.set_identifier(42, "isbn", "9780123456789")
    wdb.clear_identifier(42, "mobi-asin")  # honest no-op when absent

# A multi-book curation pass commits exactly once; any failure (a Ctrl-C
# included) rolls the whole pass back, directories created inside it too:
with WritableCalibreDB("~/Calibre Library/metadata.db") as wdb:
    with wdb.batch():
        wdb.set_pubdate(42, "1991-10-01")
        wdb.add_tag(43, "Audited")

# Every mutation queues an OPF regeneration; check what Calibre will resync:
with CalibreDB("~/Calibre Library/metadata.db") as db:
    print(db.get_dirtied_books())  # e.g. [42, 43]

Installation

pip install cquarry

API at a glance

The full per-method reference lives in API.md. One line per module:

Development

python -m pytest tests/           # full suite
python -m pytest tests/ -v        # verbose

Run with PYTHONPATH=src to exercise this checkout rather than any installed copy.

Six test modules: test_db.py (CalibreDB against fixture databases), test_helpers.py (utility functions), test_search.py (parser, matcher, and integration tests), test_write.py (opt-in write module with trigger-hazard fixtures), test_integrity.py (library integrity predicates), and test_analytics.py (analytics derivations).

See spec.md for the full contract and roadmap.md for planned work.

Acknowledgements

Carrel-calibre-web is a fork of calibre-web that uses cquarry as its search and virtual-library engine. Features proven there flow back into cquarry's roadmap (see Phase 7); calibre-web's original authors deserve the credit for the web experience that fork builds on.

Support

If cquarry is useful to you and you'd like to chip in:

• liberapay · liberapay.com/bdkl
• bitcoin

  bc1qkge6zr45tzqfwfmvma2ylumt6mg7wlwmhr05yv
  

License

MIT. See LICENSE.