Skip to main content

Booksmart's book-ingestion pipeline: parsing, structure, extraction, summaries, embeddings — as durable Stage functions consumers drive with their own Runner.

Project description

booksmart-core

Booksmart's book-ingestion pipeline as a typed library: parsing, structure detection, profile generation, knowledge extraction, summaries, and embeddings — exposed as durable, synchronous Stage functions that a consumer drives with its own Runner (see ADR 0002).

Core owns the domain: the ORM models, a single dialect-neutral Alembic history (SQLite and Postgres), object storage, the provider abstractions (Anthropic / OpenAI / Gemini, plus deterministic fakes), and the Qdrant vector store. It reads no environment at all — a consumer constructs Settings explicitly (API keys included; since 0.2.0 there is no env-var fallback) and owns engine and session lifecycle.

Consumers:

  • booksmart — the local CLI (SQLite, embedded Qdrant).
  • booksmart-api — an Inngest-backed service (Postgres) that wraps each Stage in a durable step.
from booksmart_core.database import create_engine, upgrade_to_head
from booksmart_core.runner import execute_run

upgrade_to_head(url)
run_id = execute_run(session_factory, storage_root, book_id, "full")

Search

booksmart_core.search.search is the read side of the embedding pipeline, and the single seam every consumer's search surface sits on — the CLI's search command and booksmart-api's HTTP endpoint call it with the same arguments, against embedded and served Qdrant respectively.

from booksmart_core.search import search

hits = search(session, vector_store, embedder, "how do deep modules help?", limit=5)

It embeds the query with the collection's locked model (refusing a mismatch, ADR 0001), ANN-searches Qdrant, and resolves each hit back to its detached ORM row.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

booksmart_core-0.2.0.tar.gz (35.8 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

booksmart_core-0.2.0-py3-none-any.whl (45.4 kB view details)

Uploaded Python 3

File details

Details for the file booksmart_core-0.2.0.tar.gz.

File metadata

  • Download URL: booksmart_core-0.2.0.tar.gz
  • Upload date:
  • Size: 35.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for booksmart_core-0.2.0.tar.gz
Algorithm Hash digest
SHA256 19dd593bd26ac1520cc4e1d14ba3c5f8c817f5363adc4650a23a4ef7fcba5bbf
MD5 a9e249d26d61a38a3fd03118444ce767
BLAKE2b-256 e88133b0cf28d9c9288ebc12dca128b1540ae038ea028ca21e9163eefde0ecdf

See more details on using hashes here.

Provenance

The following attestation bundles were made for booksmart_core-0.2.0.tar.gz:

Publisher: publish.yml on dworznik/booksmart

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file booksmart_core-0.2.0-py3-none-any.whl.

File metadata

  • Download URL: booksmart_core-0.2.0-py3-none-any.whl
  • Upload date:
  • Size: 45.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for booksmart_core-0.2.0-py3-none-any.whl
Algorithm Hash digest
SHA256 63f77315a8d158489ee2dfbbae67f8e199430660598045929d3400d39ccd9ac0
MD5 1cc80ec3ee6eb5960b41852c943d1d15
BLAKE2b-256 57868acecb3c6dbe9e6d551224f951e5355dfc29e0a8e84de8a2284781ffdce7

See more details on using hashes here.

Provenance

The following attestation bundles were made for booksmart_core-0.2.0-py3-none-any.whl:

Publisher: publish.yml on dworznik/booksmart

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page