logoxide for Python

Bibles, commentaries, lexicons, books, and media as versioned packages, from logoxide (its source, its issues, and lx, its command line), with topos's whole API. These bindings are its crates/logoxide-ffi, made through BoltFFI: one API for Python (this package, a native module), TypeScript and JavaScript (npm: logoxide), Swift, and Kotlin.

Topos, ToposTranslations, and the rest are the same as in topos's own package (topos-bible). A book's text is logoxide's own LogoxideBook (pkg.book(id), or LogoxideBook.parse(topos, "usx", text) for a file of your own): its verses, and passages as HTML or plain text. It was topos's ToposBook, with the same methods; its BookVerse and BookTextOptions keep their names.

The host reads and writes the files: an app's file API is often asynchronous (Obsidian's vault), and WebAssembly has none. logoxide reads the text from the bytes it's given, and hands back the cache files it wrote for the host to save. This package is native, so it also has open_package_dir, read_passage, and compare_passage, which read folders themselves (WebAssembly, npm's package, can't).

The examples

Each feature below has an example in Python. They run in order, as one program: each uses what the ones before it made (kjv, bsb, project, installer, ...). What a line prints is written after it, following → (… stands for text left out).

They're checked: examples/python/readme.py runs every example here and compares what it prints with its → comments. They run in a folder holding the KJV and BSB packages (kjv/ and bsb/) and the files in examples/fixtures, and answer the downloads the examples make offline: https://example.org/kjv and https://example.org/bsb are those packages' repositories, and https://example.org/catalog.toml and eBible.org's list and zip are fixtures, so what's printed is theirs. With real URLs the code is the same.

• Loading
• Opening a package
• References, with topos
• A book's text
• Commentaries, dictionaries, and cross-references
• Audio and video
• Versification maps
• A vault's project
• Catalogs
• Installing
• The vault's packages
• Imports
• Making packages
• Reading natively
• Errors
• Every export
• Source and building

Loading

The module is ready on import. Names are snake_case. Records are frozen dataclasses, every field given; enums without data are IntEnums (PackageKind.BIBLE), and enums with data are a class each (PackageLocationLibrary(dir=...), a subclass of PackageLocation). A field or argument named import is import_, and one named from is from_. Bytes are bytes, and so are lists of small numbers (books(), verse counts).

The host's own helpers, used by the examples below: a folder's files, writing files, and HTTP.

import dataclasses
import pathlib
import urllib.request

from logoxide import Logoxide, PackageFile, Topos


def files(folder: str) -> list[PackageFile]:
    """Every file under a folder but git's, with /-separated paths relative to it"""
    root = pathlib.Path(folder)
    return [
        PackageFile(path=p.relative_to(root).as_posix(), bytes=p.read_bytes())
        for p in sorted(root.rglob("*"))
        if p.is_file() and ".git" not in p.relative_to(root).parts
    ]


def write(folder: str, written: list[PackageFile]) -> None:
    """Writes files under a folder"""
    for f in written:
        path = pathlib.Path(folder, f.path)
        path.parent.mkdir(parents=True, exist_ok=True)
        path.write_bytes(f.bytes)


def file(path: str, text: str) -> PackageFile:
    """A file from its text"""
    return PackageFile(path=path, bytes=text.encode())


def get(url: str) -> bytes:
    """The host's HTTP GET"""
    with urllib.request.urlopen(url) as response:
        return response.read()


def get_first(urls: list[str]) -> bytes:
    """The first of several URLs that answers"""
    for url in urls[:-1]:
        try:
            return get(url)
        except OSError:
            pass
    return get(urls[-1])


logoxide = Logoxide()
topos = Topos()
print(logoxide.version())  # → 0.1.1

Opening a package

open_package takes every file under the package's folder (but .git), with paths like usx/JHN.usx, the features to turn on, and the cache files saved before. Opening counts every book's verses unless the cache has them (for USFM that means parsing the whole text, under a second for the KJV), so the host saves what cache_writes() hands back and gives it back next time. info() is the package's [package]; checksum() is what a lock file's checksum should be, to catch changed files.

from logoxide import PackageFeatures, PackageKind

features = PackageFeatures(features=[], default_features=True)
cache: list[PackageFile] = []  # the host's cache folder
kjv = logoxide.open_package(files("kjv"), features, cache)
cache += kjv.cache_writes()  # save these, and give them back next time
info = kjv.info()
print(info.id, info.abbreviation, info.aliases, info.version)  # → kjv KJV ['AV', 'King James'] 1.0.3
print(info.name, info.kind == PackageKind.BIBLE, info.language, info.right_to_left)  # → King James Version (1769) True en False
print(info.license, info.rights)  # → LicenseRef-PublicDomain Rights(redistributable=True, derivatives=<Derivatives.ALLOWED: 0>, commercial=True, attribution=False)
print(info.work, info.edition, info.edition_year, info.versification)  # → kjv 1769 1769 KJV
print(info.upstream.note)  # → fetch.bible's copy of eBible.org's eng-kjv2006
print(kjv.format(), len(kjv.books()), kjv.books()[0], len(kjv.chapter_verses(43)))  # → usfm 66 1 21
print(kjv.checksum())  # → sha256:…
print(len(logoxide.open_package(files("kjv"), features, cache).cache_writes()))  # → 0

Its files: each book's file and title (book_files, titles), what its format passed over (skipped: front matter, books logoxide doesn't know), and the hash of each verse's words (verse_hashes: verse-hash rule 1, the first 7 bytes of a SHA-256, which links keep to find a verse again in a later version).

print(next(f for f in kjv.book_files() if f.book_id == 43))  # → BookFile(book_id=43, path='usfm/jhn.usfm', title='John')
print(kjv.titles()[0], kjv.skipped())  # → TitledFile(path='usfm/1ch.usfm', title='1 Chronicles') []
hashes = kjv.verse_hashes(43)
print(len(hashes), next(v for v in hashes if (v.chapter, v.verse) == (3, 16)))  # → 879 VerseHashEntry(chapter=3, verse=16, last=16, part=0, hash='…')

open_package_dir reads a folder itself and keeps its cache in a folder of its own (so cache_writes() is empty). read_package_info reads a package's logoxide.toml alone, without its text (to name a package that's turned off).

bsb = logoxide.open_package_dir("bsb", features, "cache")
print(bsb.info().abbreviation, bsb.format(), bsb.cache_writes())  # → BSB usx []
print(logoxide.read_package_info(pathlib.Path("bsb/logoxide.toml").read_text()).name)  # → Berean Standard Bible

References, with topos

topos finds and parses references; everything it does (searching text, completing references, queries, indexes, formats) is in this package as in topos's own, and is described in topos's bindings' guide. A package adds itself to topos's ToposTranslations (add_to: its abbreviation, aliases, and verse counts), so a reference naming it is checked against its own verses; add_to_as adds it by the names a project gives it instead.

from logoxide import BookStyle, OffsetUnit, ToposTranslations

translations = ToposTranslations.create()
translations = kjv.add_to(translations)
translations = bsb.add_to_as(translations, "B", ["Berean"])
for reference in ["John 3:16 AV", "3 John 1:15 B", "3 John 1:14 Berean"]:
    found = topos.parse_with_translation(reference, OffsetUnit.CHAR, translations, BookStyle.NAME)
    print(f"{found.passage.reference} in {found.translation.name}" if found else "no such verse")
# → John 3:16 in KJV
# → no such verse
# → 3 John 1:14 in B

A book's text

book(id) parses a book once (cached) for passages as plain text or HTML, one string for each part of a reference (John 3:16; 4 has two). HTML uses logoxide-markup's classes, each with class_prefix: verse and chapter markers are empty <span class="topos-verse" data-n="16">s a stylesheet can number (data-osis-id is the verse as the book names it). Each verse runs from the end of the one before it, so a heading belongs to the verse after it.

from logoxide import BookTextOptions

john = topos.parse("John 3:16-17", BookStyle.NAME)
options = BookTextOptions(notes=False, headings=True, class_prefix="topos-")
kjv_john = kjv.book(john.book_id)
print(kjv_john.book_id(), len(kjv_john.chapters()), kjv_john.chapter_verses()[2], kjv_john.verses()[0])  # → 43 21 36 BookVerse(chapter=1, verse=1, last=1, last_chapter=1, part=0)
print(kjv_john.text(john, options))  # → ['¶ For God so loved the world, …']
html = bsb.book(43).html(john, options)[0]
print(html[html.index("<span") : html.index("</span>") + 7])  # → <span class="topos-verse" data-osis-id="JHN 3:16" data-n="16"></span>
print(len(bsb.book(43).chapter_html(3, options)) > 0)  # → True

verse_texts gives every verse of a book as reading text in one call: its words (notes and headings left out, normalized as verse-hash rule 1 says, so its hash is the one verse_hashes gives and links keep), the heading before it, and whether it starts a paragraph (or stanza) or a line of poetry, for a reader, a drill, or a search of its own. A host keeping them keys them by text_version(), which changes when a release reads files differently. words splits a text into words as matched loosely (each folded: lowercase, without accents; letters and digits alone), to compare what someone typed or said with a verse.

verses = bsb.book(43).verse_texts()
v316 = next(v for v in verses if (v.chapter, v.verse) == (3, 16))
print(v316.text[:26], v316.hash == next(h for h in bsb.verse_hashes(43) if (h.chapter, h.verse) == (3, 16)).hash)  # → For God so loved the world True
print(verses[5].heading, verses[5].paragraph, verses[6].paragraph)  # → The Witness of John True False
print(len(logoxide.text_version()) > 0, logoxide.words("“Éphraïm’s” word—so"))  # → True ['ephraims', 'word', 'so']

styled_text writes a passage as the language server shows it: verse labels, a chapter label on its own line where a chapter starts inside the passage, words of Jesus and the like in templates, lines wrapped at width, and cut at limit bytes. Its marks say what each run of it is (a verse number, a heading), by offsets in the OffsetUnit asked for (UTF-16 code units for JavaScript's strings, characters for Python's, or UTF-8 bytes).

from logoxide import BookTextStyle, KindTemplate, NodeKind, OffsetUnit

style = BookTextStyle(
    verse_label="[{verse}]", chapter_label="{name} {chapter}", notes=False, headings=False,
    templates=[KindTemplate(kind=NodeKind.WORDS_OF_JESUS, template="*{text}*")], width=None, limit=None,
)
styled = kjv_john.styled_text(topos.parse("John 3:2-3", BookStyle.NAME), style, "John", OffsetUnit.CHAR)
print(styled.text)  # → [2] The same came to Jesus by night, … [3] Jesus answered and said unto him, *Verily, verily, … kingdom of God.*
runs = lambda kind: [styled.text[m.start : m.end] for m in styled.marks if m.kind == kind]
print(runs(NodeKind.VERSE), runs(NodeKind.WORDS_OF_JESUS))  # → ['[2]', '[3]'] ['*Verily, verily, I say unto thee, Except a man be born again, he cannot see the kingdom of God.*']
print(kjv_john.styled_text(topos.parse("John 2:25-3:1", BookStyle.NAME), dataclasses.replace(style, width=40, limit=120), "John", OffsetUnit.CHAR).text)
# → [25] And needed not that any should
# → testify of man: for he knew what was in
# → man.
# → John 3
# → [1] There was a man of the Phari …

The tree itself, for a host that renders with its own engine: nodes() lists every node in document order, each naming its parent by index, with its kind, tag name, attributes, and text; verse_nodes() says where each verse is in that list (its marker; start, its first node; and end, the first node after it, so nodes[start:end] is the verse: a heading before it belongs to it). A milestone (a USX verse marker, an OSIS sID/eID pair) is MilestoneKind.START or MilestoneKind.END.

from logoxide import MilestoneKind

nodes = bsb.book(43).nodes()
at = next(v for v in bsb.book(43).verse_nodes() if (v.chapter, v.verse) == (3, 16))
marker = nodes[at.marker]
print(marker.kind == NodeKind.VERSE, marker.name, marker.milestone == MilestoneKind.START, marker.attributes[0])  # → True verse True NodeAttribute(name='number', value='16')
in_note = lambda n: n.parent is not None and (nodes[n.parent].kind == NodeKind.NOTE or in_note(nodes[n.parent]))
words = [n for n in nodes[at.start : at.end] if n.kind == NodeKind.TEXT and not in_note(n)]
print("".join(n.text for n in words))  # → For God so loved the world that He gave His one and only Son, that everyone who believes in Him shall not perish but have eternal life.

A book file of your own: usfm_to_usx turns USFM into USX (usx_to_usfm the other way, as normalized USFM), usx_to_usj and usj_to_usx between USX and USJ (USX as JSON), and convert_book from any of USFM, USX, and USJ into another (a BookFormat each); detect_format names a file's format from its name, else from how its text starts (usx, usfm, usj, osis, thml, gbf). LogoxideBook.parse reads usx or osis (an unknown markup is InvalidMarkup, and text with no verses, which isn't a book, Invalid), and LogoxideBook.usx_chapter_verses counts a USX book's verses from its markers alone, fast enough for a whole Bible. LogoxideBook.find_fragment finds a link's words (:~:text=…) in a verse whose words changed a little since, by byte offsets.

import json

from logoxide import BookFormat, LogoxideBook, LogoxideErrorException

usx = logoxide.usfm_to_usx(pathlib.Path("kjv/usfm/3jn.usfm").read_text())
third_john = LogoxideBook.parse(topos, "usx", usx)
print(third_john.book_id(), list(third_john.chapter_verses()), list(LogoxideBook.usx_chapter_verses(usx)))  # → 64 [14] [14]
try:
    LogoxideBook.parse(topos, "rtf", usx)
except LogoxideErrorException as error:
    print(error.error)  # → LogoxideErrorInvalidMarkup(message='unknown markup "rtf" (usx, osis)')
print(logoxide.usx_to_usfm(usx).startswith("\\id 3JN"))  # → True
print(json.loads(logoxide.usx_to_usj(usx))["content"][0]["code"])  # → 3JN
print(logoxide.convert_book(logoxide.usx_to_usj(usx), BookFormat.USJ, BookFormat.USFM).startswith("\\id 3JN"))  # → True
print(LogoxideBook.find_fragment("For God so greatly loved the world", "so loved the world"))  # → TextRange(start=8, end=34)
print(logoxide.detect_format("JHN.xml", usx), logoxide.detect_format("notes.md", "# Notes"))  # → usx None

A Bible of your own as loose files, with no manifest: open_bible finds each file's format (as detect_format does), opens .zips, cuts a whole-Bible OSIS or ThML file into its books, and reads each book from its best format (USX, USFM, USJ, OSIS, ThML, GBF). skipped says what it passed over: files of no format (not-text), a book another file has in a better format (duplicate), peripherals.

asv_zip = pathlib.Path("eng-asv_usfm.zip").read_bytes()
loose = logoxide.open_bible([PackageFile(path="asv.zip", bytes=asv_zip), file("3JN.xml", usx)])
print(list(loose.books()), loose.book_format(64), len(loose.book(43).verse_texts()))  # → [43, 64] usx 2
print([f"{s.reason} {s.path}" for s in loose.skipped()])  # → ['not-text asv.zip/copr.htm', 'peripheral asv.zip/00-FRTeng-asv.usfm', 'duplicate asv.zip/95-3JNeng-asv.usfm']

Commentaries, dictionaries, and cross-references

A commentary's notes on a passage are each of its notes covering any verse of it, as text and HTML (chapter:verse to last_chapter:last); with stretch, a note also covers the verses after it up to the next, so a verse the commentary skips gets the note before it.

commentary_files = [
    file("logoxide.toml", """[package]
id = "notes"
name = "Short Notes"
abbreviation = "SN"
version = "1.0.0"
language = "en"
license = "CC0-1.0"
kind = "commentary"

[formats]
osis = { path = "osis/" }
"""),
    file("osis/John.xml", """<osis><osisText osisIDWork="SN"><div type="book" osisID="John"><chapter osisID="John.3" sID="John.3"/>\
<verse osisID="John.3.16" sID="John.3.16" n="16"/>God so loved: the <hi type="italic">world</hi>.<verse eID="John.3.16"/>\
<verse osisID="John.3.17 John.3.18" sID="John.3.17" n="17-18"/>Not to condemn.<verse eID="John.3.17"/>\
<chapter eID="John.3"/></div></osisText></osis>"""),
]
commentary = logoxide.open_package(commentary_files, features, [])
for note in commentary.notes(john, options, False):
    print(f"{note.chapter}:{note.verse}-{note.last}", note.text)
# → 3:16-16 God so loved: the world.
# → 3:17-18 Not to condemn.
print(commentary.notes(john, options, False)[0].html)  # → <span class="topos-verse" data-osis-id="John.3.16" data-n="16"></span>God so loved: the <i class="topos-hi topos-italic">world</i>.
after = topos.parse("John 3:19", BookStyle.NAME)
print(len(commentary.notes(after, options, False)), commentary.notes(after, options, True)[0].text)  # → 0 Not to condemn.

A dictionary's entry for a key (G0025, g25, and G25 are one key; αγαπαω finds ἀγαπάω), None for a key it lacks; keys() lists them. info().keys says what it's keyed by.

dictionary = logoxide.open_package([
    file("logoxide.toml", """[package]
id = "strongs"
name = "Strong's Greek"
abbreviation = "STR"
version = "1.0.0"
language = "grc"
license = "CC0-1.0"
kind = "lexicon"
keys = "strongs-greek"

[formats]
dictionary = { path = "entries/" }
"""),
    file("entries/greek.xml", """<entries keys="strongs-greek" markup="tei">
<entry key="G0025" headword="ἀγαπάω"><entryFree><orth>ἀγαπάω</orth> to love</entryFree></entry>
<entry key="G26">love</entry>
</entries>
"""),
], features, [])
print(dictionary.info().keys, dictionary.keys())  # → strongs-greek ['G25', 'G26']
entry = dictionary.entry("g25", options)
print(entry.key, entry.headword, entry.text)  # → G25 ἀγαπάω ἀγαπάω to love
print(dictionary.entry("G9999", options))  # → None

A cross-reference package's cross_references for a verse (OSIS, or a reference topos reads) are its lines covering any verse of it, as OSIS, each with its weight when the package gives one.

cross_references = logoxide.open_package([
    file("logoxide.toml", """[package]
id = "tsk"
name = "Treasury"
abbreviation = "TSK"
version = "1.0.0"
language = "en"
license = "CC0-1.0"
kind = "cross-references"

[formats]
xref = { path = "xref/" }
"""),
    file("xref/gen.tsv", "Gen.1.1\tJohn.1.1-John.1.3\t1\nGen.1.1\tHeb.11.3\t0.8\nGen.1.1-Gen.1.5\tPs.33.6\n"),
], features, [])
for x in cross_references.cross_references("Genesis 1:1"):
    print(x.from_, x.to, x.weight)
# → Gen.1.1 John.1.1-John.1.3 1.0
# → Gen.1.1 Heb.11.3 0.8
# → Gen.1.1-Gen.1.5 Ps.33.6 None
print(cross_references.keys())  # → ['Gen.1.1', 'Gen.1.1-Gen.1.5']

Audio and video

A package lists its recordings by URL, size, and SHA-256 (media()), never storing them; a recording without a text of its own names the dependency whose text it reads (media_text) or the versification it's numbered by (media_versification). timings are where each verse is in a file, from its WebVTT timings (parse_timings reads a timings file that isn't in a package). Download a file when it's played (its url, else each of its mirrors), check it with verify_media (an Integrity error says what differs), and keep it at media_store_path, by its hash, so versions listing the same file share it.

audio = logoxide.open_package([
    file("logoxide.toml", """[package]
id = "bsb-audio"
name = "The Berean Standard Bible, read aloud"
abbreviation = "BSBA"
version = "1.0.0"
language = "en"

[dependencies]
bsb = { git = "https://example.org/bsb", version = "1" }

[media]
text = "bsb"
versification = "KJV"
timings = "timings/"

[[media.files]]
path = "JHN.mp3"
url = "https://example.org/audio/JHN.mp3"
size = 5
sha256 = "sha256:2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824"
book = "JHN"
"""),
    file("timings/JHN.vtt", "WEBVTT\n\nJHN 3:16\n00:04.000 --> 00:09.500\nFor God so loved the world\n"),
], features, [])
[recording] = audio.media()
print(recording.path, recording.size, recording.book, audio.media_text(), audio.media_versification())  # → JHN.mp3 5 JHN bsb KJV
print(audio.timings(recording))  # → [VerseTiming(book='JHN', book_id=43, chapter=3, verse=16, last=16, start_ms=4000, end_ms=9500)]
print(logoxide.parse_timings("WEBVTT\n\nJHN 3:17-18\n00:10.000 --> 00:15.000\n")[0].last)  # → 18
downloaded = b"hello"  # what the host downloaded from recording.url
logoxide.verify_media(recording, downloaded)
try:
    logoxide.verify_media(recording, b"hellO")
except LogoxideErrorException as error:
    print(type(error.error).__name__)  # → LogoxideErrorIntegrity
print(logoxide.media_store_path(recording))  # → media/2c/2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824

Versification maps

versifications() holds no maps; with_map adds a map package's (kind = "versification-map") and with_vrs a .vrs file's text, and map gives where a reference numbered one way is in another, with what of it has no counterpart there. A Bible's numbering is its info().versification; counts are the book's verse counts in the first numbering, for whole chapters (chapter_verses(book); None uses topos's).

mapping = logoxide.open_package([
    file("logoxide.toml", """[package]
id = "english-original"
name = "English to Original"
version = "1.0.0"
kind = "versification-map"
language = "en"
license = "CC0-1.0"
versification = "English"
maps-to = "Original"

[formats]
vrs = { path = "eng-org.vrs" }
"""),
    file("eng-org.vrs", "PSA 51:0 = PSA 51:1\nPSA 51:0 = PSA 51:2\nPSA 51:1-19 = PSA 51:3-21\nMAL 4:1-6 = MAL 3:19-24\n"),
], features, [])
print(mapping.info().kind == PackageKind.VERSIFICATION_MAP, mapping.info().maps_to)  # → True Original
maps = logoxide.versifications().with_map(mapping).with_vrs("Vulgate", "Original", "PSA 50:1-21 = PSA 51:1-21")
print(maps.names())  # → ['English', 'Original', 'Vulgate']
mapped = maps.map("Psalm 51:1-2", "English", "Original", None)
print(mapped.text, mapped.osis, mapped.verses[0])  # → Psalm 51:3-4 Ps.51.3-Ps.51.4 MappedVerse(book=19, chapter=51, verse=3)
print(maps.map("Psalm 51:1", "English", "Vulgate", kjv.chapter_verses(19)).text)  # → Psalm 50:3
missing = maps.map("Malachi 4:1", "Original", "English", None)
print(len(missing.verses), missing.missing_text, missing.missing)  # → 0 Malachi 4:1 [MappedVerse(book=39, chapter=4, verse=1)]

A vault's project

A vault's logoxide.toml and logoxide.lock say which packages it uses (parse_project, either None for a new vault). Changing a project gives a new one, its manifest_text() (comments kept, dependencies in the order added) to save, and installing it gives the lock file; dependencies() are in name order. A dependency is git (with at most one of version, tag, branch, or rev), registry, path, or import_.

from logoxide import Dependency

dependency = Dependency(
    git=None, registry=None, path=None, import_=None, version=None, tag=None, branch=None, rev=None,
    features=[], default_features=True, package_id=None, abbreviation=None, aliases=None,
)
project = logoxide.parse_project("""# My vault
[dependencies]
kjv = { git = "https://example.org/kjv", version = "1" }

[settings]
default = "kjv"
compare = ["bsb"]
catalogs = ["https://example.org/catalog.toml"]
""", None)
project = project.add_dependency("bsb", dataclasses.replace(dependency, git="https://example.org/bsb", version="1"))
print(project.dependencies(), project.unlocked(), project.packages())  # → ['bsb', 'kjv'] ['bsb', 'kjv'] []
print(project.default_bible(), project.compare())  # → kjv ['bsb']
print(len(project.catalogs()), project.catalogs()[0], project.is_default_catalog(project.catalogs()[1]))  # → 6 https://example.org/catalog.toml True
print(len(project.remove_catalog(logoxide.ebible_catalog_url()).catalogs()))  # → 5
print(len(project.add_catalog("https://example.org/more.toml").catalogs()))  # → 7
named = project.add_registry("example", "https://example.org/catalog.toml")
print(named.registries(), named.remove_registry("example").registries())  # → [Registry(name='example', location='https://example.org/catalog.toml')] []
print(project.remove_dependency("bsb").dependencies())  # → ['kjv']
pathlib.Path("vault").mkdir(exist_ok=True)
pathlib.Path("vault/logoxide.toml").write_text(project.manifest_text())
print(project.manifest_text())
# → # My vault
# → [dependencies]
# → kjv = { git = "https://example.org/kjv", version = "1" }
# → bsb = { git = "https://example.org/bsb", version = "1" }
# → [settings]
# → default = "kjv"
# → compare = ["bsb"]
# → catalogs = ["https://example.org/catalog.toml"]

Catalogs

A catalog lists packages to find by name or language: a catalog.toml anyone hosts, or a publisher's own list (eBible.org's translations.csv, fetch.bible's manifest.json, CCEL's page, a SWORD repository's mods.d.tar.gz, which is read as bytes). parse_catalog reads one's text and parse_catalog_bytes its bytes, by its location; catalogs merge, so one search covers them all. Each entry's source says how to get it: a git repository (CatalogSourceGit), a registry of release archives (CatalogSourceRegistry, its releases), or another publisher's download to import (CatalogSourceExternal).

examples = logoxide.parse_catalog("https://example.org/catalog.toml", get("https://example.org/catalog.toml").decode())
ebible = logoxide.parse_catalog_bytes(logoxide.ebible_catalog_url(), get(logoxide.ebible_catalog_url()))
print(examples.name(), examples.count(), examples.warnings(), ebible.name(), ebible.count())  # → Example Bibles 3 [] eBible.org 2
both = examples.merge(ebible)
for found in both.search("english bible", 0):
    print(found.id, found.abbreviation, type(found.source).__name__)
# → ebible-eng-asv ASV CatalogSourceExternal
# → ebible-engwebp WEBP CatalogSourceExternal
sermons = both.search("sermons", 1)[0]
print(sermons.kind == PackageKind.BOOK, sermons.source, sermons.releases[0].version, sermons.releases[0].size)  # → True CatalogSourceRegistry(catalog='https://example.org/catalog.toml') 1.0.0 2504
print([(group.work, group.name, len(group.entries)) for group in both.search_grouped("king james", 0)])  # → [('kjv', 'King James Version', 1)]
[kjv_work] = both.works("king james")
print(kjv_work.id, kjv_work.from_, [e.id for e in kjv_work.entries])  # → kjv WorkFromLogoxide() ['kjv']

An installer reads a project's catalogs (its [settings] catalogs, then the default ones) as lx search does, asking for each as a Download. A download the host can't make is a warning (provide_download_failed), unless none can be read; catalog() is then all of them merged. The same answer serves installing, below. A catalog given by path isn't read there (a warning says so): read it and merge parse_catalog_bytes's.

from logoxide import (
    FetchRequestArchive, FetchRequestDownload, FetchRequestFiles, FetchRequestImportEntry,
    FetchRequestRefs, LogoxideCatalog, LogoxideInstaller, PackageLocation, PackageLocationLibrary,
    CatalogsStepFetch,
)


def folder(location: PackageLocation) -> str:
    """Where a package's files are: in the library, or a path package's in the vault"""
    return str(pathlib.Path("library" if isinstance(location, PackageLocationLibrary) else "vault", location.dir))


def answer(installer: LogoxideInstaller, request, catalog: LogoxideCatalog | None = None) -> None:
    """Answers what an installer asks for"""
    match request:
        case FetchRequestRefs(repository=repository, url=url):
            installer.provide_refs(repository, get(url))
        case FetchRequestArchive(repository=repository, commit=commit, urls=urls):
            installer.provide_archive(repository, commit, get_first(urls))
        case FetchRequestDownload(url=url, urls=urls):
            try:
                installer.provide_download(url, get_first(urls))
            except OSError as error:
                installer.provide_download_failed(url, str(error))
        case FetchRequestImportEntry(import_=name):
            entry = catalog.find_import(name) if catalog else None
            if entry is None:
                raise LookupError(f"no catalog lists {name}")
            installer.provide_import_entry(entry)
        case FetchRequestFiles(location=location):
            if pathlib.Path(folder(location)).is_dir():
                installer.provide_files(location, files(folder(location)))
            else:
                installer.provide_missing(location)


installer = logoxide.installer()  # keep one: what it's given is kept
loading = installer.load_catalogs(project)
while isinstance(loading, CatalogsStepFetch):
    answer(installer, loading.request)
    loading = installer.load_catalogs(project)
print(len(loading.warnings), loading.warnings[0])  # → 4 …fetch.bible…404…
catalog = installer.catalog()
print(catalog.count(), len(catalog.warnings()), catalog.find_import("ebible:eng-asv").name)  # → 5 4 American Standard Version (1901)

What logoxide knows of catalogs, imports, and categories: the default catalogs, eBible.org's list, whether a URL is fetched without encryption, a SWORD repository CrossWire lists, the publishers imports come from, and the categories a package can be in (at most max_categories()).

print(len(logoxide.default_catalog_urls()), logoxide.ebible_catalog_url())  # → 5 https://ebible.org/Scriptures/translations.csv
print(logoxide.is_insecure_url("http://ftp.xiphos.org/"), logoxide.is_insecure_url("https://example.org/kjv"))  # → True False
print(logoxide.sword_catalog_url("crosswire"), logoxide.sword_catalog_url("nowhere"))  # → https://crosswire.org/ftpmirror/pub/sword/raw/mods.d.tar.gz None
print(logoxide.import_prefixes())  # → ['ebible', 'fetch', 'ccel', 'sword', 'door43', 'burrito', 'dbl']
print(logoxide.categories()[0], logoxide.max_categories())  # → Category(name='biblical-theology', description='Themes traced through Scripture (covenant, kingdom)') 5

Installing

There's no git in WebAssembly, so the host downloads over HTTP and logoxide asks for what it needs: a repository's refs (git's smart HTTP, which GitHub, GitLab, Gitea, Forgejo, and Bitbucket serve), an archive of a commit (.tar.gz, from the first of urls that answers: the forms hosts use), a registry's downloads, or the files of a package already installed. It resolves exactly as lx install does, so a vault's logoxide.lock works with lx and back. inspect reads a repository's newest release (or a tag, branch, or rev) before it's added, to name it by its id.

from logoxide import InspectStepFetch

inspecting = installer.inspect("https://example.org/bsb", None, None, None)
while isinstance(inspecting, InspectStepFetch):
    answer(installer, inspecting.request)
    inspecting = installer.inspect("https://example.org/bsb", None, None, None)
found, tag, commit = inspecting.inspected.info, inspecting.inspected.tag, inspecting.inspected.commit
print(found.id, found.version, tag, len(commit))  # → bsb 1.0.3 v1.0.3 40

install(project, update) resolves the project's dependencies, keeping its lock file's versions (UpdateModeNothing()), or moving all of them (UpdateModeAll()) or some (UpdateModeOnly(ids)) to the newest their requirements allow, and says what it needs until it's done: then write files into the library and lock as the vault's logoxide.lock. A book from a registry is added by the registry's catalog.

from logoxide import InstallStepFetch, LogoxideProject, UpdateMode, UpdateModeAll, UpdateModeNothing, UpdateModeOnly


def install(installer: LogoxideInstaller, project: LogoxideProject, update: UpdateMode, catalog: LogoxideCatalog | None = None) -> list[str]:
    """Installs a project, answering what's asked; what was asked, by kind"""
    asked = []
    step = installer.install(project, update)
    while isinstance(step, InstallStepFetch):
        asked.append(type(step.request).__name__.removeprefix("FetchRequest"))
        answer(installer, step.request, catalog)
        step = installer.install(project, update)
    write("library", step.files)
    pathlib.Path("vault/logoxide.toml").write_text(project.manifest_text())
    pathlib.Path("vault/logoxide.lock").write_text(step.lock)
    return asked


project = project.add_dependency("sermons-on-love", dataclasses.replace(dependency, registry="https://example.org/catalog.toml", version="1"))
print(install(installer, project, UpdateModeNothing()))  # → ['Files', 'Refs', 'Archive', 'Files', 'Files', 'Download']
project = logoxide.parse_project(pathlib.Path("vault/logoxide.toml").read_text(), pathlib.Path("vault/logoxide.lock").read_text())
print(install(logoxide.installer(), project, UpdateModeAll()))  # → ['Refs', 'Archive', 'Files', 'Refs', 'Archive', 'Files', 'Download', 'Files']
print(install(logoxide.installer(), project, UpdateModeOnly(ids=["bsb"])))  # → ['Refs', 'Archive', 'Files', 'Files', 'Files']

A package is what its manifest declares, so files a reader adds beside it (annotations) don't change its checksum, but changing its own files does: installed versions are checked against the lock file's checksum, and UpdateModeRepair() hands back the files of any whose own files changed, to write over them.

from logoxide import UpdateModeRepair

kjv_entry = next(e for e in project.packages() if e.name == "kjv")
pathlib.Path(folder(kjv_entry.location), "usfm/jhn.usfm").write_text("changed")
try:
    install(logoxide.installer(), project, UpdateModeNothing())
except LogoxideErrorException as error:
    print(type(error.error).__name__)  # → LogoxideErrorIntegrity
print(install(logoxide.installer(), project, UpdateModeRepair()))  # → ['Files', 'Files', 'Archive', 'Files']
print(logoxide.open_package(files(folder(kjv_entry.location)), kjv_entry.features, cache).checksum() == kjv_entry.checksum)  # → True

The vault's packages

packages() lists each package the manifest asks for, as the lock file has it, followed by what it brings in (dependency_of), with where its files are: in the library, or a path package's folder in the vault. Open each from there with its features. A dependency turned off (set_enabled) stays locked and installed, and is enabled: false for hosts to leave out.

for entry in project.packages():
    print(entry.name, entry.version, entry.location, entry.git, entry.enabled)
# → bsb 1.0.3 PackageLocationLibrary(dir='packages/bsb/1.0.3+g…') https://example.org/bsb True
# → kjv 1.0.3 PackageLocationLibrary(dir='packages/kjv/1.0.3+g…') https://example.org/kjv True
# → sermons-on-love 1.0.0 PackageLocationLibrary(dir='packages/sermons-on-love/1.0.0+a…') None True
off = project.set_enabled("bsb", False)
print(off.is_enabled("bsb"), off.enabled_dependencies(), off.disabled_dependencies(), off.packages()[0].enabled)  # → False ['kjv', 'sermons-on-love'] ['bsb'] False

The lock file doesn't say what a package is, so commentaries, dictionaries, and cross_references take each entry's logoxide.toml from the host (PackageManifest, by the entry's name) and give those in use of that kind. name_clashes says which names two or more of the Bibles a host shows answer to, so a reference naming one can't say which.

from logoxide import NamedBible, PackageManifest

write("vault/notes", commentary_files)
project = project.add_dependency("notes", dataclasses.replace(dependency, path="notes"))
install(installer, project, UpdateModeNothing())
project = logoxide.parse_project(pathlib.Path("vault/logoxide.toml").read_text(), pathlib.Path("vault/logoxide.lock").read_text())
manifests = [PackageManifest(name=e.name, manifest=pathlib.Path(folder(e.location), "logoxide.toml").read_text()) for e in project.packages()]
print([(e.name, e.location) for e in project.commentaries(manifests)], project.dictionaries(manifests), project.cross_references(manifests))  # → [('notes', PackageLocationPath(dir='notes'))] [] []
print(project.name_clashes([
    NamedBible(label="kjv 1.0.3", abbreviation="KJV", aliases=["AV"]),
    NamedBible(label="my-av", abbreviation="AV", aliases=[]),
]))  # → [NameClash(name='AV', packages=['kjv 1.0.3', 'my-av'])]

Imports

Another publisher's download is made into a package the same way for every publisher: import_downloads(entry) says what to fetch (each from the first of its urls that answers), and import_downloaded(entry, files) makes the package's files from their bytes, by name. Write them into a folder in the vault and add it as a path package, or add a Dependency with import_ and let installing make it (kept in the library by its checksum, as lx import does). book_downloads(entry) says which books an import downloads a file of its own for (fetch.bible's USX, a book each), so a reader can fetch only the books it shows and parse each with LogoxideBook.parse; it's empty for an import downloaded whole (eBible.org's .zip).

from logoxide import CatalogSourceExternal

asv = catalog.find_import("ebible:eng-asv")
assert isinstance(asv.source, CatalogSourceExternal)
print(asv.source.import_, asv.source.tracks_versions, asv.version)  # → ebible:eng-asv True 2026-10-08
downloads = logoxide.import_downloads(asv)
print(downloads)  # → [ImportDownload(name='zip', urls=['https://ebible.org/Scriptures/eng-asv_usfm.zip'])]
fetched = [PackageFile(path=d.name, bytes=get_first(d.urls)) for d in downloads]
imported = logoxide.import_downloaded(asv, fetched)
print([f.path for f in imported])  # → ['copr.htm', 'logoxide.toml', 'usfm/00-FRTeng-asv.usfm', 'usfm/73-JHNeng-asv.usfm', 'usfm/95-3JNeng-asv.usfm']
asv_package = logoxide.open_package(imported, features, [])
print(asv_package.info().abbreviation, list(asv_package.books()), asv_package.skipped()[0].reason)  # → ASV [43, 64] peripheral

A Scripture Burrito or a Digital Bible Library text release bundle someone hands over has no catalog: import_bundle_entry(location, files) makes its entry from its bytes (a .zip or .tar.gz as bundle, or a folder's files as files/<path>), named as its metadata says, and import_downloaded makes it like any import. location is where it came from (a URL, or a file:// one), which its import name keeps.

bundle = [PackageFile(path="bundle", bytes=pathlib.Path("vandyke-burrito.zip").read_bytes())]
vandyke = logoxide.import_bundle_entry("https://example.org/vandyke-burrito.zip", bundle)
assert isinstance(vandyke.source, CatalogSourceExternal)
print(vandyke.id, vandyke.name, vandyke.source.import_)  # → vdat Van Dyck Test Version burrito:https://example.org/vandyke-burrito.zip
print([f.path for f in logoxide.import_downloaded(vandyke, bundle)])  # → ['COPYRIGHT.txt', 'logoxide.toml', 'usfm/TIT.usfm']

Updating an import: find_upstream finds its entry by info().upstream; import_is_current says whether the publisher's version there is the one imported (eBible.org's and SWORD's versions follow their updates; fetch.bible's and CCEL's don't, so those are always False); else download it again and update_import(current, fresh) gives what changed (None for nothing), the version raised by what changed, the files to write, and those to delete.

from logoxide import ChangeKind

upstream = asv_package.info().upstream
print(upstream.import_, upstream.version, catalog.find_upstream(upstream).id, logoxide.import_is_current(asv, upstream.version))  # → ebible:eng-asv 2026-10-08 ebible-eng-asv True
print(logoxide.update_import(imported, imported))  # → None
# What a new download makes: here, John 3:16 reworded and 3 John dropped
fresh = [
    file(f.path, f.bytes.decode().replace("so loved", "so much loved")) if "JHN" in f.path else f
    for f in imported
    if "3JN" not in f.path
]
update = logoxide.update_import(imported, fresh)
print(update.from_, update.to, update.change == ChangeKind.MAJOR, update.description, update.removed)  # → 1.0.0 2.0.0 True 1 verse removed, 1 verse reworded ['usfm/95-3JNeng-asv.usfm']

An import dependency is made as it installs: the installer asks for its catalog entry (ImportEntry, answered from the catalogs with provide_import_entry) unless load_catalogs listed it, then for its downloads. Named other than its id (ebible-eng-asv), a dependency says which package it is with package_id.

project = project.add_dependency("asv", dataclasses.replace(dependency, import_="ebible:eng-asv", package_id="ebible-eng-asv"))
print(install(logoxide.installer(), project, UpdateModeNothing(), catalog))  # → ['ImportEntry', 'Download', 'Files', 'Files', 'Files', 'Files', 'Files']
project = logoxide.parse_project(pathlib.Path("vault/logoxide.toml").read_text(), pathlib.Path("vault/logoxide.lock").read_text())
asv_entry = next(e for e in project.packages() if e.name == "asv")
print(asv_entry.import_, asv_entry.version, asv_entry.location.dir)  # → ebible:eng-asv 1.0.0 packages/ebible-eng-asv/1.0.0+i…

Making packages

A package can be made from one file the person picks: an EPUB fills in its own title, subtitle, authors, language, and publisher (file_metadata, to fill a form); a Bible's USX or USFM needs a name and a language. What's given wins over the file; the id comes from the name, a Bible's or commentary's abbreviation from its initials (the other kinds have one only when given), and the version is 0.1.0 unless given. logoxide gives back the package's files to write into a new folder (then a git repository, to release it). license_concerns says what to know before redistributing one (shown, never enforced).

from logoxide import ConcernKind, NewPackage

epub = pathlib.Path("sermons.epub").read_bytes()
print(logoxide.file_metadata("sermons.epub", epub))  # → FileMetadata(title='Sermons on Love', subtitle='Two Short Sermons', authors=['A. Preacher'], language='en', publisher='Example Press', rights='Public domain')
given = NewPackage(
    id=None, name=None, abbreviation=None, authors=[], description=None, version="1.0.0", kind=None,
    language=None, license=None, publisher=None, homepage=None, repository=None, keywords=[],
    categories=[], about_books=["1 John"],  # a book's code or name
)
made = logoxide.new_package("sermons.epub", epub, given)
print([f.path for f in made])  # → ['epub/sermons.epub', 'logoxide.toml']
book = logoxide.open_package(made, features, [])
print(book.info().id, book.info().kind == PackageKind.BOOK, book.info().about_books, book.epub_files())  # → sermons-on-love True ['1JN'] ['epub/sermons.epub']
[concern] = book.license_concerns()
print(concern.kind == ConcernKind.NO_LICENSE, concern.serious)  # → True False

A book has no verse text: references() reads its EPUBs for the Bible references in them, each a passage in OSIS (topos.parse reads it), the file, its chapter, a cfi EPUB readers open, and its UTF-16 range, line, column, spine item, and paragraph (what ToposIndex.set_epub takes). parse_references reads a release's published logoxide.references.bin without its books: the passages cited, and how often.

first = book.references()[0]
print(first.passage, first.chapter, first.cfi, first.line, first.column, first.spine_index)  # → John.3.16 Love epubcfi(/6/2!/4/4,/1:28,/1:37) 3 29 0
print(topos.parse(first.passage, BookStyle.NAME).reference)  # → John 3:16
print(logoxide.parse_references(pathlib.Path("logoxide.references.bin").read_bytes()))  # → [CitedPassage(passage='John.3.16', count=2), …]

unzip gives a .zip's files (folders left out), so a zip of USFM or USX books someone picks is read without a zip library: pass a book's bytes to new_package. Bytes that aren't a zip, or more than 200,000 entries or 1 GiB, are an error. A collection is a package of packages, its manifest alone (new_collection); add its packages with add_dependency on it.

zipped = logoxide.unzip(pathlib.Path("eng-asv_usfm.zip").read_bytes())
print([f.path for f in zipped])  # → ['00-FRTeng-asv.usfm', '73-JHNeng-asv.usfm', '95-3JNeng-asv.usfm', 'copr.htm']
gospel = next(f for f in zipped if "JHN" in f.path)
mine = logoxide.new_package(gospel.path, gospel.bytes, dataclasses.replace(given, name="My Gospel", language="en", license="CC0-1.0", about_books=[]))
print([f.path for f in mine], logoxide.open_package(mine, features, []).info().abbreviation)  # → ['logoxide.toml', 'usfm/73-JHNeng-asv.usfm'] MG
collection = logoxide.new_collection(dataclasses.replace(given, name="My Library", about_books=[]))
print('kind = "collection"' in collection[0].bytes.decode())  # → True

Reading natively

In this package (and in Swift and Kotlin), read_passage and compare_passage read a project's Bibles from its folder and the library's, as lx read and lx compare do: a passage in the Bible the reference names (else the project's first), and a passage in several Bibles, each in its own numbering through the project's versification maps, with what of it one lacks.

plain = BookTextStyle(verse_label="[{verse}]", chapter_label=None, notes=False, headings=False, templates=[], width=None, limit=None)
read = logoxide.read_passage("vault", "library", "cache", "John 3:16 KJV", plain)
print(read.bible, read.abbreviation, read.passage.reference, read.versification)  # → kjv KJV John 3:16 KJV
print(read.text.text)  # → [16] ¶ For God so loved the world, …
cut = dataclasses.replace(plain, limit=12)
for c in logoxide.compare_passage("vault", "library", "cache", "John 3:16", ["KJV", "BSB", "ASV"], cut):
    print(c.abbreviation, c.reference, c.text.text, c.mapped, repr(c.missing))
# → KJV John 3:16 [16] ¶ For … False ''
# → BSB John 3:16 [16] For God … False ''
# → ASV John 3:16 [16] For God … False ''

Errors

Errors are raised as LogoxideErrorException (error.error is a LogoxideErrorNotFound, LogoxideErrorIo, and so on, with its message). The kind is the core's kind: NotFound, Io, Invalid, InvalidMarkup, Network, Git, Archive, Integrity, Resolve, Busy, and Conflict.

from logoxide import LogoxideErrorNotFound

try:
    logoxide.open_package([file("notes.md", "")], features, [])
except LogoxideErrorException as error:
    print(isinstance(error.error, LogoxideErrorNotFound), error.error.message)  # → True logoxide.toml: not found

Every export

Each of logoxide's classes, functions, records, and enums (each enum with data a class for each of its variants, like PackageLocationLibrary), and where an example uses it. The rest (Topos, ToposTranslations, ToposQuery, ToposFormat, ToposOptions, ToposIndex, ToposFiles, Passage, and their records and enums) are topos's: see References, with topos.

Source and building

This package is built from logoxide's repository, github.com/Logoxide/logoxide, where its source, its issues, and the checks of these examples are (crates/logoxide-ffi; its README says how to build the package and run them).