Skip to main content

SID SDK

sid-sdk turns search results into compact, model-facing document views. It assigns stable short IDs, selects relevant snippets, tracks the exact character ranges already shown, masks repeated text, and renders SID's <doc> format.

Snippet selection is implemented in Rust with Alyze Unicode UAX #29 tokenization and separate unigram and bigram BM25 scores. Returned ranges are Python character offsets into the original text.

Installation

pip install sid-sdk

Prebuilt ABI3 wheels require no Rust compiler on supported macOS, Linux, and Windows platforms. Building from the source distribution requires Python 3.12+ and Rust 1.88 or newer.

Basic use

from sid import DocumentCache

cache = DocumentCache(language="english")
cache.add_document(
    "database-id",
    {"title": "Example", "content": "The complete document text ..."},
)
view = cache.apply_snippet(
    "database-id",
    snippet_field="content",
    query="complete document",
)
print(view.render_xml())
cache.update_seen(view)

Named analyzers are available for Danish, Dutch, English, Finnish, French, German, Hungarian, Italian, Norwegian, Portuguese, Russian, Spanish, and Swedish. They lowercase tokens and remove Alyze's language-specific stopwords. Use language="generic" for lowercase Unicode tokenization without stopword removal. Chinese, Japanese, and Korean text is handled through generic UAX #29 segmentation; there are no dedicated language entries. apply_snippet accepts a one-call language= override, and forks inherit the cache default.

If every query token is a stopword, all windows have the same score and the selector deliberately returns the beginning of the document.

Character ranges

DocumentCache(range_mode="lenient") is the default. A partially overlapping range is reduced to its exact intersection with the document, while an empty, inverted, malformed, or wholly disjoint range raises InvalidCharacterRange. Use range_mode="strict" to require 0 <= start < end <= len(content) exactly. Ranges are half-open Python character offsets and are never expanded to word boundaries, reversed, or replaced with the complete document.

Call cache.resolve_char_range(data_id, snippet_field, (start, end)) before passing a model-provided range onward. get_single_span_document_view applies the same configured policy, and rendered IDs always describe the exact slice displayed.

Rendering and forks

XML document bodies and attribute values escape &, <, and >; quotes are left unchanged. Falsy attributes are omitted from XML. Markdown tables stringify their values normally, so empty strings make empty cells while 0 and False remain visible. Snippet-less views require explicit display_fields.

Forks share one document store, ID mapping, and ID stream. A document added by any family member is immediately readable by the others, and adding the same data_id again is an idempotent lookup. Each fork keeps its own seen ledger, initialized from the parent's current ledger, so agents can track what they have shown independently.

Concurrent document additions are safe across a fork family, and separate forks may run on separate threads. A single fork is intended for one agent; apply_snippet and update_seen are separate ledger operations and are not an atomic read-modify-write pair when called concurrently on the same fork.

Development

uv sync --group dev
uv run maturin develop --locked
uv run pytest -rxX --strict-markers
cargo test --locked

License

MIT. See LICENSE and THIRD_PARTY_NOTICES.md.

Metadata

Release files for sid-sdk 0.2.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for sid-sdk 0.2.0
File Size Uploaded Provenance Actions
sid_sdk-0.2.0.tar.gz 91.5 kB With provenance Details for sid_sdk-0.2.0.tar.gz

Built distributions (wheels)

Table of built distributions (wheels) for sid-sdk 0.2.0
File Actions
sid_sdk-0.2.0-cp312-abi3-win_amd64.whl CPython 3.12 abi3 Windows x86-64 Details for sid_sdk-0.2.0-cp312-abi3-win_amd64.whl
sid_sdk-0.2.0-cp312-abi3-musllinux_1_2_x86_64.whl CPython 3.12 abi3 Linux musl 1.2+ x86-64 Details for sid_sdk-0.2.0-cp312-abi3-musllinux_1_2_x86_64.whl
sid_sdk-0.2.0-cp312-abi3-musllinux_1_2_aarch64.whl CPython 3.12 abi3 Linux musl 1.2+ ARM64 Details for sid_sdk-0.2.0-cp312-abi3-musllinux_1_2_aarch64.whl
sid_sdk-0.2.0-cp312-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl CPython 3.12 abi3 Linux glibc 2.17+ x86-64 Details for sid_sdk-0.2.0-cp312-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
sid_sdk-0.2.0-cp312-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl CPython 3.12 abi3 Linux glibc 2.17+ ARM64 Details for sid_sdk-0.2.0-cp312-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
sid_sdk-0.2.0-cp312-abi3-macosx_11_0_arm64.whl CPython 3.12 abi3 macOS 11.0+ ARM64 Details for sid_sdk-0.2.0-cp312-abi3-macosx_11_0_arm64.whl
sid_sdk-0.2.0-cp312-abi3-macosx_10_12_x86_64.whl CPython 3.12 abi3 macOS 10.12+ x86-64 Details for sid_sdk-0.2.0-cp312-abi3-macosx_10_12_x86_64.whl

Total release size: 2.9 MB

Release files / sid_sdk-0.2.0.tar.gz

Download URL sid_sdk-0.2.0.tar.gz
Size 91.5 kB
Tags Source
SHA-256 checksum
How to use checksums
77bd55e946c141ce80667fb7d319527d0a535b7fbac0208460feed27c3c6c08d
BLAKE2b-256 checksum
How to use checksums
58c09b9f0a50a82af1d6115f08cb1d984d574e90fa698740d092ce5e96d002e4
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Sep 1, 2026.

Transparency log

Release files / sid_sdk-0.2.0-cp312-abi3-win_amd64.whl

Download URL sid_sdk-0.2.0-cp312-abi3-win_amd64.whl
Size 259.5 kB
Tags CPython 3.12 Windows x86-64 abi3
SHA-256 checksum
How to use checksums
cd0a7f21800aa959b0dad30ef0eacea1a3d77cc5c28351f286bdb6365dacd774
BLAKE2b-256 checksum
How to use checksums
d87ac140341370e8597c9138f5bf8d006d120cc0101c0f2f73f7aae61c4a859f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Sep 1, 2026.

Transparency log

Release files / sid_sdk-0.2.0-cp312-abi3-musllinux_1_2_x86_64.whl

Download URL sid_sdk-0.2.0-cp312-abi3-musllinux_1_2_x86_64.whl
Size 590.7 kB
Tags CPython 3.12 Linux musl 1.2+ x86-64 abi3
SHA-256 checksum
How to use checksums
75403e5deb5a22c9bd4e25b269681fbe2cb1324c5be595159ee58e59629cd0ac
BLAKE2b-256 checksum
How to use checksums
01395d7d3f859096830434eef0b8c1f41c33506787f6d8f6ba08bc137e63530d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Sep 1, 2026.

Transparency log

Release files / sid_sdk-0.2.0-cp312-abi3-musllinux_1_2_aarch64.whl

Download URL sid_sdk-0.2.0-cp312-abi3-musllinux_1_2_aarch64.whl
Size 542.9 kB
Tags CPython 3.12 Linux musl 1.2+ ARM64 abi3
SHA-256 checksum
How to use checksums
e7d884f3c9870689d05d9b04fbd8c6ed5850bc71c04e1e63a8a6e0988d090c35
BLAKE2b-256 checksum
How to use checksums
779667424d0cd5e3db1e8069fc21940e36bd6d721cd0b8a271a440b082a52f67
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Sep 1, 2026.

Transparency log

Release files / sid_sdk-0.2.0-cp312-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

Download URL sid_sdk-0.2.0-cp312-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Size 379.9 kB
Tags CPython 3.12 Linux glibc 2.17+ x86-64 abi3
SHA-256 checksum
How to use checksums
8e949ed6e868afaa1311b6969b342b62ad85aa4e2efea0ab9b1a7344dc0ff851
BLAKE2b-256 checksum
How to use checksums
05fb1c54595b5374df931034bb52ed8aafee4dfe8c98436f7a44c92453a1ff2d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Sep 1, 2026.

Transparency log

Release files / sid_sdk-0.2.0-cp312-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl

Download URL sid_sdk-0.2.0-cp312-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Size 364.9 kB
Tags CPython 3.12 Linux glibc 2.17+ ARM64 abi3
SHA-256 checksum
How to use checksums
d676481863c872760a0ee77d5b86cd5ce56789cc3d8df90ad6c36d66da68ccf6
BLAKE2b-256 checksum
How to use checksums
6af5ca6a3b412ce41dbdd4507530b1d30f0479b03d9e201e82d32c2abcc65856
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Sep 1, 2026.

Transparency log

Release files / sid_sdk-0.2.0-cp312-abi3-macosx_11_0_arm64.whl

Download URL sid_sdk-0.2.0-cp312-abi3-macosx_11_0_arm64.whl
Size 325.9 kB
Tags CPython 3.12 abi3 macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
0a1c9acb68523a92bb9d3bf5912801e2e8dc0e340f7e531171a3c6a0fb51acd1
BLAKE2b-256 checksum
How to use checksums
a955170652816d14a1854a339e9a90f4fabb445aa2af60f10908481a70fe828c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Sep 1, 2026.

Transparency log

Release files / sid_sdk-0.2.0-cp312-abi3-macosx_10_12_x86_64.whl

Download URL sid_sdk-0.2.0-cp312-abi3-macosx_10_12_x86_64.whl
Size 333.2 kB
Tags CPython 3.12 abi3 macOS 10.12+ x86-64
SHA-256 checksum
How to use checksums
6ef40b2928115f5fd3e6a7d851bf609f861dc6d36235856725b592c662d70d12
BLAKE2b-256 checksum
How to use checksums
b7a0406e94c762e725463e7fb7066f1f5bdc7349695f290b24d99c8e65d291da
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Sep 1, 2026.

Transparency log

Security for sid-sdk 0.2.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

All 8 files in this release have been uploaded with provenance metadata.

Select from 8 files

Attestation for sid_sdk-0.2.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Sep 1, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page