Setrya
Two colleagues ask an internal AI assistant about a private policy. You revoke one user’s access and clear their cached answer. The assistant now gives that user only public information. Then a background job that started before the revocation puts the private answer back in the cache.
Setrya is an open-source Python SDK and CLI for testing that race before you ship. It records what the worker did and what your app served each user. A teammate can replay the result after the app shuts down.
Test your own endpoint · Record a pytest test · Test tenant offboarding
Test access revocation with two users
A Celery task reads synthetic passages through LangChain and pgvector. Setrya pauses it before it writes to the cache. The app revokes access and evicts the user’s entry. Setrya asks the same question through the app’s authenticated HTTP endpoint as each user, first while the task is paused and again after it finishes.
| What the endpoint served | User who lost access | Colleague who kept access |
|---|---|---|
| While the task was paused, in both runs | Public answer only | Private answer from cache |
| After the broken worker wrote its answer · FAIL | Private answer from the resumed worker’s cache entry | Private answer from cache |
| After the guarded worker rejected the write · PASS | Public answer only | Private answer from cache |
Replay both published runs in a Python 3.11+ virtual environment. The bundle records the task’s write outcome and whether each response contained private material. Replay needs no running services. Setrya on PyPI
python -m pip install setrya
curl -fL -o late-cache-fill.zip https://github.com/davidahma/setrya/releases/download/v0.12.1/late-cache-fill.zip
unzip -q late-cache-fill.zip -d late-cache-fill
setrya replay late-cache-fill/broken # FAIL, exit 1
setrya replay late-cache-fill/guarded # PASS, exit 0
To run the same drill live, start the bundled app with a disposable pgvector database. It builds answers without a model API key. Run the Celery and pgvector example
Test your own endpoint
From your application checkout, generate a pytest test for your authenticated answer route.
python -m pip install 'setrya[pytest]'
setrya init endpoint
python -m pytest -q -s tests/test_setrya_endpoint.py
The first run reports NOT TESTED and names the first hook to connect, seed. Fill in the generated file with two test users, a question, your endpoint call, and your worker controls. Each run prints a command to replay the evidence after teardown. Connect the hooks and interpret the result
If your app already uses Celery and LangChain, install setrya[celery-langchain] and run setrya init endpoint --framework celery-langchain in your checkout. This starter uses Redis to coordinate the worker and controller across processes or containers. Connect a Celery worker
Run the refund-policy demo
Your team shortens the refund window from 30 days to 14. The support agent searches and gets the new policy. Then a worker resumes with the copy it read yesterday. The offline demo runs that race against a local SQLite FTS5 index.
python -m pip install setrya
setrya demo
| Verdict | What the support agent sees | What the evidence shows |
|---|---|---|
| FAIL | 30 days → 14 days → 30 days | The old worker committed. The next search returned the old policy at rank 1. |
| PASS | 30 days → 14 days | The worker rejected its stale write. Final searches found the 14-day policy and the unrelated delivery policy. |
| NOT TESTED | 30 days → 14 days | The old worker committed and search still returned 14 days. The response didn’t establish that the index had seen the write. |
Use the Python SDK in your tests
A drill names the caller, route, query, and passages that must appear or disappear. Run the bundled refund drill from Python with the core package.
from importlib.resources import as_file, files
from setrya.runner import run
drill = files("setrya.examples").joinpath("support-search", "drill.toml")
with as_file(drill) as path:
result = run(path)
print(result.verdict.status.label) # PASS
For your own application, an adapter supplies the worker checkpoint, write receipt, and search results. If pytest already drives your worker, install setrya[pytest] and use the setrya_record fixture to save those observations alongside your assertions. A PASS covers the callers, queries, and schedule you tested. Adapter guide · Record an existing pytest test · Verdict rules
Review an indexing change in a pull request
Run your test on the base and candidate commits. From your application checkout, with both refs available and the pytest extra installed, use your own drill path and pytest test ID.
setrya run-pr --repo . --base origin/main --candidate HEAD \
--drill tests/refund.toml --output refund-review \
-- python -m pytest -p setrya.pytest_plugin -q tests/test_index.py::test_refund_update
In the refund example, the base returns the forbidden 30-day policy at rank 1. On the candidate, the worker rejects the write and the same caller gets 14 days. The reviewer sees FIX VERIFIED with both observations. Keep the output as a CI artifact even when the test fails. Review the example evidence · CI recipe
Test tenant offboarding
A customer leaves, but a worker still holds a copy of their support summary. Run setrya offboarding demo --output offboarding-run to test whether the summary reappears and whether a retained tenant’s summary stays readable. The CI gate reports what the app served. A separate deletion receipt tracks one request and can still list open items, such as a backup awaiting expiry. Offboarding guide · Axiom migration
More drills and integrations
- Selective cache invalidation checks that a refund update leaves an unrelated delivery answer cached. Cache answers by source in memory · Share answers across processes with Redis · Answer dependencies · Cache metrics
- Worker schedules cover retries, duplicate deliveries, and reordered writes. OpenDesk’s two-user revocation trial follows a real summary job across a team membership change.
- Store evidence explains how a search can establish that it saw a particular write. Route coverage identifies changed read paths that lack matching evidence.
Metadata
Release files for setrya 0.14.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| setrya-0.14.0.tar.gz | 240.8 kB | With provenance | Details for setrya-0.14.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| setrya-0.14.0-py3-none-any.whl | Python 3 | none | any | Details for setrya-0.14.0-py3-none-any.whl |
Total release size: 443.1 kB
Release files / setrya-0.14.0.tar.gz
| Download URL | setrya-0.14.0.tar.gz |
|---|---|
| Size | 240.8 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
391aacea51e9fd6ebf2e06a279dfc1c2cf209c4dd098805fdb47a81f8f1aedda
|
|
BLAKE2b-256 checksum How to use checksums |
d2ae3e725548c746490f5a81df19c29929ca87a31dd05f6c61ca06803345467c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 20, 2026.
Transparency logRelease files / setrya-0.14.0-py3-none-any.whl
| Download URL | setrya-0.14.0-py3-none-any.whl |
|---|---|
| Size | 202.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
2cae97b9128d020abfd166b6b962d6aa2d26f0eee40cc46a2c73fa34553ca232
|
|
BLAKE2b-256 checksum How to use checksums |
43f818af4e7eff6dc02bf4e52f2e6f66b77e9f085b620cc46bcf9678f2583d87
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 20, 2026.
Transparency logSecurity for setrya 0.14.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for setrya-0.14.0.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 20, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.