chatsee-connector-testing
Shared offline test kit for ChatSee connectors.
A connector hooks one framework and drives chatsee.ChatseeTracker. This package
lets you test all of that with no network, no API key, no framework account and
no dev tenant — so capture bugs are found in seconds, not in a live 2×2 run.
pip install -e python/connectors/_shared
The idea
Don't mock ChatSee. Mock the framework.
Your connector's test supplies a small stub of the framework that replays a
canned turn, instruments it with a FakeTracker, and then asserts what reached
the tracker.
from chatsee_connector_testing import FakeTracker, assert_contract
tracker = FakeTracker()
chatsee = instrument_google_adk(agent_id="a", tenant_id="t", _tracker=tracker)
run_stub_agent(chatsee.plugin)
assert_contract(tracker, expected_turns=1,
expected_tools=["get_order_status"],
framework="google-adk")
Every connector accepts a private _tracker= keyword so tests can inject the
fake. Customers never see it — they pass the normal identity arguments and the
connector builds a real tracker itself.
What assert_contract checks
| Rule | The bug it catches |
|---|---|
One start_turn/end_turn per turn, correctly nested |
Pydantic AI's run_sync delegating to run; Vercel emitting 10 for 4; CrewAI capturing only the first kickoff |
bot_message non-empty |
OpenAI Agents streaming — final text in no span at all |
| Each tool once, with args and a result or an error | Tool captured but its outcome lost |
log_model_call once per model call |
Connectors that sum tokens themselves and destroy the per-call grain |
Absent is never 0 or "" |
An unmeasured turn looking like a free one |
| Required metadata present, tokens/model/tools not in metadata | Tokens buried in metadata are invisible to the analytics screens |
Behavioural checks
These take callables rather than a tracker, because they're about how the connector behaves rather than what it recorded.
assert_fails_open(run_one_turn) # ChatSee must never break the agent
assert_instrument_is_idempotent(run_turns) # instrumenting twice is a no-op
assert_concurrent_runs_isolated(run_one_turn) # state keyed per run, not per instance
assert_concurrent_runs_isolated is the one most likely to fail first: Google
ADK, LangChain and Strands all run concurrent invocations against a single
instrumented object, and the ADK POC plugin holds one shared accumulator.
Running its own tests
The suite has a self-test: a correct toy connector that must satisfy every rule, and a deliberately broken one per rule that must trip it. An assertion nobody has seen fail is not evidence of anything.
pip install -e "python/connectors/_shared[dev]"
pytest python/connectors/_shared/tests -q
Metadata
Release files for chatsee-connector-testing 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| chatsee_connector_testing-0.1.0.tar.gz | 14.3 kB | Without provenance | Details for chatsee_connector_testing-0.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| chatsee_connector_testing-0.1.0-py3-none-any.whl | Python 3 | none | any | Details for chatsee_connector_testing-0.1.0-py3-none-any.whl |
Total release size: 26.1 kB
Release files / chatsee_connector_testing-0.1.0.tar.gz
| Download URL | chatsee_connector_testing-0.1.0.tar.gz |
|---|---|
| Size | 14.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
5a2bd1c34d7ad2de2ad73945bc0230b68fc8c0fe31ecc32fe4aadaf92142728c
|
|
BLAKE2b-256 checksum How to use checksums |
32c6e543ddca0eb29b883e856728ace0bdf27d8052ad626b114ece36589c40d3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.10.11
|
Release files / chatsee_connector_testing-0.1.0-py3-none-any.whl
| Download URL | chatsee_connector_testing-0.1.0-py3-none-any.whl |
|---|---|
| Size | 11.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
ec9a40e06d78cfdaa226f39e7efb1208a51c03fde2e85fc76188625b83846bc2
|
|
BLAKE2b-256 checksum How to use checksums |
9d893dc4456ab0ee5f0f0fac400162b0e742873c5129c24291e216e30f1fde12
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.10.11
|
Security for chatsee-connector-testing 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.