superred-target-chatbot
A general chatbot target for the superred red-teaming framework: it wraps any LLM reachable through litellm as a single-turn or multi-turn chatbot under test, exposing its system prompt, user message, response, and model identity as controllables/observables.
Install
pip install superred-target-chatbot
# for AWS Bedrock providers:
pip install "superred-target-chatbot[bedrock]"
Usage
from chatbot_target import ChatbotTarget, USER_TAG
target = ChatbotTarget(model="gpt-4o-mini", api_base="https://api.openai.com/v1", api_key="sk-...")
Optional keyword arguments bound the provider call: request_timeout (default
120s, passed to every call because litellm's own default is 600-6000s),
max_attempts / retry_backoff_s / retry_max_elapsed_s (bounded retries, for
transient failures only), and max_consecutive_invalid (how many unusable
injected messages in a row end the run).
Turn outcomes
Every victim turn ends in exactly one recorded outcome. A turn that produced no
victim text leaves last_response empty and appends nothing to the conversation
sent to the provider: the target never invents a reply, and never puts an empty
assistant turn on the wire (providers either delete it, silently merging the
attacker's surrounding messages, or reject it outright).
| outcome | what happened | trace |
|---|---|---|
ok |
the victim replied with text | -- |
empty |
the victim replied with a blank string | response_empty observable |
blocked |
a provider content filter suppressed the reply | response_blocked observable, blocked_turns, last_block_reason |
no_content |
the provider returned no content field at all, after retries | response_missing observable |
invalid_injection |
the injected user message was None, not a string, or blank, so nothing was sent |
invalid_user_message observable |
Anything else -- a transient failure that survives the retries, or a persistent
one (auth, context window, malformed request) -- propagates and errors the task,
because those are infrastructure failures and must stay loud. A retried attempt
emits a response_retry observable at the system tag, so a turn that cost
three provider calls is never invisible.
Because the target does not fabricate, last_response alone cannot tell these
apart: empty, blocked, no_content and invalid_injection all leave it
empty. Use the last_turn_outcome, turn_outcomes and turns_completed
queries. In particular a task should abstain rather than score 0.0 when
last_turn_outcome == "no_content" or turns_completed == "0": those runs
contain no victim reply to judge.
Credits / upstream
Original work of the superred project, released under the MIT License (see
LICENSE). It contains no third-party code and no bundled datasets. It reaches
models through litellm (MIT) and, optionally, AWS Bedrock via boto3 (Apache-2.0),
both ordinary pip dependencies.
License
MIT
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file superred_target_chatbot-0.2.0.tar.gz.
File metadata
- Download URL: superred_target_chatbot-0.2.0.tar.gz
- Upload date:
- Size: 28.6 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
349168567748e1ffff8d52aeb956d3ad4195e2f9cd28ebaa4e0181641b3c21fc
|
|
| MD5 |
fe6388f0269678328ce0a47c2c7b92e1
|
|
| BLAKE2b-256 |
b857e710615e92be6e00dfb77d520094bc9496b82eabaf35a42b459757da15be
|
Provenance
The following attestation bundles were made for superred_target_chatbot-0.2.0.tar.gz:
Publisher:
release.yml on RoldSI/superred-modules
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
superred_target_chatbot-0.2.0.tar.gz -
Subject digest:
349168567748e1ffff8d52aeb956d3ad4195e2f9cd28ebaa4e0181641b3c21fc - Sigstore transparency entry: 2586505945
- Sigstore integration time:
-
Permalink:
RoldSI/superred-modules@7ede3307ff67a2ff60f487c699934bc223ab3640 -
Branch / Tag:
refs/tags/superred-target-chatbot-v0.2.0 - Owner: https://github.com/RoldSI
-
Access:
private
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
release.yml@7ede3307ff67a2ff60f487c699934bc223ab3640 -
Trigger Event:
push
-
Statement type:
File details
Details for the file superred_target_chatbot-0.2.0-py3-none-any.whl.
File metadata
- Download URL: superred_target_chatbot-0.2.0-py3-none-any.whl
- Upload date:
- Size: 15.2 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
461a59bced58b8b49d6a79e08cffbd0126069eaba59285ef57735ee219988aa1
|
|
| MD5 |
0b45a0a19272943d235a53c6e57c711b
|
|
| BLAKE2b-256 |
a2ffd6feb76d578cc3930c20a6793df1315bc6004b06646c4bf120a60300cc0c
|
Provenance
The following attestation bundles were made for superred_target_chatbot-0.2.0-py3-none-any.whl:
Publisher:
release.yml on RoldSI/superred-modules
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
superred_target_chatbot-0.2.0-py3-none-any.whl -
Subject digest:
461a59bced58b8b49d6a79e08cffbd0126069eaba59285ef57735ee219988aa1 - Sigstore transparency entry: 2586506335
- Sigstore integration time:
-
Permalink:
RoldSI/superred-modules@7ede3307ff67a2ff60f487c699934bc223ab3640 -
Branch / Tag:
refs/tags/superred-target-chatbot-v0.2.0 - Owner: https://github.com/RoldSI
-
Access:
private
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
release.yml@7ede3307ff67a2ff60f487c699934bc223ab3640 -
Trigger Event:
push
-
Statement type:
