redraven 0.1.5

Python SDK for Redraven (fireraven.ai) — The AI agents redteaming platform.

redraven (Python SDK)

The official Python SDK for Redraven — the LLM red-teaming and policy-evaluation platform from fireraven.ai.

Redraven generates adversarial prompts, evaluates your LLM's responses against your safety policies and certifications, and surfaces failures so you can harden your AI products.

This SDK lets you drive the full Redraven flow from your own Python code:

1. Generate (or pick) a test dataset on Redraven.
2. Run each test case locally through your own LLM — your model API key never leaves your process.
3. Submit responses back to Redraven for scoring; receive a summary you can act on.

The SDK talks only to public endpoints on the Redraven backend, authenticated with your organization API key (issued from the Redraven app).

Platforms

• Marketing site: fireraven.ai
• FireRaven app (production): app.fireraven.ai
• Redraven app (red-teaming console): app.redraven.fireraven.ai

Installation

Install from PyPI:

uv add redraven
# or
pip install redraven

uv is preferred for reproducible installs.

Quick start

import asyncio
import redraven

async def my_llm(prompt: str) -> str:
    # Call your own LLM here. Your key stays in your process.
    return f"echo: {prompt}"

async def main():
    async with redraven.Client() as client:  # reads REDRAVEN_API_KEY + REDRAVEN_BASE_URL
        handshake = await client.call_agent(
            test_id="<your-existing-test-id>",
            llm=my_llm,
            concurrency=4,
        )
        await client.wait_for_evaluation_ready(test_id="<your-existing-test-id>")
        result = await client.get_eval_summary(
            test_id="<your-existing-test-id>",
            expected_cases=handshake.expected_cases,
        )
        print(result.model_dump())

asyncio.run(main())

Core methods

Sequence: dataset ready (generate_test / wait_for_dataset_ready) → call_agent (must finish; submits all case responses) → wait for eval (wait_for_evaluation_ready and/or get_eval_summary(..., wait_for_completion=True)) → get_eval_summary to read the summary if you waited separately. See the user guide for the full table.

Documentation

• Public Python SDK docs: doc.fireraven.ai/redraven-sdk/python
• Redraven SDK overview: doc.fireraven.ai/redraven-sdk/overview

For private contributor documentation, use the repository internal docs.

License

Proprietary — © fireraven.ai. Contact us if you'd like to integrate Redraven into your product.