redraven 0.1.2

Python SDK for Redraven (fireraven.ai) — generate, run-against-client-LLM, and evaluate red-team tests.

redraven (Python SDK)

The official Python SDK for Redraven — the LLM red-teaming and policy-evaluation platform from fireraven.ai.

Redraven generates adversarial prompts, evaluates your LLM's responses against your safety policies and certifications, and surfaces failures so you can harden your AI products.

This SDK lets you drive the full Redraven flow from your own Python code:

1. Generate (or pick) a test dataset on Redraven.
2. Run each test case locally through your own LLM — your model API key never leaves your process.
3. Submit responses back to Redraven for scoring; receive a summary you can act on.

The SDK talks only to public endpoints on the Redraven backend, authenticated with your organization API key (issued from the Redraven app).

Platforms

• Marketing site: fireraven.ai
• FireRaven app (production): app.fireraven.ai
• Redraven app (red-teaming console): app.redraven.fireraven.ai

Installation

Install from PyPI:

uv add redraven
# or
pip install redraven

uv is preferred for reproducible installs.

Quick start

import asyncio
import redraven

async def my_llm(prompt: str) -> str:
    # Call your own LLM here. Your key stays in your process.
    return f"echo: {prompt}"

async def main():
    async with redraven.Client() as client:  # reads REDRAVEN_API_KEY + REDRAVEN_BASE_URL
        result = await client.run_test(
            test_id="<your-existing-test-id>",
            llm=my_llm,
            concurrency=4,
        )
        print(result.model_dump())

asyncio.run(main())

Core methods

1. generate_test(generate_kwargs, wait_for_dataset=False, ...) -> str (returns test_id)
2. wait_for_dataset_ready(test_id, ...) -> None
3. run_test(test_id, llm, ...) -> EvalSummary
4. generate_and_run_test(generate_kwargs, llm, ...) -> EvalSummary

Documentation

• Public Python SDK docs: doc.fireraven.ai/redraven-sdk/python
• Redraven SDK overview: doc.fireraven.ai/redraven-sdk/overview

For private contributor documentation, use the repository internal docs.

License

Proprietary — © fireraven.ai. Contact us if you'd like to integrate Redraven into your product.