pytest-adversarial

LLM-powered adversarial test generation for Python using Quality-Diversity algorithms.

Installation

pip install pytest-adversarial

Quick Example

Test a JSON parser for edge cases:

# target.py
import json

def parse_json(text: str) -> dict:
    """Parse JSON string."""
    return json.loads(text)

Run adversarial testing:

pytest-adversarial examples/json_parser/target.py --rounds 10

The tool will automatically:

• Generate adversarial test inputs using LLMs
• Find edge cases that crash your code
• Evolve attacks using Quality-Diversity search
• Report robustness metrics and discovered vulnerabilities

Sample output:

Round 5/10 | Robustness: 73.2% | Attacks: 42 | Types: 8
Found: malformed_unicode, extreme_nesting, special_chars...

Documentation

For detailed usage, examples, and advanced features, visit: github.com/nulone/pytest-adversarial

Requirements

• Python 3.10+
• OpenAI API key (set OPENROUTER_API_KEY environment variable)

License

MIT License - see GitHub repository for details.