pytest-agents

A pytest plugin framework with AI agent capabilities for project management, research, and code indexing.

Quick Start

# Install from PyPI
pip install pytest-agents

# Verify installation
pytest-agents verify

Features

• Pytest Plugin: Extended pytest with custom markers and AI agent integration
• PM Agent: TypeScript-based project management agent for task tracking and planning
• Research Agent: AI-powered research and documentation analysis
• Index Agent: Code indexing and intelligent search capabilities
• Prometheus Metrics: Comprehensive observability with metrics collection and HTTP endpoint
• Dependency Injection: Full DI implementation across Python and TypeScript components
• Skills System: Extensible runtime skills for specialized tasks

Quick Start

Installation

From PyPI (Recommended):

pip install pytest-agents

From Docker:

docker pull ghcr.io/kmcallorum/pytest-agents:latest
docker run ghcr.io/kmcallorum/pytest-agents:latest pytest-agents verify

From Source:

# Clone repository
git clone https://github.com/kmcallorum/pytest-agents.git
cd pytest-agents

# Install with uv
make install

# Or manually
uv pip install -e ".[dev]"

Verify Installation

make verify

Run Tests

# All tests
make test

# Python only
make test-python

# TypeScript only
make test-ts

Project Structure

pytest-agents/
├── src/pytest_agents/   # Python pytest plugin package
├── tests/               # Python tests
├── pm/                  # TypeScript PM agent
├── research/            # TypeScript Research agent
├── index/               # TypeScript Index agent
├── skills/              # Runtime skills
├── commands/            # Command documentation
└── docs/                # Documentation

Usage

Using Custom Pytest Markers

import pytest

@pytest.mark.unit
def test_basic_functionality():
    assert True

@pytest.mark.integration
@pytest.mark.agent_pm
def test_with_pm_agent(pytest_agents_agent):
    result = pytest_agents_agent.invoke('pm', 'analyze_project')
    assert result['status'] == 'success'

Parallel Agent Execution

Run multiple agents concurrently for faster test execution:

def test_multi_agent_parallel(agent_coordinator):
    """Run multiple agents in parallel."""
    results = agent_coordinator.run_parallel([
        ('pm', 'track_tasks', {'path': './src'}),
        ('research', 'analyze_document', {'path': 'README.md'}),
        ('index', 'index_repository', {'path': './src'})
    ])

    assert all(r['status'] == 'success' for r in results)

Invoking Agents

# Via Python API
from pytest_agents.agent_bridge import AgentBridge

bridge = AgentBridge()
result = bridge.invoke_agent('pm', 'track_tasks', {'path': './src'})

# Via CLI
pytest-agents agent pm --action track_tasks --path ./src

Metrics and Observability

# Start Prometheus metrics server
pytest-agents metrics

# Custom port
pytest-agents metrics --port 8080

# Configure via environment
export PYTEST_AGENTS_METRICS_ENABLED=true
export PYTEST_AGENTS_METRICS_PORT=9090

View metrics at http://localhost:9090/metrics. See Metrics Documentation for Prometheus and Grafana integration.

Development

Code Quality

# Format code
make format

# Lint code
make lint

Health Check

make doctor

Docker Support

pytest-agents is fully containerized for easy deployment and development.

Quick Start with Docker

# Build and run verification
docker-compose up pytest-agents

# Run tests in Docker
docker-compose --profile test up pytest-agents-test

# Start development shell
docker-compose --profile dev run pytest-agents-dev

See Docker Documentation for complete deployment guide.

Security

pytest-agents implements enterprise-grade security practices:

Automated Security Scanning

• CodeQL: Static analysis detecting 400+ security vulnerabilities in Python and TypeScript
• Snyk Security: Continuous vulnerability scanning for dependencies and containers
• Dependency Scanning: Automated vulnerability detection via Dependabot
• Container Scanning: Docker image vulnerability assessment
• Code Quality: Ruff linting with security-focused rules

Security Features

• Multi-stage Docker builds with minimal attack surface
• Dependency pinning for reproducible builds
• Comprehensive test coverage (61%, 230 tests)
• Automated security updates grouped by severity

Setup and Configuration

New to security scanning? See Security Setup Guide for step-by-step instructions to activate Snyk and Dependabot.

Reporting Vulnerabilities

Please report security vulnerabilities privately via GitHub Security Advisories.

See SECURITY.md for complete security policy and disclosure guidelines.

Documentation

See docs/ directory for detailed documentation:

• Metrics Guide - Prometheus metrics and observability
• Performance Benchmarks - Performance baselines and optimization
• Release Process - Automated releases and versioning
• PyPI Publishing Setup - Configure PyPI trusted publishing
• Security Setup Guide - Activate security scanning
• Docker Guide - Container deployment and development
• Developer Guide - Development workflow
• Architecture Overview - System design
• Python API Reference - Python API documentation
• TypeScript API Reference - TypeScript API documentation

License

MIT

Author

Kevin McAllorum

Security scanning now enabled!