testiq 0.2.0

TestIQ - Intelligent test analysis for finding duplicate and redundant tests using coverage analysis

TestIQ - Intelligent Test Analysis

TestIQ v0.2.0 - An enterprise-grade Python library that analyzes test coverage to identify duplicate, redundant, and inefficient tests. Save time, reduce CI costs, and improve test suite quality with intelligent analysis.

✨ What's New in v0.2.0

• 🎯 Enhanced Code Quality - Reduced cognitive complexity, better maintainability
• 📝 Constants Management - String literals extracted to constants (best practices)
• 🔧 Improved Type Support - Type stubs for better IDE autocomplete (click, PyYAML)
• ✅ Better Testing - Float comparison fixes, proper unused variable handling
• 🧹 Clean Repository - No sample/dummy data, all reports generated by testiq
• 📦 pytest Included - Now a core dependency (no separate install needed)

---

🚀 Key Features

• 🎯 Duplicate Detection - Find exact duplicates, subsets, and similar tests
• 📊 Quality Scoring - A-F grading with actionable recommendations (improved in v0.2.0)
• 🚦 CI/CD Integration - Quality gates, baselines, and trend tracking
• ⚡ Performance - Parallel processing and caching for large test suites
• 🔌 Extensible - Plugin system with custom hooks
• 📝 Multiple Formats - HTML, CSV, and Markdown reports (all formats in sync)
• 🧪 pytest Integration - Built-in plugin for per-test coverage tracking (now included!)

---

📦 Installation

pip install testiq

---

🚀 How to Use TestIQ

Quick Start

See it in action immediately:

# Run demo with sample data
testiq demo

Generate per-test coverage data:

TestIQ needs per-test coverage (which lines each test executes). Use our pytest plugin:

# Run tests with TestIQ plugin (easiest method - recommended)
pytest --testiq-output=testiq_coverage.json

# Or specify your test directory
pytest tests/ --testiq-output=testiq_coverage.json

# Then analyze with TestIQ
testiq analyze testiq_coverage.json --format html --output report.html

Alternative methods:

# Method 2: Use pytest coverage contexts (requires pytest-cov)
pytest --cov=src --cov-context=test --cov-report=json
python -m testiq.coverage_converter coverage.json --with-contexts -o testiq_coverage.json

# Method 3: Convert standard pytest coverage (limited - aggregated only)
pytest --cov=src --cov-report=json
python -m testiq.coverage_converter coverage.json -o testiq_coverage.json

Manual sample data:

# Create sample coverage data (for testing)
cat > testiq_coverage.json << 'EOF'
{
  "test_login_success": {
    "auth.py": [10, 11, 12, 15, 20, 25],
    "user.py": [5, 6, 7, 8]
  },
  "test_login_failure": {
    "auth.py": [10, 11, 12, 15, 16],
    "user.py": [5, 6]
  }
}
EOF

# Analyze the sample data
testiq analyze testiq_coverage.json

📖 See Pytest Integration Guide for complete setup instructions

CLI Usage

Global Options

# Check version
testiq --version

# Use custom config file
testiq --config my-config.yaml analyze testiq_coverage.json

# Set log level
testiq --log-level DEBUG analyze testiq_coverage.json

# Save logs to file
testiq --log-file testiq.log analyze testiq_coverage.json

# Run demo with sample data
testiq demo

1. Analyze Your Tests

# Basic analysis with terminal output (text format)
testiq analyze testiq_coverage.json

# With custom similarity threshold (default: 0.3)
testiq analyze testiq_coverage.json --threshold 0.8

# Generate beautiful HTML report
testiq analyze testiq_coverage.json --format html --output reports/report.html

# CSV export for spreadsheet analysis
testiq analyze testiq_coverage.json --format csv --output reports/results.csv

# JSON output for automation
testiq analyze testiq_coverage.json --format json --output reports/results.json

# Markdown format
testiq analyze testiq_coverage.json --format markdown --output reports/results.md

# Get quality score and recommendations
testiq quality-score testiq_coverage.json --output reports/quality.txt

2. CI/CD Integration

# Quality gates (exit code 2 if failed, 1 if duplicates found, 0 if success)
testiq analyze testiq_coverage.json --quality-gate --max-duplicates 5

# Save baseline for future comparisons
testiq analyze testiq_coverage.json --save-baseline my-baseline

# Compare against baseline (fail if quality worsens)
testiq analyze testiq_coverage.json --quality-gate --baseline my-baseline

# Combined: quality gate with threshold and baseline
testiq analyze testiq_coverage.json \
  --quality-gate \
  --max-duplicates 5 \
  --threshold 0.8 \
  --baseline production \
  --format html \
  --output reports/ci-report.html

# Manage baselines
testiq baseline list                    # List all saved baselines
testiq baseline show my-baseline        # Show baseline details
testiq baseline delete old-baseline     # Delete a baseline

3. Python API

from testiq.analyzer import CoverageDuplicateFinder
import json

# Create analyzer with performance options
finder = CoverageDuplicateFinder(
    enable_parallel=True,
    max_workers=4,
    enable_caching=True
)

# Load coverage data
with open('coverage.json') as f:
    coverage_data = json.load(f)

# Add test coverage
for test_name, test_coverage in coverage_data.items():
    finder.add_test_coverage(test_name, test_coverage)

# Find issues
exact_duplicates = finder.find_exact_duplicates()
subset_duplicates = finder.find_subset_duplicates()
similar_tests = finder.find_similar_coverage(threshold=0.8)

# Generate reports
from testiq.reporting import HTMLReportGenerator
html_gen = HTMLReportGenerator(finder)
html_gen.generate(Path("reports/report.html"), threshold=0.8)

# Quality analysis
from testiq.analysis import QualityAnalyzer
analyzer = QualityAnalyzer(finder)
score = analyzer.calculate_score(threshold=0.8)
print(f"Quality Score: {score.overall_score}/100 (Grade: {score.grade})")

4. Examples & Testing

See complete working examples:

• 📁 Python API Examples - Complete demonstration of all features
• 📁 Bash Examples - Quick CLI testing scripts
• 📁 CI/CD Examples - Jenkins & GitHub Actions integration
• 📖 Manual Testing Guide - Comprehensive testing guide

CI/CD Integration:

• Jenkinsfile Example - Complete Jenkins pipeline with quality gates
• GitHub Actions Example - Full workflow with error handling

---

Next Steps

1. Try the demo: testiq demo
2. Analyze your tests: testiq analyze coverage.json
3. Run examples: python examples/python/manual_test.py
4. Integrate with CI/CD: See examples/cicd/

---

� Understanding TestIQ Results

Quality Score Components

TestIQ calculates a comprehensive quality score (0-100) with letter grade (A+ to F):

• Duplication Score (40%) - Based on exact duplicates found
• Coverage Efficiency (30%) - Tests with broad vs. narrow coverage
• Uniqueness Score (30%) - Based on similar/subset tests

Important Notes:

• Coverage Efficiency requires source metadata - Use pytest --cov with --cov-report=json to generate .coverage file with source line info
• Without source metadata, efficiency score is 0 - This lowers overall grade but doesn't indicate actual problems
• Run make test-complete or sequential coverage+TestIQ analysis for full scoring

Interpreting Duplicate Detection

TestIQ uses coverage-based duplicate detection - it identifies tests that execute the same code paths. This can include:

True Duplicates (Should Review):

• ✅ Multiple tests with identical coverage and same purpose
• ✅ Copy-pasted tests with minor naming differences
• ✅ Tests that add no unique code coverage value

False Positives (Expected Behavior):

• ⚠️ Tests with same coverage but different assertions/logic
• ⚠️ Tests that exercise different input values (same code path)
• ⚠️ Tests focused on behavior verification vs. code coverage

Example: Two tests that both create a dataclass instance will show identical coverage (import paths), but one might test validation while another tests defaults - both are valuable!

Recommendations:

1. Review high-priority duplicates first - These have highest impact
2. Check test intent, not just coverage - Different assertions mean different value
3. Focus on exact duplicates - Subset/similar tests may be intentional
4. Use quality score as a guide - Not an absolute metric
5. Combine with test execution time - Slow duplicate tests are higher priority

Running Complete Analysis

For best results, run coverage and duplicate detection separately:

# Option 1: Use make target (recommended)
make test-complete

# Option 2: Run manually
pytest --cov=testiq --cov-report=term --cov-report=html  # Coverage first
pytest --testiq-output=testiq_coverage.json -q            # TestIQ second
testiq analyze testiq_coverage.json --format html --output reports/duplicates.html

# Option 3: Use provided script
./run_complete_analysis.sh

Why separate runs? Python's sys.settrace() allows only ONE active tracer. Running both together causes conflicts (19% coverage vs 81% separate).

---

💡 Why TestIQ?

• ✅ Reduce Test Count - Remove 10-30% of redundant tests
• ✅ Faster CI/CD - Shorter test execution times
• ✅ Lower Costs - Reduced CI resource usage
• ✅ Better Quality - Focus on valuable, unique tests
• ✅ Smart Analysis - Coverage-based duplicate detection with quality scoring

---

📚 Documentation

Essential Documentation

Document	Description
Interpreting Results	Guide to understanding TestIQ scores and recommendations
CHANGELOG	Version history and v0.2.0 release notes
Pytest Integration	Generate per-test coverage data
CLI Reference	Command-line interface documentation
API Reference	Python API documentation
Contributing	How to contribute
Security Policy	Security and vulnerability reporting

Additional Resources (examples/)

• Python Examples - Complete API demonstrations
• CI/CD Examples - GitHub Actions, GitLab CI, Jenkins
• Sample Data - examples/sample_coverage.jsonribute | | Changelog | Version history |

---

🤝 Contributing

We welcome contributions! See CONTRIBUTING.md for guidelines.

---

📝 License

MIT License - see LICENSE for details.

---

🔒 Security

For security vulnerabilities, see SECURITY.md for responsible disclosure.

---

Made with ❤️ by Kiran K Kotari