deltatest-cli 0.4.2

Run only the tests affected by your code changes.

Pintest

Run only the tests affected by your code changes - automatically!

Pintest integrates with your git workflow as a pre-commit hook, running only tests that cover the code you've changed. This dramatically speeds up your development cycle while ensuring quality.

🎯 Key Features

• 🚀 Fast: Run only affected tests, not the entire suite
• 🔒 Safe: Blocks commits if affected tests fail
• 📊 Intelligent: Uses SQLite-based test-to-code mapping for instant lookups
• 🔄 Incremental Coverage: Combines coverage from test runs
• ✨ New Test Detection: Always runs newly added tests
• 🧠 Auto-Discovery: Detects unmapped tests and builds mapping automatically
• 🎣 Pre-commit Hook: Automatic integration with git workflow
• 🐳 Docker Integration: Ensures containers are running before tests
• 🔧 Preflight Checks: Validates database connections (postgres, neo4j)
• 📦 Git LFS Support: Shares pre-built mapping database across team (30-60min savings)
• 🚀 Auto-Push: Automatically pushes commits with updated mapping database

⚡ Quick Start (Pre-Commit Hook)

cd ~/workspace/myproject

# Install the pre-commit hook
~/workspace/all/pintest/scripts/install_pre_commit.sh

# That's it! Now every commit will:
# 1. Find tests affected by your changes
# 2. Run only those tests
# 3. Block commit if tests fail
# 4. Combine coverage with existing data

📋 How It Works

Pre-Commit Flow

Developer commits changes
         ↓
Pre-commit hook triggered
         ↓
Compare staged changes vs development branch
         ↓
Query SQLite mapping: "Which tests cover these lines?"
         ↓
Detect unmapped tests (dry-run pytest --collect-only)
         ↓
Run unmapped tests all-at-once (build mapping)
         ↓
Run mapped affected tests with coverage
         ↓
Tests pass? → Combine coverage → Allow commit ✅
Tests fail? → Block commit ❌

Auto-Discovery of Unmapped Tests

The pre-commit hook automatically detects tests that exist in your codebase but aren't in the mapping database yet:

1. Collect all tests: Runs pytest --collect-only to find all available tests
2. Compare with mapping: Queries .test_mapping.db to see which tests are already mapped
3. Find delta: Identifies tests that exist but have never been run with coverage
4. Run all-at-once: Executes all unmapped tests together with --cov-context=test (fastest)
5. Update mapping: After completion, updates the mapping database with new coverage

Why this matters:

• New tests you write are automatically added to the mapping
• Tests added by teammates get mapped when you first commit
• No need to manually regenerate the entire mapping
• Mapping database grows organically over time
• All tests run in a single pytest invocation (fastest possible)

Example:

$ git commit -m "Fix bug in auth.py"

📝 Found 1 changed Python file
📊 Mapping DB: 1234 tests, 567 files, 45678 mappings
🔍 Collected 1250 total tests from pytest
⚠️  Found 16 unmapped test(s)

================================================================================
Building coverage mapping for 16 unmapped test(s)
================================================================================

  Running unmapped test: unit_tests/test_new_feature.py::test_case_1
    ✓ Mapping updated for: unit_tests/test_new_feature.py::test_case_1
  Running unmapped test: unit_tests/test_new_feature.py::test_case_2
    ✓ Mapping updated for: unit_tests/test_new_feature.py::test_case_2
  ...
✓ Successfully mapped 16 test(s)

================================================================================
Running 3 affected test(s)...
================================================================================
...

Mapping Database

The mapping is stored in .test_mapping.db (SQLite) at your repo root:

CREATE TABLE test_coverage (
    test_name TEXT,
    file_path TEXT,
    line_number INTEGER,
    PRIMARY KEY (test_name, file_path, line_number)
);

Fast lookups: "Which tests cover file X, line Y?"

🔧 Installation

Prerequisites

• Python 3.8+
• pytest >= 7.0
• pytest-cov >= 4.0
• Git repository

Step 1: Install Package

cd ~/workspace/all/pintest
pip install -e .

Step 2: Generate Initial Coverage

cd ~/workspace/myproject

# Run all tests with coverage context enabled
pytest --cov=. --cov-context=test --cov-report=

This creates .coverage file with test-to-code mappings.

Step 3: Create Mapping Database

Choose one of two methods:

Option A: Iterative (Recommended for large test suites )

# Build mapping by running tests one-by-one (resumable)
pintest build-mapping --repo-root ~/workspace/myproject --verbose

# If interrupted, resume with:
pintest build-mapping --repo-root ~/workspace/myproject --resume --verbose

Benefits:

• ✅ Resumable after interruption or failure
• ✅ Progress saved every 10 tests
• ✅ Safe for 20,000+ test suites
• ✅ Continues even if some tests fail

Time: ~2-4 hours , but can be interrupted and resumed anytime.

Option B: Batch (Faster, but not resumable)

# Run all tests with coverage at once
pytest --cov=. --cov-context=test --cov-report=

# Build mapping from coverage
pintest update-mapping --repo-root ~/workspace/myproject --verbose

Benefits:

• ✅ Faster (45-90 minutes )
• ✅ Single command

Drawbacks:

• ❌ Not resumable if interrupted
• ❌ Stops on first failure
• ❌ Must regenerate from scratch

📖 See ITERATIVE_SETUP_GUIDE.md for detailed guide

Option C: Use Pre-built Database (Git LFS)

If your repo uses Git LFS to store .test_mapping.db, you can skip building:

# Install Git LFS (one-time)
brew install git-lfs  # macOS
# or: sudo apt-get install git-lfs  # Linux

# Initialize in repo
cd ~/workspace/myproject
git lfs install

# Pull the pre-built database
git lfs pull

Benefits:

• ✅ Instant - no rebuild needed
• ✅ Shared across team
• ✅ Always available after clone/pull

See GIT_LFS_SETUP.md in your repo.

Step 4: Install Pre-Commit Hook

cd ~/workspace/myproject
~/workspace/all/pintest/scripts/install_pre_commit.sh

Done! Now every commit will run only affected tests.

📖 Usage

Pre-Commit Hook (Automatic)

Just commit normally:

git add src/my_module.py
git commit -m "Fix bug in authentication"

# Hook runs automatically:
# - Finds tests covering src/my_module.py
# - Runs only those tests
# - Blocks commit if tests fail
# - Combines coverage on success

Manual Test Running

# Show what tests would run
pintest run --repo-root ~/workspace/myproject --dry-run --verbose

# Run affected tests manually
pintest run --repo-root ~/workspace/myproject

# Compare against different branch
pintest run --repo-root ~/workspace/myproject --base-branch develop

# Pass pytest arguments
pintest run --repo-root ~/workspace/myproject -- -x --pdb

Update Mapping

Update the mapping database when you add new tests or change coverage significantly:

Regular updates (batch):

# Run tests with coverage
pytest --cov=. --cov-context=test --cov-report=

# Update mapping database
pintest update-mapping --repo-root ~/workspace/myproject --verbose

After adding many tests (iterative):

# Resume where you left off (only runs unmapped tests)
pintest build-mapping --repo-root ~/workspace/myproject --resume --verbose

When to update:

• After adding new test files
• After significant refactoring
• Weekly (recommended for active projects)
• When mapping feels "stale" (tests not being selected correctly)

🎛️ Commands

pintest run

Run affected tests based on changes.

pintest run [OPTIONS] [-- PYTEST_ARGS]

Options:
  --repo-root PATH        Repository root (default: current directory)
  --base-branch BRANCH    Branch to compare against (default: master)
  --coverage-file PATH    Path to .coverage file
  --dry-run              Show tests without running
  --min-tests N          Minimum tests required
  -v, --verbose          Detailed output

pintest update-mapping

Update test mapping database from coverage data.

pintest update-mapping [OPTIONS]

Options:
  --repo-root PATH        Repository root (default: current directory)
  --coverage-file PATH    Source .coverage file
  --mapping-db PATH       Target mapping database path
  -v, --verbose          Detailed output

🔍 How Test Selection Works

1. Changed Files Detection

git diff --cached development...HEAD

Identifies:

• Which Python files changed
• Which lines were added/modified
• Which files are test files

2. Test Mapping Lookup

SELECT DISTINCT test_name
FROM test_coverage
WHERE file_path = 'src/auth.py'
AND line_number IN (42, 43, 44)

Fast O(1) lookup using SQLite indexes.

3. Test Categories

Category	Always Run?	Example
New tests	✅ Yes	test_new_feature.py (added in commit)
Modified tests	✅ Yes	test_auth.py (changed in commit)
Coverage-based	✅ Yes	Tests that cover changed lines
Unaffected	❌ No	Tests not covering any changes

4. Coverage Combining

# Existing coverage from previous runs
.coverage (before)

# New coverage from affected tests
.coverage.new

# Combined result
.coverage (after) = .coverage (before) + .coverage.new

Uses coverage combine to merge data safely.

💡 Examples

Example 1: Bug Fix

# You fix a bug in src/auth.py line 42
vim src/auth.py

git add src/auth.py
git commit -m "Fix login timeout bug"

# Hook runs:
# 📝 Found 1 changed Python file
# 📊 Mapping DB: 1234 tests, 567 files
# 📍 src/auth.py: 3 test(s) cover changed lines
#     - unit_tests/test_auth.py::test_login_timeout
#     - unit_tests/test_auth.py::test_login_retry
#     - integration_tests/test_full_auth_flow.py::test_complete_flow
#
# Running 3 affected test(s)...
# ✅ All tests passed. Commit allowed.

Example 2: New Test Added

# You add a new test file
vim unit_tests/test_new_feature.py

git add unit_tests/test_new_feature.py
git commit -m "Add tests for new feature"

# Hook runs:
# ✨ New test files (always run): 1
#   + unit_tests/test_new_feature.py
#
# Running 1 test file...
# ✅ All tests passed. Commit allowed.

Example 3: Refactoring

# You refactor a utility function used everywhere
vim src/utils/helpers.py

git add src/utils/helpers.py
git commit -m "Refactor helper function"

# Hook runs:
# 📍 src/utils/helpers.py: 47 test(s) cover changed lines
#
# Running 47 affected test(s)...
# (Only runs 47, not all 1234 tests!)

🚫 Bypassing the Hook

For urgent commits or when hook is broken:

git commit --no-verify -m "Urgent hotfix"

🔧 Configuration

Change Base Branch

Edit .git/hooks/pre-commit:

python3 -m pintest.pre_commit_hook \
    --repo-root "$REPO_ROOT" \
    --base-branch main \  # Changed from development
    --verbose

Customize Coverage Options

The hook runs pytest with:

pytest --cov=. --cov-context=test --cov-append --cov-report=

To customize, edit pintest/pre_commit_hook.py.

📊 Statistics

Typical time savings:

Project Size	Total Tests	Avg Affected	Time Before	Time After	Savings
Small (500 tests)	500	~15	2 min	10 sec	92%
Medium (2000 tests)	2000	~30	8 min	30 sec	94%
Large (5000 tests)	5000	~50	20 min	1 min	95%

Actual savings depend on test distribution and change scope.

🐛 Troubleshooting

"Mapping database not found"

pintest update-mapping --repo-root ~/workspace/myproject

"No tests selected"

• Check if .coverage file exists and is recent
• Update mapping: pintest update-mapping
• Run with --verbose to see what's happening

"Tests that should run aren't selected"

Regenerate coverage and mapping:

pytest --cov=. --cov-context=test --cov-report=
pintest update-mapping --verbose

"Hook runs all tests"

Check that mapping database exists:

ls -lh .test_mapping.db

If missing, generate it with update-mapping.

📁 Files Created

• .test_mapping.db - SQLite database with test-to-code mappings (gitignored)
• .coverage - pytest coverage data (gitignored)
• .git/hooks/pre-commit - Pre-commit hook script

🤝 Contributing

Pintest is an open-source developer productivity tool.