spec-check 0.1.3

Tools for spec-driven development - validate specifications, test coverage, file structure, and documentation links

spec-check

Tools for spec-driven development - a toolkit for managing and validating project specifications and files.

Features

Lint Tool

The lint tool validates that all files in your repository match patterns in an allowlist. Think of it as the inverse of .gitignore - instead of specifying what to ignore, you specify what files are allowed.

Key features:

• Uses gitignore-style glob patterns for flexible file matching
• Respects .gitignore patterns by default
• Supports complex patterns including character classes (e.g., [0-9])
• Validates that all tracked files match at least one allowlist pattern
• Reports unmatched files for easy identification

Markdown Link Validator

The check-links tool validates hyperlinks in markdown files to ensure documentation stays up-to-date and accessible.

Key features:

• Validates internal links (relative paths) resolve correctly
• Checks anchor links point to existing headings
• Validates external URLs are accessible
• Supports private URL patterns that are skipped during validation
• Concurrent external URL checking for performance
• Respects .gitignore patterns by default

Spec Coverage Validator

The check-coverage tool ensures 100% traceability between specification requirements and tests.

Key features:

• Extracts requirement IDs from spec files (e.g., REQ-001, NFR-001)
• Validates every requirement has at least one corresponding test
• Reports coverage percentage and uncovered requirements
• Uses pytest markers for machine-readable test-to-requirement linking
• Identifies tests without requirement markers

Structure Validator

The check-structure tool enforces consistent spec-to-test structure alignment.

Key features:

• Verifies each spec file has a corresponding test file or directory
• Supports flexible naming conventions (kebab-case to snake_case)
• Allows unit tests without corresponding specs
• Reports specs without tests
• Ensures consistent project organization

Installation

Using uv (recommended)

uv pip install spec-check

Using pip

pip install spec-check

Development installation

# Clone the repository
git clone https://github.com/TradeMe/spec-check.git
cd spec-check

# Install with uv
uv venv
uv pip install -e ".[dev]"

Configuration

spec-check can be configured via pyproject.toml for seamless integration with Python projects. This allows you to set default options without needing to pass command-line arguments every time.

pyproject.toml Configuration

Add a [tool.spec-check] section to your pyproject.toml:

[tool.spec-check.lint]
allowlist = ".specallowlist"
use_gitignore = true

[tool.spec-check.check-links]
config = ".speclinkconfig"
timeout = 15
max_concurrent = 5
check_external = true
use_gitignore = true

[tool.spec-check.check-schema]
config = ".specschemaconfig"
use_gitignore = true

Configuration Options

Lint Command ([tool.spec-check.lint])

• allowlist (string): Path to allowlist file (default: .specallowlist)
• use_gitignore (boolean): Respect .gitignore patterns (default: true)

Check Links Command ([tool.spec-check.check-links])

• config (string): Path to config file for private URLs (default: .speclinkconfig)
• timeout (integer): Timeout for external URL requests in seconds (default: 10)
• max_concurrent (integer): Maximum concurrent external URL requests (default: 10)
• check_external (boolean): Validate external URLs (default: true)
• use_gitignore (boolean): Respect .gitignore patterns (default: true)

Check Schema Command ([tool.spec-check.check-schema])

• config (string): Path to schema config file (default: .specschemaconfig)
• use_gitignore (boolean): Respect .gitignore patterns (default: true)

Configuration Precedence

Configuration values are resolved in the following order (highest to lowest precedence):

1. Command-line arguments (e.g., --timeout 30)
2. pyproject.toml configuration (e.g., timeout = 15 in [tool.spec-check.check-links])
3. Built-in defaults

This means you can set project defaults in pyproject.toml and override them on the command line when needed.

Usage

Lint Command

Create a .specallowlist file in your project root with gitignore-style patterns:

# Documentation
*.md
docs/**/*.rst

# Source code
spec_check/**/*.py
tests/**/*.py

# Configuration files
*.toml
*.yaml
*.yml

# Specs with specific naming convention
specs/research-[0-9][0-9][0-9]-*.md
specs/design-*.md

Then run the linter:

# Lint the current directory
spec-check lint

# Lint a specific directory
spec-check lint /path/to/project

# Use a custom allowlist file
spec-check lint --allowlist .myallowlist

# Don't respect .gitignore patterns
spec-check lint --no-gitignore

# Verbose output
spec-check lint --verbose

Exit Codes

• 0: All files match the allowlist patterns
• 1: Some files don't match or an error occurred

This makes it easy to integrate into CI/CD pipelines:

# .github/workflows/ci.yml
- name: Validate file allowlist
  run: spec-check lint

Check Links Command

Validate all hyperlinks in your markdown documentation:

# Check links in current directory
spec-check check-links

# Check links in a specific directory
spec-check check-links /path/to/docs

# Skip external URL validation (faster)
spec-check check-links --no-external

# Use a custom config file for private URLs
spec-check check-links --config .myconfigfile

# Set timeout for external URLs (default: 10 seconds)
spec-check check-links --timeout 30

# Limit concurrent requests (default: 10)
spec-check check-links --max-concurrent 5

# Verbose output
spec-check check-links --verbose

Private URL Configuration

Create a .speclinkconfig file to specify private URL patterns that should not be validated:

# Private domains (will skip any URL containing these domains)
internal.company.com
localhost

# Private URL prefixes (exact prefix match)
https://private.example.com/
http://localhost:
http://127.0.0.1:

What Gets Validated

• Internal links: [text](./file.md) - checked relative to the markdown file
• Anchor links: [text](#heading) - validated against headings in the file
• Cross-file anchors: [text](./other.md#section) - validates both file and heading
• External URLs: [text](https://example.com) - HTTP request to verify accessibility
• Private URLs: URLs matching configured patterns are skipped

CI/CD Integration

# .github/workflows/ci.yml
- name: Validate documentation links
  run: spec-check check-links --no-external  # Skip external URLs in CI

Check Coverage Command

Ensure all spec requirements have corresponding tests:

# Check coverage in current directory
spec-check check-coverage

# Check coverage in a specific directory
spec-check check-coverage /path/to/project

# Use custom specs and tests directories
spec-check check-coverage --specs-dir my-specs --tests-dir my-tests

Marking Tests with Requirements

Use pytest markers to link tests to requirements:

import pytest

@pytest.mark.req("REQ-001")
def test_inline_link_parsing():
    """Test that inline links are parsed correctly."""
    # Test implementation
    assert True

# For tests covering multiple requirements:
@pytest.mark.req("REQ-002", "REQ-003")
def test_reference_style_links():
    """Test reference-style link parsing."""
    # Test implementation
    assert True

CI/CD Integration

# .github/workflows/ci.yml
- name: Validate spec coverage
  run: spec-check check-coverage

Check Structure Command

Validate spec-to-test structure alignment:

# Check structure in current directory
spec-check check-structure

# Check structure in a specific directory
spec-check check-structure /path/to/project

# Use custom specs and tests directories
spec-check check-structure --specs-dir my-specs --tests-dir my-tests

Structure Conventions

For a spec file specs/feature-name.md, the tool expects either:

• tests/test_feature_name.py (single test file)
• tests/feature_name/ (test directory)

This allows unit tests without corresponding specs while ensuring all specs have requirement tests.

CI/CD Integration

# .github/workflows/ci.yml
- name: Validate spec structure
  run: spec-check check-structure

Pattern Syntax

The allowlist uses gitignore-style glob patterns:

• * - matches any number of characters (except /)
• ** - matches any number of directories
• ? - matches a single character
• [abc] - matches one character in the set
• [0-9] - matches one character in the range
• !pattern - negates a pattern (matches files that don't match the pattern)

Examples

# Match all Python files
*.py

# Match Python files in src directory and subdirectories
src/**/*.py

# Match numbered specification files
specs/spec-[0-9][0-9][0-9].md

# Match files with specific naming pattern
docs/architecture-*.md
docs/design-*.md

# Match multiple file types
*.{yml,yaml}

Use Cases

1. Enforce file organization: Ensure all files follow your project's structure
2. Validate spec compliance: Make sure all files match expected patterns
3. CI/CD validation: Automatically check that no unexpected files are committed
4. Documentation enforcement: Ensure all code files have matching documentation
5. Naming convention enforcement: Validate files follow naming standards
6. Test-to-requirement traceability: Guarantee 100% requirement coverage with automated validation
7. Spec-driven development: Enforce consistent spec-to-test structure for better maintainability
8. Documentation quality: Ensure all links in documentation are valid and up-to-date

Example: Research Paper Repository

# .specallowlist
# Papers must follow naming: research-NNN-title.md
papers/research-[0-9][0-9][0-9]-*.md

# Supporting data files
papers/data-[0-9][0-9][0-9]-*.csv
papers/figures-[0-9][0-9][0-9]-*.{png,jpg,svg}

# Project files
*.md
LICENSE
.gitignore
.specallowlist

Development

Running Tests

# Activate virtual environment
source .venv/bin/activate

# Run tests
pytest tests/ -v

# Run tests with coverage
pytest tests/ -v --cov=spec_check --cov-report=term-missing

Linting

# Check code with ruff
ruff check spec_check/ tests/

# Format code with ruff
ruff format spec_check/ tests/

Building

# Build with flit
uv pip install flit
flit build

Roadmap

Future tools planned for spec-check:

• spec-graph: Visualize dependencies between spec files
• spec-init: Initialize new spec-driven projects
• spec-sync: Keep specs in sync with implementation
• spec-extract: Extract requirements from code comments into spec files

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

MIT License - see LICENSE file for details.