vcti-path-format 1.2.0

File format identification framework with heuristic evaluators and feature validators for Python

Path Format

File format identification framework with heuristic evaluators and feature validators for Python.

Installation

pip install vcti-path-format>=1.2.0

In pyproject.toml dependencies

dependencies = [
    "vcti-path-format>=1.2.0",
]

---

Quick Start

from pathlib import Path
from vcti.pathformat import (
    FormatDescriptor,
    FormatIdentifier,
    FormatRegistry,
    MatchConfidence,
)
from vcti.pathformat.evaluator import HeuristicEvaluator

# Define a format descriptor with validators
hdf5_descriptor = FormatDescriptor(
    id="hdf5-file",
    name="HDF5 File",
    evaluator=(
        HeuristicEvaluator()
        .check_magic_bytes(b"\x89HDF\r\n\x1a\n")  # GATE
        .check_extension([".h5", ".hdf5", ".he5"])  # EVIDENCE
    ),
    attributes={"path_type": "file", "structure": "hdf5"},
)

# Register in a format registry
registry = FormatRegistry()
registry.register(hdf5_descriptor)

# Identify a file
identifier = FormatIdentifier(registry)
results = identifier.identify_file_format(Path("data.h5"))

for result in results:
    print(f"{result.descriptor.name}: {result.confidence.name}")

# Get best match above a confidence threshold
best = identifier.get_best_match(
    Path("data.h5"),
    min_confidence=MatchConfidence.LIKELY,
)

---

Core Concepts

FormatDescriptor

Extends Descriptor[Evaluator] from vcti-plugin-catalog. Wraps an evaluator with format metadata and attributes.

FormatRegistry

Extends Registry[FormatDescriptor]. Central catalog of known formats with attribute-based filtering via registry.lookup.

FormatIdentifier

Evaluates a path against all (or filtered) registered formats and returns results sorted by confidence.

HeuristicEvaluator

Builder-pattern evaluator that aggregates validation evidence:

evaluator = (
    HeuristicEvaluator()
    .check_magic_bytes(b"\x89PNG\r\n\x1a\n")  # GATE
    .check_extension([".png"])                   # EVIDENCE
    .add_validator(custom_validator)             # Custom
)

Heuristic rules:

• Failed GATE -> CERTAINLY_NOT
• All passed + GATE present -> DEFINITE
• All passed + no GATE -> LIKELY
• Some EVIDENCE failed -> UNLIKELY
• No validators -> CANT_EVALUATE

Feature Validators

Validator	Role	Tier	Checks
MagicBytesValidator	GATE	IDENTIFICATION	File signature bytes
ExtensionValidator	EVIDENCE	IDENTIFICATION	File extension

Custom validators implement the FeatureValidator protocol.

---

Validation Tiers

Control evaluation depth with max_tier:

Tier	Cost	Examples
IDENTIFICATION	Cheap	Magic bytes, file extension
STRUCTURE	Medium	Schema validation, header parsing
SEMANTIC	Expensive	Content analysis, business logic

from vcti.pathformat import ValidationTier

# Only run cheap checks
results = identifier.identify_file_format(path, max_tier=ValidationTier.IDENTIFICATION)

---

Custom Validators

Implement the FeatureValidator protocol to add domain-specific checks:

from pathlib import Path
from vcti.pathformat.feature_validator import (
    FeatureValidator,
    ValidationResult,
    ValidationTier,
    ValidatorRole,
)

class HeaderValidator:
    """Checks for a text header line in the first line of a file."""

    id = "header-check"
    description = "Header line validator"
    role = ValidatorRole.EVIDENCE
    tier = ValidationTier.STRUCTURE

    def __init__(self, expected_header: str):
        self.expected_header = expected_header

    def validate(self, path: Path) -> ValidationResult:
        try:
            first_line = path.read_text(encoding="utf-8").split("\n", 1)[0]
            is_passed = first_line.strip() == self.expected_header
        except (OSError, UnicodeDecodeError):
            is_passed = False
        return ValidationResult(
            validator_id=self.id,
            role=self.role,
            is_passed=is_passed,
            details=f"Header {'matches' if is_passed else 'mismatch'}",
        )

# Use with the builder pattern
evaluator = (
    HeuristicEvaluator()
    .check_extension([".csv"])
    .add_validator(HeaderValidator("id,name,value"))
)

---

Evaluator Caching

HeuristicEvaluator includes an LRU cache keyed by (path, max_tier):

# Default: 128 entries
evaluator = HeuristicEvaluator(cache_size=128)

# Disable caching
evaluator = HeuristicEvaluator(cache_size=0)

# Bypass cache for a single call
report = descriptor.evaluate(path, use_cache=False)

# Inspect and manage
info = evaluator.cache_info()  # (hits, misses, maxsize, currsize) or None
evaluator.clear_cache()

Cache entries become stale if file contents change. Call clear_cache() after known file modifications, or pass use_cache=False for one-off re-evaluation.

---

Pre-filtering with Rules

from vcti.lookup import Rule

# Only evaluate formats with structure="hdf5"
results = identifier.identify_file_format(
    path,
    rules=[Rule("structure", "==", "hdf5")],
)

---

Error Handling

The framework raises typed exceptions:

Exception	When
FileNotFoundError	Path does not exist
PathAccessError	Path is not a file or directory, or cannot be read
EvaluatorError	Base class for evaluator errors
ValidationError	A validator raised an unexpected exception
InvalidValidatorError	Invalid validator passed to builder

from vcti.pathformat import PathAccessError

try:
    results = identifier.identify_file_format(path)
except FileNotFoundError:
    print("File not found")
except PathAccessError as e:
    print(f"Cannot access path: {e}")

---

Ecosystem

This package is the identification engine in a three-repo system:

Package	Role
vcti-path-format	Framework: evaluators, validators, registry, identifier
vcti-path-format-attributes	Vocabulary: standardized attribute enums
vcti-path-format-descriptors	Built-in format definitions (HDF5, CAX, etc.)

---

Dependencies

• vcti-plugin-catalog (>=1.0.1) — descriptor/registry framework
• vcti-lookup (>=1.0.1) — attribute-based filtering