ancpbids 0.4.1

ancpBIDS is a modular Python library for reading, querying, validating, and writing BIDS datasets. Its architecture is designed for extensibility and maintainability.

Key Features

• BIDS Dataset Loading
  Load BIDS datasets of any size or complexity, with support for multiple BIDS schema versions.

• Flexible Query Engine
  Query files, folders, and metadata using a powerful, Pythonic API. Supports entity-based, scope-based, and custom queries.

• Validation
  Validate datasets against the BIDS specification and custom rules using a plugin-based validation system.

• Writing and Derivatives
  Write and update BIDS datasets, including support for creating and saving BIDS derivatives.

• Lazy Loading
  Efficiently handle large datasets with optional lazy loading, reducing memory usage and speeding up initial access.

• Plugin Architecture
  Extend or customize core functionality (validation, file handling, schema, etc.) via a robust plugin system.

• PyBIDS Compatibility Layer
  Drop-in compatibility for many pybids API calls, easing migration from or integration with existing codebases.

• Synthetic and Real Data Support
  Works with both synthetic test datasets and real-world BIDS datasets.

• CI/CD Ready
  Includes a comprehensive suite of automated and manual tests, with synthetic datasets for reproducibility.

• Extensible and Versioned Schema
  Supports multiple BIDS schema versions and allows for easy extension as the BIDS standard evolves.

Architecture

• Core Models:
  The in-memory graph is hand-maintained in ancpbids/model_base.py. Versioned enums are loaded at runtime from official BIDS schema.json files vendored in ancpbids/schema/.

• Plugin System:
  The plugin mechanism (see ancpbids/plugin.py) allows for dynamic extension of core functionality. Plugins can hook into schema modification, dataset processing, file handling, writing, and validation. Graph methods (query, get_schema, …) live on the model classes; SchemaPlugin remains an extension point.

• Query Engine:
  The query logic is implemented in ancpbids/query.py, providing flexible access to dataset contents and metadata.

• Compatibility Layer:
  ancpbids/pybids_compat.py provides compatibility with the pybids API for easier migration and integration.

• Utilities:
  Helper functions and utilities are in ancpbids/utils.py.

• Testing:
  The tests/ directory is organized into auto (CI-safe) and manual (non-deterministic or performance) tests, with synthetic datasets under tests/data/.

Plugin System

The plugin system is a core feature for extensibility:

• Plugin Types:

  • SchemaPlugin: Modify or extend the BIDS schema.
  • DatasetPlugin: Operate on in-memory dataset graphs.
  • FileHandlerPlugin: Register custom file readers/writers.
  • WritingPlugin: Add files/folders during dataset writing.
  • ValidationPlugin: Add custom validation rules.

• Registration and Discovery:
  Plugins are registered via register_plugin or discovered with load_plugins_by_package. They are prioritized by a ranking value.

• Execution:
  At key points (e.g., dataset load, write, validate), the system retrieves and executes all relevant plugins using get_plugins.

• How to Add a Plugin:

  1. Subclass the appropriate plugin base class from plugin.py.
  2. Implement the required execute method.
  3. Register your plugin using register_plugin or by placing it in a discoverable package.

Versioning and Schema Evolution

• The codebase supports multiple BIDS schema versions via vendored schema.json files.
• The schema is loaded dynamically based on the dataset version, allowing for forward compatibility.

Testing and CI

• Unit Tests:
  Located in tests/auto/, these are run automatically in CI.
• Manual/Performance Tests:
  Located in tests/manual/, these are for benchmarking or non-deterministic checks.
• Synthetic Data:
  All tests use synthetic datasets in tests/data/ to ensure reproducibility.

Developer Guidelines

• Extending the Model:
  Add a new ancpbids/schema/schema_v<version>.json (see Model Generation Utility). Type stubs and model_latest update from the vendored JSON files.
• Adding Plugins:
  Follow the plugin system described above.
• Testing:
  Add new tests to tests/auto/ for CI-safe code, and to tests/manual/ for performance or integration tests.
• Documentation:
  Update README.md and docstrings for any new features or changes.

Code Quality

• The codebase uses type hints and docstrings for clarity.
• Contributions should follow PEP8 and include tests and documentation.

Model Generation Utility

The in-memory graph (ancpbids/model_base.py) is hand-maintained. tools/generatemodel.py fetches an official BIDS schema.json into ancpbids/schema/ and writes version-specific type stubs (v1_X_Y.pyi) so IDEs can see enum literals such as SuffixEnum.bold. Enums are still built from JSON at runtime.

Usage:

uv run --with requests python tools/generatemodel.py [--schema-version <version>]
uv run python tools/generatemodel.py --stubs-only

• If --schema-version is omitted, the latest available schema version is used.
• --stubs-only regenerates stubs from JSON files already in ancpbids/schema/.
• Output: ancpbids/schema/schema_v<version>.json, ancpbids/schema/v1_X_Y.pyi, and ancpbids/schema/aliases.pyi.

Further Reading

• BIDS Specification
• ancpBIDS Documentation