VibFrame to AnnData

Python package for staged conversion of TWave VibFrame data into reusable AnnData datasets.

Public user and API documentation: https://413hq.github.io/vibframe-anndata-docs/

The workflow is intentionally split in two phases:

1. Ingest raw VibFrame data once into an AnnData base (obs, obsm, uns).
2. Add, recalculate or remove features from the raw signals already stored in AnnData, without reopening the original VibFrame.

For datasets that fit comfortably in memory:

from vibframe_anndata import add_features, import_raw, remove_features, write_h5ad

adata = import_raw("dataset.vibframe.zip")
write_h5ad(adata, "dataset_raw.h5ad")

adata = add_features(
    adata,
    {
        "version": 1,
        "features": [
            {"name": "rms", "source": "waveform"},
            {"name": "kurtosis", "source": "waveform"},
        ],
    },
)

adata = remove_features(adata, ["kurtosis"])
write_h5ad(adata, "dataset_features.h5ad")

Complete evaluation metadata is opt-in: snapshot and waveform labels stay outside X, while original DiagGT, manifests and scenario documents are preserved losslessly. See docs/GROUND_TRUTH.md for alignment, file access and existing-H5AD upgrades.

adata = import_raw(
    "dataset.vibframe.zip",
    config={"version": 1, "ground_truth": {"enabled": True, "scope": "all"}},
)

For datasets larger than RAM, use the blockwise H5AD APIs:

from vibframe_anndata import import_raw_to_h5ad, add_features_to_h5ad

base = import_raw_to_h5ad(
    "dataset.vibframe.zip",
    "dataset_raw.h5ad",
    config={
        "version": 1,
        "raw_import": {"on_missing_signal": "nan"},
        "ground_truth": {"enabled": True, "on_missing": "error"},
        "output": {"dtype": "float32"},
    },
    block_size_mib=8,
)

add_features_to_h5ad(
    base,
    {
        "version": 1,
        "output": {"dtype": "float32"},
        "feature_policy": {"on_error": "nan"},
        "features": [{"name": "rms", "source": "waveform"}],
    },
    output="dataset_features.h5ad",
    block_rows=256,
)

The large-file path is transactional: raw samples are read in observation blocks and the destination is replaced only after all feature blocks succeed.

Install from PyPI:

python -m pip install vibframe-anndata

Install for development:

python -m pip install -e ".[dev]"

The repository intentionally does not version the project data/ directory. The mandatory CI fixture is generated deterministically from versioned synthetic source.

License

The software package is released under the BSD 3-Clause License, allowing academic and research users to inspect, modify and redistribute the code while retaining the copyright and license notices.

The BSD license applies to the software source code distributed by this project. External or reference datasets, VibFrame captures and other third-party material are not relicensed by this repository unless explicitly stated. Release archives exclude VibFrame reference binaries and the local data/ tree.

Documentation

User-facing and API documentation is published independently of this repository and versioned with package releases:

• Public documentation: https://413hq.github.io/vibframe-anndata-docs/
• Versioned API reference: https://413hq.github.io/vibframe-anndata-docs/latest/reference/api/
• Known limitations: https://413hq.github.io/vibframe-anndata-docs/latest/reference/limitations/

Internal development, architecture, benchmark and requirements documentation remains under docs/ in this repository.

Ground truth without loading raw samples

from vibframe_anndata import (
    add_ground_truth_to_h5ad, get_waveform_ground_truth, list_evaluation_files,
)

# Existing features and raw arrays are retained, not recalculated.
add_ground_truth_to_h5ad("dataset_features.h5ad", "dataset.vibframe.zip")
print(list_evaluation_files("dataset_features.h5ad"))
labels = get_waveform_ground_truth("dataset_features.h5ad")

Numeric snapshot keys are microseconds UTC; cropped capture t is kept distinct from snap_t. Arbitrary diagnostic intervals are preserved, not guessed into per-snapshot targets.