OSF — Python Implementation

Target Platform

Data analytics, scientific computing, and AI/ML pipelines. The Python bindings are the primary entry point for data exploration and the foundation for ecosystem integrations (Arrow, PyTorch, TensorFlow, LangChain).

The crate sits on top of the Rust osf-core library via PyO3 — see DECISIONS §18. One codebase, two audiences.

Status

In progress. Reader, manager, and writer bindings are functional; the Python wheel builds via maturin with abi3 so a single artefact covers Python 3.9 / 3.10 / 3.11 / 3.12 / 3.13.

Distribution name vs. import name

• PyPI: pip install osfdata
• Python: import osf

The split follows the established Python convention (scikit-learn imports as sklearn, PyYAML imports as yaml, beautifulsoup4 imports as bs4). The PyPI name osf is registered to an unrelated 2015 package; osfdata is the Optimeas distribution.

Installation

PyPI (recommended)

pip install osfdata

The PyPI distribution name is osfdata (the short name osf is taken by an unrelated 2015 package); the Python import name is osf for brevity. Wheels are published with abi3 for Python 3.9–3.13 on Linux (x86_64 + aarch64), macOS arm64, and Windows x64; other platforms build from the sdist and need a local Rust toolchain.

Development build (source checkout)

Build the native extension from a source checkout — the right path for local Rust-side hacking or an unsupported platform.

git clone https://github.com/optimeas/osf
cd osf/implementations/python

# Create a virtual environment.
uv venv
.venv/Scripts/Activate.ps1            # Windows PowerShell
# source .venv/bin/activate           # Linux / macOS

# Install build + test tooling.
uv pip install maturin pytest

# Build the native extension and install it editably into the venv.
maturin develop --release

# Run the test suite.
pytest tests/

maturin develop produces an editable install — code changes to python/osf/*.py and python/osf/*.pyi are picked up immediately; Rust changes need another maturin develop.

For a complete walkthrough of the toolchain, build process, and release pipeline, see BUILD.md and RELEASE.md. They explain the PyO3 + maturin stack from local setup through the Trusted-Publishing release to PyPI, intended for developers new to Python's packaging conventions.

Quick start

import osf
import numpy as np

# Reader: convenience path
mgr = osf.load("examples/steam_loco.osf")
print(f"Channels: {len(mgr)}")
print(f"Compressed: {mgr.stats.compressed}")
print(mgr.stats)

# Channel access by name (DECISIONS §10)
ch = mgr.channel("GPS.PosFixMode")
print(f"{ch.name}: {ch.data_type}, {ch.sample_count} samples")

arr = ch.samples()           # NumPy float64 array
ts = ch.timestamps_ns()      # NumPy int64 array

# Equidistant segments are first-class
for seg in ch.segments:
    print(f"  start={seg.start_timestamp_ns} rate={seg.sample_rate_hz} n={seg.sample_count}")

# Writer: convenience path — round-trip an existing manager
osf.save(mgr, "out.osf")

# Writer: builder path — construct from scratch
b = osf.WriterBuilder().creator("my-app").tag("preview")
idx = b.add_channel(
    name="Sensor.Temp",
    data_type="double",
    channel_type="scalar",
    physical_unit="°C",
)
b.add_equidistant_segment(
    idx,
    start_ns=1_700_000_000_000_000_000,
    sample_rate_hz=1.0,
    values=np.array([18.4, 18.5, 18.6], dtype=np.float64),
)
b.write_to_file("synthetic.osf")

Both osf.load() and osf.save() always emit OSF5 (DECISIONS §6), so an OSF4 source file becomes an OSF5 target after a round-trip.

API surface

NumPy is the data type for every numeric and gpslocation channel. gpslocation arrives as a (N, 3) float64 array with columns [latitude, longitude, altitude]. string channels return list[str]; binary channels return list[bytes].

Performance

osf.load("examples/steam_loco.osf") (123 channels, ~164 k samples) measures ~3 ms on a release-build extension on the dev box — same order of magnitude as the underlying Rust read. Channel access plus NumPy array conversion adds ~0.3 ms per channel. The clone strategy (each mgr.channel(name).samples() call clones the Vec<T> once) is fast enough that an Arc<Channel> optimisation is unnecessary at this point.

Spec revision tracked

OSF specification revision 2026-05-04 (English, Deutsch). All spec-level constraints implemented in osf-core carry through automatically: removed datatypes raise OsfError, deprecated channel-level fields produce a log warning and are dropped, equidistant blocks limit to float / double, and the writer never emits OSFZ.

Dependencies

Build-time only:

pyo3 and numpy must agree on their major version per the rust-numpy README; bumping one requires bumping the other.

Next steps

1. pandas DataFrame convenience — build a DataFrame from a DataManager (one column per channel, optional time alignment).

CI builds the wheel matrix (Linux / macOS / Windows × Python 3.9–3.13) and the release workflow publishes to PyPI automatically on a v* tag (see RELEASE.md).

Relationship to python-osf

osfdata is the modern successor to the existing python-osf package. While python-osf is a pure-Python implementation supporting OSF4 reading only, osfdata provides:

• Full OSF4 and OSF5 support (read and write)
• Significantly higher performance via a Rust foundation
• Complete data type coverage including binary, gpslocation, and unsigned integers
• Compatibility with the current spec revision (2026-05-04)
• Transparent OSFZ decompression (zlib + gzip)

python-osf will be deprecated in favor of osfdata once feature parity for all production use cases is verified.

License

MIT. © 2026 Optimeas GmbH.