Skip to main content

AIND Ephys IBL GUI Conversion

CI semantic-release: angular License ruff uv Copier

Convert AIND Neuropixels electrophysiology data into IBL ALF format for consumption by the IBL ephys alignment GUI. It extracts spike data from SpikeInterface sorting output and computes continuous QC metrics (RMS, spectral density, per-band coherence) from raw Neuropixels recordings.

Installation

From source (no PyPI release):

git clone https://github.com/AllenNeuralDynamics/aind-ephys-ibl-gui-conversion.git
cd aind-ephys-ibl-gui-conversion
pip install .

Or add as a dependency:

uv add "aind-ephys-ibl-gui-conversion @ git+https://github.com/AllenNeuralDynamics/aind-ephys-ibl-gui-conversion.git"

Usage

This package is a library -- there is no CLI. The two top-level entry points are extract_spikes() and extract_continuous(), both imported from the package root:

from pathlib import Path
from aind_ephys_ibl_gui_conversion import extract_spikes, extract_continuous

sorting_folder = Path("/path/to/<session>_sorted")
results_folder = Path("/path/to/output")

# Convert sorted spikes into IBL ALF format
extract_spikes(sorting_folder, results_folder)

# Compute continuous QC metrics (RMS, PSD, coherence)
extract_continuous(sorting_folder, results_folder)

Both functions write one subdirectory per probe under results_folder.

extract_spikes(sorting_folder, results_folder, ...)

Reads the postprocessed/ directory inside sorting_folder and writes IBL ALF spike files per probe.

Parameter Default Description
sorting_folder -- Path to the sorted session folder (name typically ends with _sorted)
results_folder -- Output directory (one subfolder per probe will be created)
stream_to_use None Filter to a single Open Ephys stream name (e.g. "Record Node 104#Neuropix-PXI-100.ProbeA-AP"). If None, all streams are processed.
min_duration_secs 300 Skip recordings shorter than this many seconds

extract_continuous(sorting_folder, results_folder, ...)

Reads compressed Neuropixels recordings from ecephys_compressed/ and writes per-band RMS, PSD, and coherence files in IBL ALF format.

Parameter Default Description
sorting_folder -- Path to the sorted session folder
results_folder -- Output directory
stream_to_use None Filter to a single Open Ephys stream
main_recording_min_secs 600 Minimum duration (s) for the main recording block
probe_surface_finding None Optional path to a separate surface-finding asset
rms_window_interval 30.0 Gap between analysis windows (s); auto-shortened on short recordings to guarantee >= 20 windows
rms_window_duration 4.0 Length of each analysis window (s)
num_parallel_jobs 4 Concurrent block processing
coherence_block_source "all" Blocks contributing to correlation/coherency: "all", "main", or "surface". PSD, RMS, and channel metadata still use all blocks.
extra_metrics None Metric extensions sharing the existing sampled reads and FFTs; None runs DEFAULT_EXTRA_METRICS (raw activity), a tuple replaces them, () runs none

Per-contact AP event rates (raw activity) run by default and share recording reads with the other metrics.

Coherence bands (from metrics.COHERENCE_BANDS):

Band Range (Hz)
delta 0.5 -- 4
theta 4 -- 12
alpha 12 -- 30
beta 30 -- 100
gamma 100 -- 300

Input data expectations

Discovery can inventory heterogeneous experiment/stream membership through the metadata-only inventory API and versioned contract. It retains recording geometry, sorting outputs, candidate source joins, and validation findings. A separate saved plan controls worker extraction.

The conversion-plan layer selects the longest recorded block by default and keeps its analyzer family together. It records overrides, missing physical shanks, and ambiguous sources. Workers validate and execute the saved plan for both spike and continuous extraction.

Both entry points expect the AIND session layout:

<session>/
|-- ecephys_clipped/                    # Open Ephys binary (read by SpikeInterface)
|-- ecephys_compressed/                 # Zarr + WavPack compressed recordings
|   |-- experiment1_<stream>.zarr
|   +-- experiment2_<stream>.zarr
|
+-- <session>_sorted/                   # SORTING_FOLDER argument points here
    +-- postprocessed/
        |-- experiment1_<stream>_recording1.zarr        # SortingAnalyzer
        +-- experiment1_<stream>_recording1_group<N>.zarr  # Multi-shank (one per shank)

The package handles both Neuropixels 1.0 (separate AP/LFP streams) and 2.0 (combined wideband) probes, as well as multi-shank probes.

Stream naming

Open Ephys stream names follow the pattern:

Record Node <id>#Neuropix-PXI-<model>.<probe_name>[-AP|-LFP]

For example, Record Node 104#Neuropix-PXI-100.ProbeA-AP is parsed into probe_name = "ProbeA".

Output format

All outputs are written under results_folder/<probe_name>/ in IBL ALF format.

From extract_spikes

File Shape / dtype Description
spikes.times.npy (n_spikes,) float64 Spike times in seconds
spikes.clusters.npy (n_spikes,) Cluster ID per spike
spikes.amps.npy (n_spikes,) float64 Spike amplitudes (negated)
spikes.depths.npy (n_spikes,) Spike depths along the probe (um)
spike_shank_indices.npy (n_spikes,) 0-based shank index per spike
clusters.channels.npy (n_clusters,) Peak channel per cluster
clusters.peakToTrough.npy (n_clusters,) Peak-to-trough duration (samples)
clusters.waveforms.npy (n_clusters, n_samples, n_channels) Mean waveforms
clusters.metrics.csv -- SpikeInterface quality metrics, QC and classification labels
unit_shank_indices.npy (n_clusters,) 0-based shank index per cluster

QC and classification labels come from the analyzer's sorting properties. If either label is missing, conversion also checks the matching folder under curated/. Analyzer labels take precedence, and a missing curated folder is allowed. A loaded curated sorting must have the same unit IDs in the same order. decoder_label takes precedence over unitrefine_label within each source; classification is written to the unitrefine_label CSV column.

From extract_continuous

File Description
_iblqc_ephysTimeRmsAP.rms.npy (*Main* if multi-block) AP-band RMS time series, shape (n_windows, n_channel_table_rows); rows the block did not record are NaN
_iblqc_ephysTimeRmsAP.timestamps.npy Window centers (s)
_iblqc_ephysTimeRmsLF*.rms.npy LFP-band RMS time series
_iblqc_ephysTimeRmsLF*.timestamps.npy Window centers (s)
_iblqc_ephysSpectralDensityLF.power.npy PSD (V^2/Hz), shape (n_freqs, n_channels)
_iblqc_ephysSpectralDensityLF.freqs.npy Frequencies (Hz)
band_corr/<band>_mean_corr.npy Full per-band correlation matrix in channel-table row order
band_corr/<band>_coherency.npy Full per-band complex coherency matrix in channel-table row order
band_corr/<band>_shank<i>_mean_corr.npy Per-shank compatibility view sliced from the full matrix
band_corr/<band>_shank<i>_coherency.npy Per-shank compatibility view sliced from the full matrix
band_corr/row_channels.json Matrix row-to-channel table map (always written)
band_corr/channel_blocks.json Legacy multi-block channel map (only if surface-finding blocks are present)
channels.localCoordinates.npy Unique channel positions (x, y)
channels.rawInd.npy Legacy positional channel indices
channels.shankInd.npy Zero-based producer shank ID per channel-table row; physical IDs may be sparse
channels.contactId.npy Physical contact ID per channel-table row; empty string when unavailable
_iblqc_metrics.method.json Method metadata (FFT params, CMR flag)

Metrics method

For the directory boundary shared with preprocessing and the GUI, channel identity, shank derivation, matrix row maps, and split-stream examples, read the ephys collection contract. The datapackage references the collection directory, not individual QC files. Keep ALF objects at its root and correlation/coherency products in band_corr/.

RMS and PSD are computed via FFT, not time-domain filtering:

  • Hann-windowed segments, Welch-style PSD
  • Band-limited RMS via Parseval's theorem (integrate PSD over each band)
  • AP band: > 300 Hz (neural noise); LFP band: 1 -- 300 Hz
  • Across-channel common-median referencing is applied.
  • Correlation and coherency combine all blocks by default. Set coherence_block_source="surface" to exclude main-recording artifacts while retaining surface-finding blocks; explicit asset provenance takes precedence over the legacy duration heuristic.

Adapting to your own data

The package is tightly coupled to the AIND session layout and Open Ephys stream naming. To adapt it to a different setup, the main integration points are:

  1. Recording discovery: recording_utils.get_ecephys_stream_names() and io.load_probe_streams() enumerate compressed .zarr recordings under ecephys_compressed/. Replace these if your recordings live elsewhere or use a different container.
  2. Stream name parsing: recording_utils._stream_to_probe_name() extracts the probe name via a regex over the Open Ephys stream ID. Replace this if your stream names follow a different convention.
  3. Sorting output: extract_spikes() assumes SpikeInterface SortingAnalyzer output in <sorting_folder>/postprocessed/. If you have a different SortingAnalyzer location or a different post-processed format, point at it directly.
  4. Surface-finding recordings: If you use separate "surface finding" recordings (short probe-movement blocks used to map the cortical surface), pass the asset path via probe_surface_finding. Otherwise leave it unset.

The output format is IBL ALF and is decoupled from input assumptions -- it will work with any IBL GUI consumer.

Development

See CONTRIBUTING.md for developer setup, checks, and repository conventions. Coding agents should also read AGENTS.md.

License

This project is licensed under the MIT License -- see the LICENSE file for details.

Metadata

Release files for aind-ephys-ibl-gui-conversion 0.8.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for aind-ephys-ibl-gui-conversion 0.8.0
File Size Uploaded Provenance Actions
aind_ephys_ibl_gui_conversion-0.8.0.tar.gz 633.8 kB Without provenance Details for aind_ephys_ibl_gui_conversion-0.8.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for aind-ephys-ibl-gui-conversion 0.8.0
File Interpreter ABI Platform Actions
aind_ephys_ibl_gui_conversion-0.8.0-py3-none-any.whl Python 3 none any Details for aind_ephys_ibl_gui_conversion-0.8.0-py3-none-any.whl

Total release size: 714.2 kB

Release files / aind_ephys_ibl_gui_conversion-0.8.0.tar.gz

Download URL aind_ephys_ibl_gui_conversion-0.8.0.tar.gz
Size 633.8 kB
Tags Source
SHA-256 checksum
How to use checksums
84ef33c34eaf61ec76e3945cd786c62e62f55bbe6efa8a50996fd9a0f19da2ce
BLAKE2b-256 checksum
How to use checksums
aeefa377ba424aa2042ac0f90faabf834d2281082c0917ecec9a3a5a53e04e00
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.13.9 {"installer":{"name":"uv","version":"0.13.9","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Release files / aind_ephys_ibl_gui_conversion-0.8.0-py3-none-any.whl

Download URL aind_ephys_ibl_gui_conversion-0.8.0-py3-none-any.whl
Size 80.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
2293f363293bcc107c71d2274f107d4251e5f2848c728cd51a69e052d1026b0f
BLAKE2b-256 checksum
How to use checksums
3dda012a2b67764c480a3de9fc63ba133a9293b3728b9f5448dea53408e84a95
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.13.9 {"installer":{"name":"uv","version":"0.13.9","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Security for aind-ephys-ibl-gui-conversion 0.8.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page