ndx-microscopy Extension for NWB

A Neurodata Without Borders (NWB) extension for storing microscopy data and associated metadata in a standardized format. This extension integrates with ndx-ophys-devices to provide comprehensive optical component specifications.

Features

Comprehensive Neurodata Types

• Experiment metadata container:
  • MicroscopyExperimentMetadata
• Microscope and optical component metadata (integration with ndx-ophys-devices):
  • MicroscopeModel
  • Microscope
  • MicroscopyRig
  • ExcitationSource / PulsedExcitationSource
  • BandOpticalFilter / EdgeOpticalFilter
  • DichroicMirror
  • Photodetector
  • ObjectiveLens
  • Indicator
  • ViralVector
  • ViralVectorInjection
  • StereotacticPosition
• Microscopy channel configurations:
  • MicroscopyChannel
• Imaging space definitions:
  • PlanarImagingSpace
  • VolumetricImagingSpace
• Illumination pattern classes:
  • IlluminationPattern
  • LineScan
  • PlaneAcquisition
  • RandomAccessScan
• Support for 2D and 3D imaging:
  • PlanarMicroscopySeries
  • VolumetricMicroscopySeries
  • PlanarMicroscopyStaticImage
  • VolumetricMicroscopyStaticImage
  • MultiPlaneMicroscopyContainer
  • MultiChannelMicroscopyContainer
• ROI/segmentation storage:
  • PlanarSegmentation
  • VolumetricSegmentation
  • SegmentationContainer
  • SummaryImage
  • MicroscopyResponseSeries
  • MicroscopyResponseSeriesContainer
• Abstract Neurodata types: ImagingSpace, MicroscopySeries, MicroscopyStaticImage, Segmentation

Entity Relationship Diagrams

In the diagrams, fields are required unless marked optional. Imaging data and masks are listed in (time, [depth], height, width) order; pixel and voxel sizes and dimensions are listed x first.

Experiment Metadata Components

%%{init: {'theme': 'base', 'themeVariables': {'primaryColor': '#ffffff', 'primaryBorderColor': '#144E73', 'lineColor': '#D96F32'}}}%%

classDiagram
    direction TB

    class DeviceModel{
        <<Device>>
        --------------------------------------
        attributes
        --------------------------------------
        manufacturer : text
        model_number : text, optional
    }

    class Device{
        <<Device>>
        --------------------------------------
        attributes
        --------------------------------------
        serial_number : text, optional
        --------------------------------------
        links
        --------------------------------------
        model : DeviceModel, optional
    }

    class MicroscopeModel {
        <<DeviceModel>>
    }

    class Microscope {
        <<Device>>
        --------------------------------------
        attributes
        --------------------------------------
        technique : text, optional
    }

    class MicroscopyExperimentMetadata {
        <<LabMetaData>>
        --------------------------------------
        groups
        --------------------------------------
        microscopy_rigs : MicroscopyRig[0..*]
        viral_vectors : ViralVector[0..*]
        viral_vector_injections : ViralVectorInjection[0..*]
        indicators : Indicator[0..*]
    }

    class MicroscopyRig {
        <<NWBContainer>>
        --------------------------------------
        attributes
        --------------------------------------
        description : text
        --------------------------------------
        links
        --------------------------------------
        microscope : Microscope
        excitation_source : ExcitationSource, optional
        excitation_filter : OpticalFilter, optional
        dichroic_mirror : DichroicMirror, optional
        photodetector : Photodetector, optional
        emission_filter : OpticalFilter, optional
        objective_lens : ObjectiveLens, optional
        --------------------------------------
        datasets
        --------------------------------------
        optical_path_scheme : Image, optional
    }

    class ViralVector {
        <<NWBContainer>>
        --------------------------------------
        attributes
        --------------------------------------
        construct_name : text
        titer_in_vg_per_ml : numeric
        manufacturer : text
        description : text, optional
    }
    class ViralVectorInjection {
        <<NWBContainer>>
        --------------------------------------
        attributes
        --------------------------------------
        description : text, optional
        volume_in_uL : numeric
        injection_date : text, optional
        --------------------------------------
        groups
        --------------------------------------
        viral_injection_coordinates : StereotacticPosition, optional
        --------------------------------------
        links
        --------------------------------------
        viral_vector : ViralVector
    }
    class StereotacticPosition {
        <<NWBContainer>>
        --------------------------------------
        attributes
        --------------------------------------
        anatomical_target : text
        origin : text
        orientation : text
        x_in_mm : numeric
        y_in_mm : numeric
        z_in_mm : numeric
        pitch_in_deg : numeric, optional
        yaw_in_deg : numeric, optional
        roll_in_deg : numeric, optional
    }
    class Indicator {
        <<NWBContainer>>
        --------------------------------------
        attributes
        --------------------------------------
        label : text
        description : text, optional
        manufacturer : text, optional
        --------------------------------------
        links
        --------------------------------------
        viral_vector_injection : ViralVectorInjection, optional
    }


    DeviceModel <|-- MicroscopeModel : extends
    Device <|-- Microscope : extends
    Microscope o--> MicroscopeModel : links
    MicroscopyRig o--> Microscope : links
    MicroscopyExperimentMetadata *-- MicroscopyRig : contains
    MicroscopyExperimentMetadata *-- ViralVector : contains
    MicroscopyExperimentMetadata *-- ViralVectorInjection : contains
    MicroscopyExperimentMetadata *-- Indicator : contains
    ViralVectorInjection o--> ViralVector : links
    ViralVectorInjection *-- StereotacticPosition : contains
    Indicator o--> ViralVectorInjection : links

Illumination Pattern Components

%%{init: {'theme': 'base', 'themeVariables': {'primaryColor': '#ffffff', 'primaryBorderColor': '#144E73', 'lineColor': '#D96F32'}}}%%

classDiagram
    direction TB

    class IlluminationPattern {
        <<NWBContainer>>
        --------------------------------------
        attributes
        --------------------------------------
        description : text, optional
    }

    class LineScan {
        <<IlluminationPattern>>
        --------------------------------------
        attributes
        --------------------------------------
        scan_direction : text, optional
        line_rate_in_Hz : float64, optional
        dwell_time_in_s : float64, optional
    }

    class PlaneAcquisition {
        <<IlluminationPattern>>
        --------------------------------------
        attributes
        --------------------------------------
        point_spread_function : text, optional
        illumination_angle_in_degrees : float64, optional
        plane_rate_in_Hz : float64, optional
    }

    class RandomAccessScan {
        <<IlluminationPattern>>
        --------------------------------------
        attributes
        --------------------------------------
        max_scan_points : int32, optional
        dwell_time_in_s : float64, optional
        scanning_pattern : text, optional
    }

    class ImagingSpace {
        <<NWBContainer>>
        --------------------------------------
        attributes
        --------------------------------------
        description : text, optional
        anatomical_target : text
        --------------------------------------
        groups
        --------------------------------------
        illumination_pattern : IlluminationPattern
        fov_center_coordinates : StereotacticPosition, optional
    }

    IlluminationPattern <|-- LineScan : extends
    IlluminationPattern <|-- PlaneAcquisition : extends
    IlluminationPattern <|-- RandomAccessScan : extends
    ImagingSpace *-- IlluminationPattern : contains
    ImagingSpace *-- StereotacticPosition : contains

Microscopy Series, Static Image and Imaging Space Components

%%{init: {'theme': 'base', 'themeVariables': {'primaryColor': '#ffffff', 'primaryBorderColor': '#144E73', 'lineColor': '#D96F32'}}}%%

classDiagram
    direction TB

    class MicroscopyChannel {
        <<NWBContainer>>
        --------------------------------------
        attributes
        --------------------------------------
        description : text, optional
        excitation_wavelength_in_nm : float
        emission_wavelength_in_nm : float
        --------------------------------------
        links
        --------------------------------------
        indicator : Indicator
    }

    class MicroscopySeries {
        <<TimeSeries>>
        --------------------------------------
        links
        --------------------------------------
        microscopy_rig : MicroscopyRig
        --------------------------------------
        groups
        --------------------------------------
        microscopy_channel : MicroscopyChannel

    }

    class PlanarMicroscopySeries {
        <<MicroscopySeries>>
        --------------------------------------
        datasets
        --------------------------------------
        data : numeric[frames, height, width]
        --------------------------------------
        groups
        --------------------------------------
        planar_imaging_space : PlanarImagingSpace
    }

    class VolumetricMicroscopySeries {
        <<MicroscopySeries>>
        --------------------------------------
        datasets
        --------------------------------------
        data : numeric[frames, depths, height, width]
        --------------------------------------
        groups
        --------------------------------------
        volumetric_imaging_space : VolumetricImagingSpace
    }

    class MicroscopyStaticImage {
        <<NWBDataInterface>>
        --------------------------------------
        attributes
        --------------------------------------
        description : text
        --------------------------------------
        links
        --------------------------------------
        microscopy_rig : MicroscopyRig
        --------------------------------------
        groups
        --------------------------------------
        microscopy_channel : MicroscopyChannel
    }

    class PlanarMicroscopyStaticImage {
        <<MicroscopyStaticImage>>
        --------------------------------------
        datasets
        --------------------------------------
        data : numeric[height, width]
        --------------------------------------
        groups
        --------------------------------------
        planar_imaging_space : PlanarImagingSpace
    }

    class VolumetricMicroscopyStaticImage {
        <<MicroscopyStaticImage>>
        --------------------------------------
        datasets
        --------------------------------------
        data : numeric[depths, height, width]
        --------------------------------------
        groups
        --------------------------------------
        volumetric_imaging_space : VolumetricImagingSpace
    }

    class MultiPlaneMicroscopyContainer {
        <<NWBDataInterface>>
        --------------------------------------
        groups
        --------------------------------------
        planar_microscopy_series : PlanarMicroscopySeries[0..*]
        planar_microscopy_static_images : PlanarMicroscopyStaticImage[0..*]
    }


    class MultiChannelMicroscopyContainer {
        <<NWBDataInterface>>
        --------------------------------------
        groups
        --------------------------------------
        microscopy_series : MicroscopySeries[0..*]
        microscopy_static_images : MicroscopyStaticImage[0..*]
        multi_plane_microscopy_containers : MultiPlaneMicroscopyContainer[0..*]
    }

    class ImagingSpace {
        <<NWBContainer>>
        --------------------------------------
        attributes
        --------------------------------------
        description : text, optional
        anatomical_target : text
        --------------------------------------
        groups
        --------------------------------------
        illumination_pattern : IlluminationPattern
        fov_center_coordinates : StereotacticPosition, optional
    }

    class PlanarImagingSpace {
        <<ImagingSpace>>
        --------------------------------------
        datasets
        --------------------------------------
        pixel_size_in_um : float64[x, y], optional
        dimensions_in_pixels : int32[x, y], optional
        --------------------------------------
        methods
        --------------------------------------
        get_FOV_size()
    }

    class VolumetricImagingSpace {
        <<ImagingSpace>>
        --------------------------------------
        datasets
        --------------------------------------
        voxel_size_in_um : float64[x, y, z], optional
        dimensions_in_voxels : int32[x, y, z], optional
        --------------------------------------
        methods
        --------------------------------------
        get_FOV_size()
    }

    class MicroscopyRig {
        <<NWBContainer>>
        --------------------------------------
        attributes
        --------------------------------------
        description : text
        --------------------------------------
        links
        --------------------------------------
        microscope : Microscope
        excitation_source : ExcitationSource, optional
        excitation_filter : OpticalFilter, optional
        dichroic_mirror : DichroicMirror, optional
        photodetector : Photodetector, optional
        emission_filter : OpticalFilter, optional
        objective_lens : ObjectiveLens, optional
        --------------------------------------
        datasets
        --------------------------------------
        optical_path_scheme : Image, optional
    }

    MicroscopySeries <|-- PlanarMicroscopySeries : extends
    MicroscopySeries <|-- VolumetricMicroscopySeries : extends
    ImagingSpace <|-- PlanarImagingSpace : extends
    ImagingSpace <|-- VolumetricImagingSpace : extends

    PlanarMicroscopySeries *-- PlanarImagingSpace : contains
    VolumetricMicroscopySeries *-- VolumetricImagingSpace : contains
    MicroscopyStaticImage <|-- PlanarMicroscopyStaticImage : extends
    MicroscopyStaticImage <|-- VolumetricMicroscopyStaticImage : extends
    PlanarMicroscopyStaticImage *-- PlanarImagingSpace : contains
    VolumetricMicroscopyStaticImage *-- VolumetricImagingSpace : contains
    MultiPlaneMicroscopyContainer *-- PlanarMicroscopySeries : contains
    MultiPlaneMicroscopyContainer *-- PlanarMicroscopyStaticImage : contains
    MultiChannelMicroscopyContainer *-- MicroscopySeries : contains
    MultiChannelMicroscopyContainer *-- MicroscopyStaticImage : contains
    MultiChannelMicroscopyContainer *-- MultiPlaneMicroscopyContainer : contains
    MicroscopySeries o--> MicroscopyRig : links
    MicroscopySeries *-- MicroscopyChannel : contains
    MicroscopyStaticImage o--> MicroscopyRig : links
    MicroscopyStaticImage *-- MicroscopyChannel : contains
    MicroscopyChannel o--> Indicator : links

Segmentation Components

%%{init: {'theme': 'base', 'themeVariables': {'primaryColor': '#ffffff', 'primaryBorderColor': '#144E73', 'lineColor': '#D96F32'}}}%%

classDiagram
    direction TB

    class Segmentation {
        <<DynamicTable>>
        --------------------------------------
        attributes
        --------------------------------------
        description : text
        --------------------------------------
        groups
        --------------------------------------
        summary_images : SummaryImage[0..*]
    }

    class PlanarSegmentation {
        <<Segmentation>>
        --------------------------------------
        datasets
        --------------------------------------
        image_mask : VectorData[num_roi, num_y, num_x], optional
        pixel_mask_index : VectorIndex, optional
        pixel_mask : VectorData, optional
        --------------------------------------
        groups
        --------------------------------------
        planar_imaging_space : PlanarImagingSpace
        --------------------------------------
        methods
        --------------------------------------
        add_roi()
        create_roi_table_region()
        pixel_to_image()
        image_to_pixel()
    }

    class VolumetricSegmentation {
        <<Segmentation>>
        --------------------------------------
        datasets
        --------------------------------------
        volume_mask : VectorData[num_roi, num_z, num_y, num_x], optional
        voxel_mask_index : VectorIndex, optional
        voxel_mask : VectorData, optional
        --------------------------------------
        groups
        --------------------------------------
        volumetric_imaging_space : VolumetricImagingSpace
        --------------------------------------
        methods
        --------------------------------------
        add_roi()
        create_roi_table_region()
        voxel_to_volume()
        volume_to_voxel()
    }

    class SummaryImage {
        <<NWBContainer>>
        --------------------------------------
        datasets
        --------------------------------------
        data : numeric[height, width] or numeric[depth, height, width]
        --------------------------------------
        attributes
        --------------------------------------
        description : text
    }

    class SegmentationContainer {
        <<NWBDataInterface>>
        --------------------------------------
        groups
        --------------------------------------
        segmentations : Segmentation[1..*]
        --------------------------------------
        methods
        --------------------------------------
        add_segmentation()
    }

    class MicroscopyResponseSeries {
        <<TimeSeries>>
        --------------------------------------
        datasets
        --------------------------------------
        data : numeric[number_of_frames, number_of_rois]
        rois : DynamicTableRegion
        --------------------------------------
        links
        --------------------------------------
        microscopy_series : MicroscopySeries, optional
    }

    class MicroscopyResponseSeriesContainer {
        <<NWBDataInterface>>
        --------------------------------------
        groups
        --------------------------------------
        microscopy_response_series : MicroscopyResponseSeries[1..*]
    }

    Segmentation <|-- PlanarSegmentation : extends
    Segmentation <|-- VolumetricSegmentation : extends
    SegmentationContainer *-- Segmentation : contains
    Segmentation *-- SummaryImage : contains
    MicroscopyResponseSeries o--> MicroscopySeries : links
    MicroscopyResponseSeriesContainer *-- MicroscopyResponseSeries : contains
    PlanarSegmentation *-- PlanarImagingSpace : contains
    VolumetricSegmentation *-- VolumetricImagingSpace : contains

This extension was created using ndx-template.

Installation

pip install ndx-microscopy

Documentation

For detailed documentation, including API reference and additional examples, please visit our documentation site.

Contributing

To help ensure a smooth Pull Request (PR) process, please always begin by raising an issue on the main repository so we can openly discuss any problems/additions before taking action.

The main branch of ndx-microscopy is protected; you cannot push to it directly. You must upload your changes by pushing a new branch, then submit your changes to the main branch via a Pull Request. This allows us to conduct automated testing of your contribution, and gives us a space for developers to discuss the contribution and request changes. If you decide to tackle an issue, please make yourself an assignee on the issue to communicate this to the team. Don't worry - this does not commit you to solving this issue. It just lets others know who they should talk to about it.

From your local copy directory, use the following commands.

If you have not already, you will need to clone the repo:

$ git clone https://github.com/catalystneuro/ndx-microscopy

First create a new branch to work on

$ git checkout -b <new_branch>

Make your changes. Add new objects related to optical experiment or add more attributes on the existing ones. To speed up the process, you can write mock function (see _mock.py) that would be used to test the new neurodata type

We will automatically run tests to ensure that your contributions didn't break anything and that they follow our style guide. You can speed up the testing cycle by running these tests locally on your own computer by calling pytest from the top-level directory. Push your feature branch to origin (i.e. GitHub)

$ git push origin <new_branch>

Once you have tested and finalized your changes, create a pull request (PR) targeting main as the base branch: Ensure the PR description clearly describes the problem and solution. Include the relevant issue number if applicable. TIP: Writing e.g. "fix #613" will automatically close issue #613 when this PR is merged. Before submitting, please ensure that the code follows the standard coding style of the respective repository. If you would like help with your contribution, or would like to communicate contributions that are not ready to merge, submit a PR where the title begins with "[WIP]."

Update the CHANGELOG.md regularly to document changes to the extension.

License

BSD 3-Clause License. See LICENSE.txt for details.