ibl-to-nwb 0.1.0

Tools to convert IBL data to NWB format

IBL-to-nwb

This repository houses the modules used to convert IBL specific neurophysiology data in the open source ONE format (Alyx + ALF) into NWB data standard.

• Alyx: a data base that contains all the metadata associated with an experiment: session details, subject details, probe information etc. This data has a one-to-one mapping to supported metadata of NWB.
• ALF: format for storage of all the experimental data: electrophysiology time series (raw + processed), trials data, sorted spikes data, behavior (raw + processed), stimulus. The figure below shows the mapping from ALF/ALyx to NWB:

Usage:

1. IBL to NWB conversion (using API):

   1. Installation:

      cd desired-path
      git clone https://github.com/catalystneuro/IBL-to-nwb.git
      cd IBL-to-nwb
      

      create virtual environment and install dependencies from requirements.txt:

      conda env create -f conda_environment.yaml
      conda activate IBL2NWB
      

   2. Retrive the id of the experiment of interest using ONE api:

      from oneibl.one import ONE
      one=ONE()
      # use the ONE doc to use correct search terms to retrieve the eid
      eid = one.search(date_range=['2020-03-23', '2020-03-24'],subject='CSH_ZAD_011')[0]
      # example eid:
      eid = 'da188f2c-553c-4e04-879b-c9ea2d1b9a93'
      

   3. Using the eid, generate a json file containing all the collected data/metadata from the servers (Example output file):

      from .ibl_nwb import Alyx2NWBMetadata
      metadata_object = Alyx2NWBMetadata(eid=eid,one_obj=one)
      # alternatively, you can also provide one search **kwargs directly:
      metadata_obj = Alyx2NWBMetadata(date_range=['2020-03-23', '2020-03-24'],subject='CSH_ZAD_011')
      json_save_loc = r'path-to-save-json-file.json'
      metadata_obj.write_metadata(json_save_loc)
      

   4. Generate nwb file using the saved json file:

      from .ibl_nwb import Alyx2NWBConverter
      nwb_saveloc = r'nwb-save-path.nwb'
      save_raw = False # keep as true if you want to add raw (ephysData.raw.* , camera.raw*) files, these are large files and will take time to download and create the nwbfile!!
      converter=Alyx2NWBConverter(nwb_metadata_file=json_save_loc, saveloc=nwb_saveloc, save_raw=save_raw)
      # alternatively you can also provide the metadata object:
      converter=Alyx2NWBConverter(metadata_obj=metadata_obj, saveloc=nwb_saveloc)
      # create nwb file: 
      converter.run_conversion()
      converter.write_nwb()
      

   This should create an nwb file. Example file.

2. IBL to NWB conversion (using GUI):

   from .ibl_nwb import Alyx2NWBGui
   Alyx2NWBGui(eid=eid, nwbfile_saveloc=nwb_saveloc, metadata_fileloc=json_save_loc)
   #alternatively provide the one search kwargs:
   Alyx2NWBGui(nwbfile_saveloc=nwb_saveloc, metadata_fileloc=json_save_loc, dataset_types=['_iblmic_audioSpectrogram.frequencies''])
   

   This opens up a gui which will allow you to edit nwbfile/ibl session related metadata and also convert to nwb using run_conversion button. Check the animation below on how to navigate this gui:

   

3. Visualization of nwbfile using nwbwigets in a Jupyter notebook:

   from pynwb import NWBHDF5IO
   from nwbwidgets import nwb2widget
   from IPython.display import display
   io = NWBHDF5IO(r"path-to-saved-nwb-file.nwb", mode='r', load_namespaces=True)
   nwb = io.read()
   a=nwb2widget(nwb)
   display(a)