npc_sessions
neuropixels cloud sessions
Tools for accessing data and metadata for behavior and epyhys sessions from the Mindscope Neuropixels team - in the cloud.
quickstart
Make a conda environment with python>=3.9 and simply pip install the npc_sessions package:
conda create -n npc_sessions python>=3.9
conda activate npc_sessions
pip install npc_sessions
>>> from npc_sessions import DynamicRoutingSession, get_sessions;
# each object is used to get metadata and paths for a session:
>>> session = DynamicRoutingSession('668755_2023-08-31')
>>> session.is_ephys
True
>>> session.stim_paths[0].stem
'DynamicRouting1_668755_20230831_131418'
# data is processed on-demand to generate individual pynwb modules:
>>> session.subject # doctest: +SKIP
subject pynwb.file.Subject at 0x1999418231888
Fields:
age: P205D
age__reference: birth
date_of_birth: 2023-02-06 20:23:02-08:00
genotype: wt/wt
sex: M
species: Mus musculus
strain: C57BL6J(NP)
subject_id: 668755
# a full NWBFile instance can also be generated with all currently-available data:
>>> session.nwb # doctest: +SKIP
root pynwb.file.NWBFile at 0x...
Fields:
acquisition: {
lick spout <class 'ndx_events.events.Events'>
}
devices: {
18005102491 <class 'pynwb.device.Device'>,
18005114452 <class 'pynwb.device.Device'>,
18005123131 <class 'pynwb.device.Device'>,
18194810652 <class 'pynwb.device.Device'>,
19192719021 <class 'pynwb.device.Device'>,
19192719061 <class 'pynwb.device.Device'>
}
...
# loop over all currently-tracked sessions using the session-generator:
>>> all(s.session_start_time.year >= 2022 for s in get_sessions()) # doctest: +SKIP
True
>>> trials_dfs = {}
>>> for session in get_sessions(): # doctest: +SKIP
... trials_dfs[session.id] = session.trials[:]
to develop with conda
To install with the intention of contributing to this package:
- create a conda environment:
conda create -n npc_sessions python>=3.9
conda activate npc_sessions
- clone npc_sessions from github:
git clone git@github.com:AllenInstitute/npc_sessions.git
- pip install all dependencies:
cd npc_sessions
pip install -e .
Hierarchy of required packages
Current NWB components
key data types
(the following all have a description field, as well as other type-specific attributes)
-
DynamicTable: for general tabular data- e.g.
nwb.units - each column in the table is stored as a vector, which can be accessed individually (fast)
- can be accessed as a pandas dataframe with
nwb.units[:], but requires reading all data in all columns (slow) - as well as individual values, cells in the table can contain multidimensional
arrays. These are represented differently depending on location:
- in the pandas dataframe, these are represented as one would expect:
nwb.units[:].spike_times.iloc[0]is a 1-D arraynwb.units[:].waveform_mean.iloc[0]is a 2-D array (time x channels)
- in the non-dataframe memory representation, there are sometimes two
components, where the
*_indexis the one that should be used:nwb.units.spike_timesis a 1-D array of all spike times for all units, in chronological order (float)nwb.units.spike_times_indexis a list (len = num units) of arrays (len = num spikes for each unit)
- on disk, these columns separated columns are different again, for example:
/nwb/units/spike_timesis a 1-D array of all spike times for all units, in chronological order (float)/nwb/units/spike_times_indexis a 1-D array of values corresponding to the end of each unit's times in/nwb/units/spike_times:- the first unit's spike times are in
spike_times[: spike_times_index[0]] - the second unit's are in
spike_times[spike_times_index[0]: spike_times_index[1]]
- the first unit's spike times are in
- in the pandas dataframe, these are represented as one would expect:
- e.g.
-
TimeIntervals: for tabular data where each row is an interval of time- a subclass of
DynamicTablewhich must have astart_timeandstop_timecolumn, plus any other user-defined columns
- a subclass of
-
TimeSeries: for general array data- has an array of
data(1-D or N-D, with time as first dimension) - has
unitsas a string - has either:
timestamps(same length asdata)starting_timeandrate(assumed to be constant)
- has an array of
-
ElectricalSeries: for ephys array data- a subclass of
TimeSerieswith units fixed as volts
- a subclass of
-
Events: an NWB extension for discrete event times- like the
TimeSeriesclass, but only hastimestamps, without values fordata(think: lick times)
- like the
-
session metadata (multiple attributes)
-
subject (multiple attributes)
-
devices:
DynamicTable- physical probes (model, serial number)
- currently only neuropixels probes
-
electrode_groups:
DynamicTable- represents the group of channels on one probe inserted in the brain
- has session-specific info, like position relative to other probes or stereotactic coords
-
electrodes:
DynamicTable- individual channels on a probe
- has CCF coords
-
units:
DynamicTable- metrics, links to
electrodesviapeak_channel
- metrics, links to
-
epochs:
TimeIntervals- start/stop time of each stim block
- has a list of tags (includes
TaskControlsubclass name)
-
intervals:
Mapping[str, TimeIntervals]- 1x table per stim epoch with trials
- behavior performance table (each block an interval)
-
trials:
TimeIntervals- same as
intervals[DynamicRouting1]
- same as
-
invalid_times:
TimeIntervals -
acquisition:
Mapping[str, Any]raw data- if is_ephys:
- raw AP:
Mapping[str, ElectricalSeries] - raw LFP:
Mapping[str, ElectricalSeries]
- raw AP:
- if is_sync:
- lick_sensor_rising_edges:
Events - lick_sensor_falling_edges:
Events
- lick_sensor_rising_edges:
- if is_task:
- rewards:
Events
- rewards:
- if is_video:
- video frame times: 1x
Eventsper camera
- video frame times: 1x
- if is_ephys:
-
processing:
Mapping[str, Any]processed/filtered data- behavior:
Mapping[str, Any]- licks:
Events- from sync or stim file
- running_speed:
TimeSeries- from stim file, enhanced with sync info if available
- licks:
- ecephys:
Mapping[str, Any]
- behavior:
-
analysis:
Mapping[str, Any]derived data, results- if is_ephys:
- all_spike_histograms: 1x
TimeSeriesper probe - drift_maps:
ImageSeries
- all_spike_histograms: 1x
- if is_task:
- performance:
TimeIntervals
- performance:
- if is_ephys:
Todo:
- filtered LFP
- stimulus templates (vis, aud, opto)
- OptogeneticStimulusSite
- analysis -> RFMaps
- per-unit response metric for each stim modality
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
npc_sessions-0.0.288.tar.gz
(89.1 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file npc_sessions-0.0.288.tar.gz.
File metadata
- Download URL: npc_sessions-0.0.288.tar.gz
- Upload date:
- Size: 89.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
uv/0.11.30 {"installer":{"name":"uv","version":"0.11.30","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
d425377f61fc007fce5a6d32009ee017f351b23ff33e88734730ee08edb530b1
|
|
| MD5 |
d3f52ee340b5c9ffb9f63f73d5925437
|
|
| BLAKE2b-256 |
afe3ad4f18eac13c7f02d77c75c159481959521ca99d3bf31a4d12acc077d192
|
File details
Details for the file npc_sessions-0.0.288-py3-none-any.whl.
File metadata
- Download URL: npc_sessions-0.0.288-py3-none-any.whl
- Upload date:
- Size: 93.0 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
uv/0.11.30 {"installer":{"name":"uv","version":"0.11.30","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
b7d7ed41ce01444d3b49cf09584996a80444e989edb9d422edeb2aa2d6553c09
|
|
| MD5 |
d41b5d6ceeb53930b77d5e8075db4aa2
|
|
| BLAKE2b-256 |
e02c5f9a790b3ea916dcdaf77989a057cdd16bd2c2fe54de5d0fad1a31f25230
|
