npc_sessions
neuropixels cloud sessions
Tools for accessing data and metadata for behavior and epyhys sessions from the Mindscope Neuropixels team - in the cloud.
quickstart
Make a conda environment with python>=3.9 and simply pip install the npc_sessions package:
conda create -n npc_sessions python>=3.9
conda activate npc_sessions
pip install npc_sessions
>>> from npc_sessions import DynamicRoutingSession, get_sessions;
# each object is used to get metadata and paths for a session:
>>> session = DynamicRoutingSession('668755_2023-08-31')
>>> session.is_ephys
True
>>> session.stim_paths[0].stem
'DynamicRouting1_668755_20230831_131418'
# data is processed on-demand to generate individual pynwb modules:
>>> session.subject # doctest: +SKIP
subject pynwb.file.Subject at 0x1999418231888
Fields:
age: P205D
age__reference: birth
date_of_birth: 2023-02-06 20:23:02-08:00
genotype: wt/wt
sex: M
species: Mus musculus
strain: C57BL6J(NP)
subject_id: 668755
# a full NWBFile instance can also be generated with all currently-available data:
>>> session.nwb # doctest: +SKIP
root pynwb.file.NWBFile at 0x...
Fields:
acquisition: {
lick spout <class 'ndx_events.events.Events'>
}
devices: {
18005102491 <class 'pynwb.device.Device'>,
18005114452 <class 'pynwb.device.Device'>,
18005123131 <class 'pynwb.device.Device'>,
18194810652 <class 'pynwb.device.Device'>,
19192719021 <class 'pynwb.device.Device'>,
19192719061 <class 'pynwb.device.Device'>
}
...
# loop over all currently-tracked sessions using the session-generator:
>>> all(s.session_start_time.year >= 2022 for s in get_sessions()) # doctest: +SKIP
True
>>> trials_dfs = {}
>>> for session in get_sessions(): # doctest: +SKIP
... trials_dfs[session.id] = session.trials[:]
to develop with conda
To install with the intention of contributing to this package:
- create a conda environment:
conda create -n npc_sessions python>=3.9
conda activate npc_sessions
- clone npc_sessions from github:
git clone git@github.com:AllenInstitute/npc_sessions.git
- pip install all dependencies:
cd npc_sessions
pip install -e .
Hierarchy of required packages
Current NWB components
key data types
(the following all have a description field, as well as other type-specific attributes)
-
DynamicTable: for general tabular data- e.g.
nwb.units - each column in the table is stored as a vector, which can be accessed individually (fast)
- can be accessed as a pandas dataframe with
nwb.units[:], but requires reading all data in all columns (slow) - as well as individual values, cells in the table can contain multidimensional
arrays. These are represented differently depending on location:
- in the pandas dataframe, these are represented as one would expect:
nwb.units[:].spike_times.iloc[0]is a 1-D arraynwb.units[:].waveform_mean.iloc[0]is a 2-D array (time x channels)
- in the non-dataframe memory representation, there are sometimes two
components, where the
*_indexis the one that should be used:nwb.units.spike_timesis a 1-D array of all spike times for all units, in chronological order (float)nwb.units.spike_times_indexis a list (len = num units) of arrays (len = num spikes for each unit)
- on disk, these columns separated columns are different again, for example:
/nwb/units/spike_timesis a 1-D array of all spike times for all units, in chronological order (float)/nwb/units/spike_times_indexis a 1-D array of values corresponding to the end of each unit's times in/nwb/units/spike_times:- the first unit's spike times are in
spike_times[: spike_times_index[0]] - the second unit's are in
spike_times[spike_times_index[0]: spike_times_index[1]]
- the first unit's spike times are in
- in the pandas dataframe, these are represented as one would expect:
- e.g.
-
TimeIntervals: for tabular data where each row is an interval of time- a subclass of
DynamicTablewhich must have astart_timeandstop_timecolumn, plus any other user-defined columns
- a subclass of
-
TimeSeries: for general array data- has an array of
data(1-D or N-D, with time as first dimension) - has
unitsas a string - has either:
timestamps(same length asdata)starting_timeandrate(assumed to be constant)
- has an array of
-
ElectricalSeries: for ephys array data- a subclass of
TimeSerieswith units fixed as volts
- a subclass of
-
Events: an NWB extension for discrete event times- like the
TimeSeriesclass, but only hastimestamps, without values fordata(think: lick times)
- like the
-
session metadata (multiple attributes)
-
subject (multiple attributes)
-
devices:
DynamicTable- physical probes (model, serial number)
- currently only neuropixels probes
-
electrode_groups:
DynamicTable- represents the group of channels on one probe inserted in the brain
- has session-specific info, like position relative to other probes or stereotactic coords
-
electrodes:
DynamicTable- individual channels on a probe
- has CCF coords
-
units:
DynamicTable- metrics, links to
electrodesviapeak_channel
- metrics, links to
-
epochs:
TimeIntervals- start/stop time of each stim block
- has a list of tags (includes
TaskControlsubclass name)
-
intervals:
Mapping[str, TimeIntervals]- 1x table per stim epoch with trials
- behavior performance table (each block an interval)
-
trials:
TimeIntervals- same as
intervals[DynamicRouting1]
- same as
-
invalid_times:
TimeIntervals -
acquisition:
Mapping[str, Any]raw data- if is_ephys:
- raw AP:
Mapping[str, ElectricalSeries] - raw LFP:
Mapping[str, ElectricalSeries]
- raw AP:
- if is_sync:
- lick_sensor_rising_edges:
Events - lick_sensor_falling_edges:
Events
- lick_sensor_rising_edges:
- if is_task:
- rewards:
Events
- rewards:
- if is_video:
- video frame times: 1x
Eventsper camera
- video frame times: 1x
- if is_ephys:
-
processing:
Mapping[str, Any]processed/filtered data- behavior:
Mapping[str, Any]- licks:
Events- from sync or stim file
- running_speed:
TimeSeries- from stim file, enhanced with sync info if available
- licks:
- ecephys:
Mapping[str, Any]
- behavior:
-
analysis:
Mapping[str, Any]derived data, results- if is_ephys:
- all_spike_histograms: 1x
TimeSeriesper probe - drift_maps:
ImageSeries
- all_spike_histograms: 1x
- if is_task:
- performance:
TimeIntervals
- performance:
- if is_ephys:
Todo:
- filtered LFP
- stimulus templates (vis, aud, opto)
- OptogeneticStimulusSite
- analysis -> RFMaps
- per-unit response metric for each stim modality
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
npc_sessions-0.0.285.tar.gz
(89.7 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file npc_sessions-0.0.285.tar.gz.
File metadata
- Download URL: npc_sessions-0.0.285.tar.gz
- Upload date:
- Size: 89.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
uv/0.11.28 {"installer":{"name":"uv","version":"0.11.28","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
d6a4e55e2b304275ba3563d6f09374c2713108a52f7b367c42e83e7065be6a0e
|
|
| MD5 |
51cf4f1fe264404aa3a7a8a6e05b2394
|
|
| BLAKE2b-256 |
b6a1cf514437da1c517abc3f98bef84f6b5841135bd49cf7be26077e5b154964
|
File details
Details for the file npc_sessions-0.0.285-py3-none-any.whl.
File metadata
- Download URL: npc_sessions-0.0.285-py3-none-any.whl
- Upload date:
- Size: 93.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
uv/0.11.28 {"installer":{"name":"uv","version":"0.11.28","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
98341243caef2a94a2c4c19362be7ab36f0c9b701321a58616e46762ac795b7e
|
|
| MD5 |
9e5ae8d606e7badb6579d4834e0f44b7
|
|
| BLAKE2b-256 |
b5b4b8af723c472851e14085731f85923e548319d81820438a66f6e4d308b1e7
|
