zombie-squirrel 0.15.1

Generated from aind-library-template

ZOMBIE Squirrel

zombie-squirrel is a set of one-line functions that handle the entire process of caching and retrieving data (and metadata) from AIND data assets.

In the background, the ZOMBIE squirrel repackages data/metadata into dataframes and stores them on S3 in a flat bucket, or in memory for testing.

Installation

pip install zombie-squirrel

Usage

Set backend

export FOREST_TYPE='S3'

Options are 'S3', 'MEMORY'.

Scurry (fetch) data

from zombie_squirrel import unique_project_names

project_names = unique_project_names()

Function	Description	Parameters
unique_project_names	Fetch unique project names from docdb
unique_subject_ids	Fetch unique subject IDs from docdb
asset_basics	Fetch basic asset metadata including modalities, projects, and subject info
source_data	Fetch source data references for derived records
raw_to_derived	Fetch mapping of raw records to their derived records
qc	Fetch QC dataframe for a single or multiple records	str or list[str]
custom	Store or retrieve a user-defined DataFrame by name	name: str, df: pd.DataFrame (optional)

Custom acorn

The custom function allows you to store and retrieve your own user-defined DataFrames in the cache by name. This requires write authentication to the active backend.

from zombie_squirrel import custom
import pandas as pd

df = pd.DataFrame({"col": [1, 2, 3]})
custom("my_data", df)

retrieved_df = custom("my_data")

Hide all the acorns

We run a nightly capsule on Code Ocean with this code to hide all acorns (not the custom ones).

from zombie_squirrel.sync import hide_acorns
hide_acorns()