np_codeocean
Tools for uploading Mindscope Neuropixels experiments to S3 (for Code Ocean).
Requires running as admin on Windows in order to create remote-to-remote symlinks on the Isilon.
Development workflow
- clone the repo, or pull with rebase
- sync the development environment using
uv(see below) - push changes to main (always commit changes to
uv.lock) - github action formats and publishes a new version
- pull the bump commit
Install
Setup/sync the development environment for working on a specific project:
uv sync --extra <dynamicrouting|openscope>
This ensures all developers are using the same package versions.
Add dependencies
For shared utilities:
uv add <package-name>
For project-specific utilities (added to optional dependency groups):
uv add <package-name> --optional <dynamicrouting|openscope>
Update dependencies
All:
uv lock --upgrade
Single package:
uv lock --upgrade-package <package-name>
Usage
-
uploadCLI tool is provided, which uses thenp_sessioninterface to find and upload raw data for one ecephys session:pip install np_codeocean upload <session-id>where session-id is any valid input to
np_session.Session(), e.g.:- a lims ID (
1333741475) - a workgroups foldername (
DRPilot_366122_20230101) - a path to a session folder (
\\allen\programs\mindscope\workgroups\np-exp\1333741475_719667_20240227)
- a lims ID (
-
a folder of symlinks pointing to the raw data is created, with a new structure suitable for the KS2.5 sorting pipeline on Code Ocean
-
the symlink folder, plus metadata, are entered into a csv file, which is submitted to
http://aind-data-transfer-service, which in turn runs theaind-data-transfertool on the HPC, which follows the symlinks to the original data, median-subtracts/scales/compresses ephys data, then uploads with the AWS CLI tool -
all compression/zipping acts on copies in temporary folders: the original raw data is not altered in anyway
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file np_codeocean-0.3.19.tar.gz.
File metadata
- Download URL: np_codeocean-0.3.19.tar.gz
- Upload date:
- Size: 25.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
uv/0.12.5 {"installer":{"name":"uv","version":"0.12.5","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
5fe691af0d2462b97e30eafe9da78c4093dfbedf34b495576a3ca98ba4894922
|
|
| MD5 |
9041d5422a11392fe563ba1278641ac1
|
|
| BLAKE2b-256 |
5bf85e5ee13bf4cfd8fd440891a846304287fc7c5cdd46c9e3a439df254de6cd
|
File details
Details for the file np_codeocean-0.3.19-py3-none-any.whl.
File metadata
- Download URL: np_codeocean-0.3.19-py3-none-any.whl
- Upload date:
- Size: 25.8 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
uv/0.12.5 {"installer":{"name":"uv","version":"0.12.5","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
3277fef18fb2c412e92f1746d93f28efce9aa8717a82b66d4dd872a26e797633
|
|
| MD5 |
4b1c818f8bbf8f79785b9e9ef877eaca
|
|
| BLAKE2b-256 |
e9ad2b8b4cb97e063411828d41179ff6d3ff09906c51ef03820989cdfe008dc9
|
