np_codeocean
Tools for uploading Mindscope Neuropixels experiments to S3 (for Code Ocean).
Requires running as admin on Windows in order to create remote-to-remote symlinks on the Isilon.
Development workflow
- clone the repo, or pull with rebase
- sync the development environment using
uv(see below) - push changes to main (always commit changes to
uv.lock) - github action formats and publishes a new version
- pull the bump commit
Install
Setup/sync the development environment for working on a specific project:
uv sync --extra <dynamicrouting|openscope>
This ensures all developers are using the same package versions.
Add dependencies
For shared utilities:
uv add <package-name>
For project-specific utilities (added to optional dependency groups):
uv add <package-name> --optional <dynamicrouting|openscope>
Update dependencies
All:
uv lock --upgrade
Single package:
uv lock --upgrade-package <package-name>
Usage
-
uploadCLI tool is provided, which uses thenp_sessioninterface to find and upload raw data for one ecephys session:pip install np_codeocean upload <session-id>where session-id is any valid input to
np_session.Session(), e.g.:- a lims ID (
1333741475) - a workgroups foldername (
DRPilot_366122_20230101) - a path to a session folder (
\\allen\programs\mindscope\workgroups\np-exp\1333741475_719667_20240227)
- a lims ID (
-
a folder of symlinks pointing to the raw data is created, with a new structure suitable for the KS2.5 sorting pipeline on Code Ocean
-
the symlink folder, plus metadata, are entered into a csv file, which is submitted to
http://aind-data-transfer-service, which in turn runs theaind-data-transfertool on the HPC, which follows the symlinks to the original data, median-subtracts/scales/compresses ephys data, then uploads with the AWS CLI tool -
all compression/zipping acts on copies in temporary folders: the original raw data is not altered in anyway
1 maintainer
Metadata
1 maintainer
Release files for np_codeocean 0.3.19
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| np_codeocean-0.3.19.tar.gz | 25.0 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| np_codeocean-0.3.19-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 50.7 kB
Release files / np_codeocean-0.3.19.tar.gz
| Download URL | np_codeocean-0.3.19.tar.gz |
|---|---|
| Size | 25.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
5fe691af0d2462b97e30eafe9da78c4093dfbedf34b495576a3ca98ba4894922
|
|
BLAKE2b-256 checksum How to use checksums |
5bf85e5ee13bf4cfd8fd440891a846304287fc7c5cdd46c9e3a439df254de6cd
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.5 {"installer":{"name":"uv","version":"0.12.5","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / np_codeocean-0.3.19-py3-none-any.whl
| Download URL | np_codeocean-0.3.19-py3-none-any.whl |
|---|---|
| Size | 25.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
3277fef18fb2c412e92f1746d93f28efce9aa8717a82b66d4dd872a26e797633
|
|
BLAKE2b-256 checksum How to use checksums |
e9ad2b8b4cb97e063411828d41179ff6d3ff09906c51ef03820989cdfe008dc9
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.5 {"installer":{"name":"uv","version":"0.12.5","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
