omero_quay 1.0.0

Omero Data Import Export Queue

omero-quay

![Actions Status][actions-badge]

omero-quay is a microscopy data transport layer between data management tools.

Currently, it supports the iRODS — OMERO architecture built at France BioImaging fbi-omero.

Objectives

• Interact with data management tools APIs
• Trigger actions:
  • Create / Read / Update / Delete
  • Transport / Import
• Synch databases :warning:

Schema driven development

• manifest.yml defines the data schema in LinkML
• follows an Investigation Study Assay hierarchy
• maps APIs, objects ids, ACLs
• only transports "crates", not individual files

ZeroMQ

{ width="70%" }

• Message = json dump of a manifest instance
• Central brocker routes messages based on state

Example route:

{ width="60%" }

Install

This is early stage and only meant for developers.

For now, install from source.

1. Get and install a conda distribution. We prefer micromamba
2. Download omero-quay from this repository (as an archive or with git)
3. In the omero-quay directory:
   • create the conda environment conda env create -f environment.yml
   • activate it conda activate quay
   • install omero-quay in editable mode pip install -e .[dev]

For now, this software alone is not very useful, and works only within the omero-fbi frameworks. Work on setting up a docker compose view of this format is in the works. Look into the tests folder for examples.

Configuration

The configuration is described in quay.yml. On the client or server machine, adapt this file and copy it to /home/user/.config/quay.yml

Usage

Remote client

Install

• Install omero-quay (see above)
• add irods server IP to /etc/hosts
• Write the client configuration file and save it to $HOME/.config/quay.yml

---
irods:
  # This is the configuration for Montpellier
  IRODS_HOST: irods-mtp.omero-fbi.fr
ingest:
  POST_URL: https://mtp.omero-fbi.fr/post

Import data

(all paths below are relative to the omero-quay root directory where this README file resides)

• Put an excel file with the relevant data at the root of the directory you want to import

• Run the import client (make sure the quay environment is activated):

python scripts/desktop_cli.py other/path/to/your.xlsx path_in_irods

Service

On the OMERO server machine:

python scripts/run_service.py

Excel client

On a client machine

python scripts/excel_import.py

Autoingest with iRODS

TODO

Tests

In order to run tests, you need to boot up docker containers for the omero server, irods catalog provider and nfsrods.

This is achieved by running up.sh script in tests/containers

cd tests/containers
./up.sh

After some minutes and possible connection retries, you should have the containers running.

You can then run the tests from the root of the repo with pytest