morphology-workflows 0.3.0

Workflows used for morphology processing.

Morphology Workflows

This project contains several workflows for processing morphologies:

• Fetch: download morphologies from online database (like Allen, NeuroMorpho or MouseLight).
• Curate: from raw morphologies, ensures that morphologies can be used with the rest of BBP codes
• Annotate: create various annotations on morphologies needed by specific BBP codes
• Repair: process morphologies to correct for artifacts of in-vitro reconstruction

In a nutshell, the user provides a list of morphologies in a .csv file, with their names and paths and a luigi.cfg configuration file. Each workflow is run independently and creates an output folder, with one subfolder per task. In each, there will be a report.csv and a data folder containing the output files of the task if any. In the report.csv file, columns contain paths to these files, additional information, error messages if the task failed on that morphologies, as well as a flag is_valid, used in subsequent tasks to filter valid morphologies. At the end of each workflow, another report.csv file is created, with the main output columns of each tasks, and a report.pdf containing a human readable summary of the result of the workflow.

The complete documentation can be found here:

• stable: https://morphology-workflows.readthedocs.io/en/stable/
• latest: https://morphology-workflows.readthedocs.io/en/latest/

Installation

This should be installed using pip:

pip install morphology-workflows

Usage

This workflow is based on the luigi library but can be run via the command line interface:

morphology_workflows --local-scheduler Curate

NOTE This command must be executed from a directory containing a luigi.cfg file. An example of such file is given in the examples directory.

More details can be found in the command line interface section of the documentation.

Examples

The examples folder contains a simple example that will process a set of morphologies. A dataset.csv file is provided which is taken as input for the workflows. A luigi.cfg file is also provided to give a default configuration for the workflows. This example can simply be run using the following command:

./run_curation.sh

This script will create a new directory out_curated which will contain the report and all the results.

Funding & Acknowledgment

The development of this software was supported by funding to the Blue Brain Project, a research center of the École polytechnique fédérale de Lausanne (EPFL), from the Swiss government’s ETH Board of the Swiss Federal Institutes of Technology.

For license and authors, see LICENSE.txt and AUTHORS.md respectively.

Copyright © 2021-2022 Blue Brain Project/EPFL