uval 0.2.2

UVal - Unified eValuation framework for 3D X-ray data

This python package is meant to provide a high level interface to facilitate the evaluation of object detection and segmentation algorithms that operate on 3D volumetric data.

• There is a growing need for high performance detection algorithms using 3D data, and it is very important to be able to compare them. By far, there has not been a trivial solution for a straightforward comparison between different 3D detection algorithms.
• This framework seeks a way to address the aforementioned problem by introducing a simple and standard layout of the popular HDF5 data format as input.
• Each detection algorithm can export the results and groundtruth data according to the defined layout principles. Then, UVal can evaluate the performance and come up with common comparison metrics.

Installation (non-development)

If you are not developing and only using UVal, you can simply install it as a pypi package (requires Python 3.8 or higher); simply run:

pip install uval

If you would like to have UVal installation to be independant of a specific python environment, simply use pipx instead of pip.

To run the code you can type:

uval --config-file ${workspaceFolder}/output/example/config.yaml

For an example of the outputs see here and the report here.

For the details of each entry in the config file please see here.

Development setup

• First, please clone the UVal's git repository by executing the following command:
  git clone https://gitlab.com/smithsdetection/uval.git

• You will need a python >= 3.8 environment to develop Uval.
  We recommend using Anaconda due to its ease of use:

  curl -L -O "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
  bash Miniforge3-$(uname)-$(uname -m).sh
  

  For x86_64(amd64) use "Miniforge3-Linux-x86_64". For other operating systems see here.

• Close and reopen your terminal session.

• Setting up a conda virtual environment with poetry using the following commands:

  mamba env create -f environment.yml
  mamba activate UVALENV
  poetry install
  pre-commit install
  

  Alternatively, you can create your own conda environment from scratch. and follow with poetry and pre-commit installations.

Example code

• A step-by-step walkthrough for reading and evaluating data with the UVal is available as a jupyter document:
  • jupyter notebook demo

• Hint: Prior to running the demo jupyter notebook walkthrough, the following steps must be performed:

  • The ipykernel conda package must be installed

    conda install -c anaconda ipykernel
    

  • The uvalenv environment must be added as an ipykernel:

    python3 -m ipykernel install --user --name uvalenv --display-name "uvalenv Python38"
    

  • The uvalenv Python38 kernel, which includes all the required python packages must be selected in jupyter environment to run the code.

Documentations

Read the docs: https://uval.readthedocs.io/

Release History

• 0.1.x
  • The first ready-to-use version of UVal releases publicly

Meta

Smiths Detection – @Twitter – uval@smithsdetection.com

UVal is released under the GPL V3.0 license.

Contributing

1. Fork it (https://gitlab.com/smithsdetection/uval/fork)
2. Create your feature branch (git checkout -b feature/fooBar)
3. Commit your changes (git commit -am 'Add some fooBar')
4. Push to the branch (git push origin feature/fooBar)
5. Create a new merge Request

Citing UVal

If you use UVal in your research or wish to refer to the results, please use the following BibTeX entry.

@misc{smithsdetection2022uval,
  author =       {Philipp Fischer, Geert Heilmann, Mohammad Razavi, Faraz Saeedan},
  title =        {UVal},
  howpublished = {\url{https://gitlab.com/smithsdetection/uval}},
  year =         {2022}
}