Installation

Processing scripts require Python 3.6 or higher.

You can install stable version by:

pip install spi_em_class

To install from git, clone the repository and run commands:

cd spi_em_class
pip install .

File format

Input data should be in CXI format. The following internal structure is expected:

file.cxi 
   ├─ cxi_version
   └─ entry_1
       ├─ image_1
       │   ├─ data (N, y, x)
       │   ├─ mask (y, x)
       │   └─ image_center (3) [x, y, z]
       │
       ├─ image_2
       │   ├─ data
       │   ├─ mask
       │   └─ image_center
...
       └─ image_M
           ├─ data
           ├─ mask
           └─ image_center

Each image_k group should have data dataset with images, mask dataset indicate bad pixels and image_center dataset contain coordinates of image center in pixels. Other groups and datasets are optional.

As a result new 1D dataset in each image_k group will be created with length equal to first dimention of data.

EM classification

To classify diffraction images via Expectation-Maximization algorithm please perform the following steps:

Create new EM classification

Create blank data file for a new EM classification. A data file is HDF5 archive. It includes EM configuration, temporary data and EM results. You must provide path for CXI file with input images.

spi_em_create.py [-h] [-d DATA] FILE

Options:

• FILE - CXI file with input images.
• -d DATA - path to a new data file. Default: "em_class_data.h5".

Configure EM classification

You can change configuration of EM classification. There are two possible ways:

1. Edit configuration as text file.
2. Provide parameters in command line.

If you change parameters for EM classification with finished iterations, you may need to reset classification.

spi_em_config.py [-h] [-d DATA] [--cxi FILE] [--q_min Q_MIN]
                 [--q_max Q_MAX] [--num_rot NUM_ROT]
                 [--num_class NUM_CLASS] [--friedel] [--no-friedel]
                 [--logscale] [--no-logscale] [--best] [--no-best]
                 [--binning BINNING] [-s] [-e]

Options:

• -d DATA - path to the data file. Default: "em_class_data.h5".
• -e - open configuration editor. This is default behavior when spi_em_config.py is run without arguments. You can set your preferred editor via EDITOR environment value.
• -s - print current configuration and exit.

Parameter	CMD option	Description
cxi_file	--cxi FILE	Path to CXI file with input images
q_min	--q_min Q_MIN	Minimum q-radius used in classification (pixels)
q_max	--q_max Q_MAX	Maximum q-radius used in classification (pixels)
num_rot	--num_rot NUM_ROT	Number of considered rotation angles
num_class	--num_class NUM_CLASS	Number of EM classes
friedel	--friedel OR --no-friedel	Force central symmetry (Friedel's law)
logscale	--logscale OR --no-logscale	Apply log-scaling to input images
best_proba	--best OR --no-best	Consider only one orientation with best probability for each frame
binning	--binning BIN	Bin input frames (Combine pixels in BIN*BIN groups together)

Run EM classification

To start EM classification please run:

spi_em_run.py [-h] [-d DATA] iter

If there are finished EM iterations, EM process will continue from last iteration.

Options:

• -d DATA - path to the data file. Default: "em_class_data.h5".
• iter - number of EM iterations to perform.

Reset EM classification

You may need to reset finished EM iterations and delete temporary data. Please run:

spi_em_reset.py [-h] [-d DATA]

Options:

• -d DATA - path to the data file. Default: "em_class_data.h5".

Create EM classification report

To check EM results, you can create PDF report. Please run:

spi_em_report.py [-h] [-o OUT] [-d DATA]

Options:

• -d DATA - path to the data file. Default: "em_class_data.h5".
• -o OUT - path to a new PDF file. Default: "em_class_report.pdf".

Save EM classification (manually)

You can save result of last EM iteration into CXI file as a 1D dataset along with input images. By default, results are added to the input CXI file.

spi_em_save.py [-h] [-o OUT] [-c CLASS_DSET] [-s SELECT_DSET] [-d DATA]
               select_class [select_class ...]

Options:

• -d DATA - path to the data file. Default: "em_class_data.h5".
• -o OUT - path to a output CXI file. It is a copy of input CXI file with classification data.
• select_class - Numbers for selected classes. Result dataset will contain 1 for images in selected classes and 0 for other images.
• -s SELECT_DSET - name of a selection dataset within image_n groups in CXI file. Default: "em_class/select".
• -c CLASS_DSET - name of a class distribution dataset within image_n groups in CXI file. This dataset will contain class number for every image. Default: "em_class/classes".

Save EM classification by symmetry of class models.

You can select EM classes automatically by analisys of model symmetry. First, symmetry scores are computed for first 50 symmetry orders by Discrete Cosine Transform of model autocorrelation in polar coordinates. Second, symmetry value is a ratio between symmetry score for selected symmetry order and sum of all symmetry scores. Symmetry value lies in [0,1]. Finally, if symmetry value > threshold, class is selected for saving.

spi_em_save_by_symmetry.py [-h] [-t THRESHOLD] [-o OUT] [-c CLASS_DSET]
                                [-s SELECT_DSET] [-d DATA] order

Options:

• -d DATA - path to the data file. Default: "em_class_data.h5".
• -o OUT - path to a output CXI file. It is a copy of input CXI file with classification data.
• order - Symmetry order that is used for selection.
• -t THRESHOLD - threshold for symmetry values. Default: 0.5.
• -s SELECT_DSET - name of a selection dataset within image_n groups in CXI file. Default: "em_class/select".
• -c CLASS_DSET - name of a class distribution dataset within image_n groups in CXI file. This dataset will contain class number for every image. Default: "em_class/classes".