Recip Slice

Recip Slice reconstructs physically scaled crystallographic sections from raw rotation electron-diffraction images processed with XDS. Each detector observation is converted to a scattering vector, unrotated into a fixed crystal frame, selected against one or more reciprocal-space slabs, and accumulated directly onto an orthonormal Cartesian grid in Å⁻¹.

The principal use is to generate interpretable hk0, h0l, 0kl, offset-layer, and diagonal reciprocal-space sections from a rotation series without first allocating a dense 3D reciprocal-space volume.

Main outputs

By default, reconstructions share one results directory; the result name is the filename prefix:

<XDS directory>/recip_slices/<result name>_*

The directory contains, for every selected plane:

• *_sum.npy: count-preserving accumulation;
• *_mean.npy: coverage-normalized intensity;
• *_geometry_coverage.npy: valid detector/rotation/plane sampling coverage, independent of intensity thresholding;
• *_signal_coverage.npy: sampling weight retained after signal selection;
• *_coverage.npy: deprecated compatibility alias for signal_coverage;
• *_qx.npy and *_qy.npy: Cartesian reciprocal-space coordinates;
• *_manifest.json: plane definitions, unit-cell and hkl grid-direction metadata, grid metadata, filenames, frame-selection provenance, and reconstruction diagnostics.

Each manifest plane also reports sampling diagnostics from geometry_coverage: covered and uncovered requested-grid fractions, contributing frame and detector-sample counts, and coverage binned by physical |q| in Å^-1. These metrics describe the measured reciprocal-space support; they do not interpolate or fill missing regions.

The saved arrays are never modified by display controls.

Installation

The same source package is used on Linux, Windows, and macOS. Published binary wheels contain the native exact-order sparse accumulator, so normal wheel installation does not require a compiler:

python -m pip install recip-slice

For development from a checkout:

python -m pip install -e .

PyQt5 is installed as a core dependency because the program is GUI-first. The legacy .[gui] extra remains accepted for compatibility.

Source installs attempt to compile the small native sparse bilinear accumulator from the shipped generated C source. The default build policy is RECIP_SLICE_NATIVE=auto: if compilation is unavailable, installation succeeds with the scientifically equivalent serial NumPy fallback. Performance-critical or release builds can set RECIP_SLICE_NATIVE=require to fail rather than silently fall back; RECIP_SLICE_NATIVE=disable explicitly requests the NumPy backend. Run with --verbose to see Sparse accumulator backend: native or numpy-serial; the same value is stored in the manifest.

On an offline machine with NumPy and the build dependencies already installed, python -m pip install -e . --no-build-isolation avoids fetching build dependencies. See docs/installation.md for Linux, Windows, macOS, and wheel release details.

fabio is required to read SMV detector frames. SciPy remains a core dependency for supported detector/noise-processing paths.

Command line

Reconstruct the three principal planes in one frame-reading pass:

recip-slice reconstruct /path/to/xds \
  --xparm GXPARM.XDS \
  --planes hk0 h0l 0kl \
  --q-max 1.0 \
  --dq 0.002 \
  --slice-thickness 0.008 \
  --central-beam-q-cutoff 0.02 \
  --intensity-cutoff 10 \
  --prefix sample_name \
  --output-dir /path/to/xds/recip_slices \
  --no-plot --verbose

By default the command reads NAME_TEMPLATE_OF_DATA_FRAMES and DATA_RANGE from XDS.INP; frame numbers come from the ? field in the XDS template. Raw-pixel processing does not apply EXCLUDE_DATA_RANGE, XDS region/pixel masks, or detector spatial distortion by default. Enable those explicitly with --apply-xds-frame-exclusions, --apply-xds-region-exclusions, and --apply-detector-distortion. DATA_RANGE remains the experimental image range in either mode; the preflight and GUI report whether that range is complete for the requested policy. Use --glob only for an explicit manual image selection. Manual paths are first matched through the XDS template; an incompatible filename must contain one unambiguous frame-number group, or the run stops rather than guessing from a run/suffix number. A normal run requires XDS.INP, a refined GXPARM.XDS or XPARM.XDS, and every frame required by the selected mode.

Noise reduction is disabled by default. To enable the CMOS-aware cleanup, add --noise-reduction cmos-robust; it removes an additive detector baseline, local charge-spread halos, isolated column streaks, and a conservative radial residual without averaging neighbouring reflection pixels. The cmos-mixed mode applies the same cleanup but restores 10% of the removed detector residual as a deterministic partial correction. Both modes leave the raw detector frames untouched.

SMV reads use a bounded four-worker read-ahead queue by default. On a fast local disk this can be tuned with --prefetch-workers N, or disabled with --no-prefetch when memory is constrained. Temporal plane-candidate reuse is disabled by default. It can be enabled explicitly with --temporal-plane-cache; --no-temporal-plane-cache forces the baseline selection path explicitly.

For a guided custom Miller-index plane, use --custom-plane with an optional offset and in-plane orientation. The offset is the right-hand side of the fractional equation and the slab thickness remains physical reciprocal-space distance:

recip-slice reconstruct /path/to/xds \
  --custom-plane 1 1 0 \
  --plane-name hhl_layer \
  --plane-offset-hkl 2 \
  --in-plane-basis 1 -1 0 0 0 1 \
  --q-max 1.0 --dq 0.002 --slice-thickness 0.008 \
  --no-plot

This reconstructs h + k = 2. The resolved Cartesian normal and orthonormal in-plane basis, equation, offset, units, and sampling settings are recorded in the result manifest and survive result loading. A custom plane is a single plane in this milestone slice; the GUI layer builder and offset-series controls remain planned work.

Open the lightweight viewer:

recip-slice view /path/to/result/folder --prefix sample_name

Open the Qt workspace for a new analysis session (the default when no subcommand is supplied):

recip-slice

Open the Qt workspace:

recip-slice gui /path/to/xds --prefix sample_name

From a checkout, the same GUI can be launched directly without installing the package in editable mode:

python run.py /path/to/xds --prefix sample_name

Interactive GUI launches print a short ASCII welcome banner with the current version and a pointer to recip-slice --help in the command window. The reconstruction, preflight, viewer, and JSON-reporting commands keep their output focused for scripts and pipelines.

GUI workflow

The workspace keeps a simple three-step route:

1. Data: select one parent folder. Every usable nested XDS dataset is indexed and can then be switched from a compact selector without another scan.
2. Planes: select principal, layer, or diagonal sections. All selected planes share one detector-frame pass.
3. Settings: review the resolution limit and workload; optional sampling and filtering controls remain collapsed for beginners.

The run/open actions remain visible from every step. Expert mode removes the guidance and exposes technical controls for a direct folder-to-run workflow. The right-hand workspace prioritises the reciprocal-space image; optional appearance and export controls are collapsed until requested.

Under Settings → Advanced settings, Run all indexed datasets… applies the current reconstruction settings to every usable XDS dataset in the selected folder. A confirmation summary appears first, followed by separate dataset- and detector-frame progress bars.

Recent experiment folders, named plane/settings presets, and display preferences are retained between sessions. Recent folders remain available from the Recent Projects menu, but a new GUI instance never selects the previous folder automatically. Saved reconstructions are selectable as individual runs, with provenance and quality summaries available without opening the process log. Re-running an occupied result name requires an explicit choice to create a timestamped version or replace the existing output.

Compare opens a second viewer for another indexed XDS dataset. Choose dataset… reuses the searchable Find… dataset browser and can browse to another folder. Saved slices are loaded automatically; if none are available, the GUI offers to reconstruct dataset B using the current A settings and shows a cancellable progress bar. Side-by-side and blink views work for any loaded pair; numerical difference and ratio views are available only when the plane and physical q-coordinate grids match exactly, so the GUI never performs hidden scientific resampling. Appearance controls apply to whichever pane was last clicked, allowing A and B to be adjusted independently; the active pane is indicated only by a coloured frame. Hovering either pane reports the nearest hkl coordinate and both intensities. Comparison figures can be exported for reporting.

Scientific scope

The output is geometrically precession-like, but it is not a simulation of a physical precession experiment. Intensities retain the sampling, detector response, dynamical scattering, and other limitations of the original rotation data. sum is the weighted sum of signal-selected detector intensities. mean is exactly sum / signal_coverage; it is not a crystallographic intensity correction. geometry_coverage describes valid detector/rotation/plane sampling before intensity thresholding, while signal_coverage describes samples retained after thresholding.

If XDS spatial correction tables (X-CORRECTIONS.cbf and Y-CORRECTIONS.cbf) or BKGPIX.cbf are present, they are detected and recorded, but they are not applied unless the corresponding opt-in flags are used. The manifest records both available and applied corrections/masks, the refined geometry actually used, frame selection, q axes, preflight warnings, and saved filenames. It also records detector samples considered, samples rejected by the central-beam and optional XDS masks, signal-threshold rejections, processed frames, and per-plane geometry/signal-covered fractions.

See docs/algorithm.md for the coordinate model, docs/architecture.md for the software design, docs/gui.md for the interaction model, and docs/reproducibility.md for reporting guidance. The prioritized, cross-platform feature plan is in docs/roadmap.md.

For an external transform check against indexed XDS observations, run:

recip-slice validate-spot /path/to/xds --xparm GXPARM.XDS

This requires an indexed SPOT.XDS beside the XDS inputs; the package does not fabricate experimental validation data. The diagnostic reports total, indexed, unindexed, and validated-indexed records separately; XDS (0, 0, 0) unindexed records are not included in hkl residuals. SPOT detector positions may be in XDS's spatially corrected coordinate system, so this check does not automatically enable detector distortion correction for reconstruction.

Testing

python tools/project_cycle.py check

PyQt5 is a core dependency. GUI integration tests run headlessly in CI with QT_QPA_PLATFORM=offscreen.

Project production cycle

Project housekeeping is separate from dataset housekeeping. From a checkout, run python tools/project_cycle.py release to validate the source, run tests, check every Python file with Ruff and mypy, quarantine known generated artefacts, build the wheel and source distribution, and verify that the canonical packaged style.qss and SVG icons are included in both packages. The standalone clean command is a dry run unless --apply is supplied; it never targets experiment folders or reconstruction results.