Skip to main content

tsaad — Time Series Analysis of Area Detectors

Calibration-free estimation of detector response statistics—absorbed quantum gain ($G$), cascade factor ($\gamma$), and pixel noise factor ($\psi$)—directly from diffraction rotation-scan datasets.

Overview

Following the detector response model of Waterman & Evans (2010), J. Appl. Cryst. 43, 1356–1371, the signal variance on an integrating or charge-sharing detector is governed by:

$$\sigma_p^2 = \bar{p} , G \gamma + N \psi$$

Local single-frame dispersion estimators (such as dials.estimate_gain) underestimate the true response product $G\gamma$ whenever a point-spread function (PSF) or charge-sharing kernel $K(\mathbf{u})$ introduces spatial correlations between neighboring pixels, damping single-pixel variance by the zero-lag PSF autocorrelation factor $S_2 = \alpha(\mathbf{0}) = \sum_{\mathbf{v}} K(\mathbf{v})^2 < 1$.

tsaad (Time Series Analysis of Area Detectors) treats a rotation scan as an ensemble of coupled pixel time series to separate smooth spatiotemporal background trends and transient outliers (Bragg reflections, cosmic-ray zingers) from stochastic detector fluctuations, recovering $(G, \gamma, \psi)$ and the spatial autocorrelation kernel $\alpha(\mathbf{u})$ via lag-summed spatial cross-covariance across intensity regimes.


Environment, Architecture & Long-Term Packaging

Standalone Design (No Dependency on DIALS)

tsaad is designed as a standalone package independent of DIALS so that users and external pipelines without a full DIALS installation can install and use it directly:

  • Core package (tsaad): Pure Python, depending only on numpy and scipy (with matplotlib optional for diagnostic plotting).
  • dxtbx integration (tsaad.dxtbx_loader): Optionally depends on dxtbx to read diffraction image formats, .expt files, ImageSets, and detector Panel models, and replicates the single-frame Kabsch local dispersion baseline (dials.estimate_gain) in pure NumPy without importing dials.
  • Future conda-forge & DIALS Integration: Because tsaad depends only on numpy, scipy, matplotlib, and (optionally) dxtbx—and never on dials—it can be packaged on conda-forge without circular dependencies. Downstream DIALS builds can then install tsaad from conda-forge and import it directly inside a replacement implementation of dials.estimate_gain.

Installing into a Standard Python Environment

python -m venv .venv
source .venv/bin/activate
pip install -e ".[plot,dev]"

Installing into an Existing DIALS Environment

If you have a DIALS installation (for example, a DIALS conda_base environment or developer build):

# Using the pip inside DIALS's conda_base environment:
/path/to/dials/conda_base/bin/pip install -e .

# Or, with a DIALS environment sourced (source /path/to/dials/dials_env.sh):
libtbx.pip install -e .

Simulator Usage (tsaad.simulator)

The tsaad.simulator module generates synthetic diffraction rotation-scan image stacks $\mathbf{Y}(\mathbf{x}, t)$ with exact ground-truth detector response statistics $(G, \gamma, \psi)$, 2D point-spread kernels $K(\mathbf{u})$, non-stationary incident beam flux $I_0(t)$, radially varying and rotating background scatter, transient 3D Bragg reflections, single-frame cosmic-ray zingers, and static bad/shadowed pixels.

1. Simulating a Rotation Scan with Built-in Detector Presets

Three benchmark detector presets are provided out of the box:

  • DetectorConfig.ideal_pad(): Ideal photon-counting Pixel Array Detector ($G = 1.0$, $\gamma = 1.0$, $\psi = 0.0$, $\delta$-PSF with $S_2 = 1.0$).
  • DetectorConfig.pt_ccd_waterman_evans(): Phosphor-taper-CCD parameterized after Waterman & Evans (2010) ($G = 1.44\text{ ADU}$, $\gamma = 1.54$, $\psi = 9.08\text{ ADU}^2$, Lorentzian PSF with $\text{FWHM} = 1.3\text{ px}$ and $S_2 \approx 0.177$).
  • DetectorConfig.ed_pad_timepix(): Hybrid PAD operated in 200 keV 3D electron diffraction mode after Vypritskaia et al. (2025) ($G = 2.9$, $\gamma = 1.35$, $\psi = 0.0$, discrete multi-pixel charge-sharing cluster PSF with $S_2 \approx 0.455$).
from tsaad.simulator import (
    DetectorConfig,
    SceneConfig,
    simulate_rotation_scan,
)

# 1. Choose a detector preset (or customize parameters)
detector = DetectorConfig.pt_ccd_waterman_evans(
    gain=1.44,
    cascade_factor=1.54,
    pedestal=50.0,
)

# 2. Configure the rotation-scan scene
scene = SceneConfig(
    n_frames=60,
    height=128,
    width=128,
    bg_min_quanta=2.0,          # Minimum background (absorbed quanta / pixel / frame)
    bg_max_quanta=60.0,         # Maximum background near beam center / solvent ring
    beam_drift_amplitude=0.05,  # 5% slow sinusoidal drift across scan
    beam_topup_period=25,       # Synchrotron top-up sawtooth period (frames)
    beam_topup_jump=0.04,       # 4% jump at top-up injection
    beam_jitter_sigma=0.01,     # 1% frame-to-frame random source jitter
    n_bragg_spots=80,           # Number of 3D Gaussian Bragg reflections
    n_zingers=15,               # Single-frame cosmic-ray spikes
    n_hot_pixels=10,
    n_dead_pixels=10,
    shadow_margin_px=4,
)

# 3. Run the simulation
result = simulate_rotation_scan(detector, scene, seed=42)

print("Image stack shape (T, H, W):", result.images.shape)
print("True G * gamma:", detector.g_gamma)
print("Zero-lag PSF factor S_2:", detector.s2_factor)
print("Apparent single-pixel dispersion slope:", detector.apparent_single_pixel_gain)

2. Inspecting the SimulationResult

simulate_rotation_scan returns a SimulationResult dataclass containing both the observed image stack and the exact ground-truth components for benchmarking estimators:

Attribute Shape / Type Description
result.images (n_frames, height, width) float64 Observed pixel intensities $Y(y, x, t)$ in ADU (including pedestal, noise, spots, zingers, and bad pixels).
result.expected_background_adu (n_frames, height, width) float64 True noiseless expected background signal $\mu(y, x, t)$ in ADU above pedestal (after PSF blurring).
result.incident_flux_scale (n_frames,) float64 True frame-by-frame global incident beam scale factor $I_0(t)$ (normalized to unit mean).
result.spot_mask (n_frames, height, width) bool Ground-truth 3D mask marking pixels contaminated by Bragg reflections or cosmic-ray zingers.
result.static_bad_mask (height, width) bool Ground-truth 2D mask marking static hot pixels, dead pixels, and shadowed detector borders.

3. Customizing Detector Response & PSF Kernels

You can construct custom detectors with analytical or empirical 2D PSF kernels (delta_psf, gaussian_psf, lorentzian_psf, timepix_cluster_psf, or any normalized 2D NumPy array with odd dimensions) and query their theoretical spatial autocovariance:

from tsaad.simulator import DetectorConfig, gaussian_psf

custom_detector = DetectorConfig(
    name="custom_cmos",
    gain=2.2,
    cascade_factor=1.25,
    pixel_factor=3.5,
    pedestal=100.0,
    psf_kernel=gaussian_psf(sigma_px=0.85, radius=3),
    psf_mode="continuous",  # or "discrete_cluster" for integer cluster splitting
)

# Normalized 2D PSF autocorrelation alpha(u) = (K * K)(u)
alpha = custom_detector.psf_autocorrelation

# Exact theoretical 2D autocovariance matrix C(u; mu) at expected signal mu = 50 ADU
cov_matrix = custom_detector.theoretical_covariance_kernel(mu_adu=50.0)

Estimating Detector Response Statistics (tsaad.pipeline)

Use estimate_detector_statistics to run the complete four-stage pipeline (anomaly masking $\rightarrow$ spatiotemporal detrending $\rightarrow$ lagged cross-covariance regression $\rightarrow$ $G$/$\gamma$ separation) on any 3D rotation-scan image stack (n_frames, height, width):

from tsaad import (
    DetectorConfig,
    SceneConfig,
    estimate_detector_statistics,
    plot_detector_stats_report,
    simulate_rotation_scan,
)

# Simulate a 3D ED Timepix rotation scan with a low-flux corner
det = DetectorConfig.ed_pad_timepix(gain=2.9, cascade_factor=1.35)
scene = SceneConfig(n_frames=60, height=128, width=128, include_sparse_corner=True)
sim = simulate_rotation_scan(det, scene, seed=42)

# Estimate (G, gamma, psi) directly from the image stack
report = estimate_detector_statistics(
    sim.images,
    pedestal=det.pedestal,
    trend_method="hollow_stencil_kriging",  # or "low_rank_spline", "temporal_difference"
)

print(f"Estimated G * gamma : {report.g_gamma:.3f} +- {report.covariance.g_gamma_se:.3f}")
print(f"Estimated gain G    : {report.gain:.3f} (method: {report.separation.separation_method})")
print(f"Estimated cascade γ : {report.cascade_factor:.3f}")
print(f"Estimated noise ψ   : {report.psi:.3f} +- {report.covariance.psi_se:.3f}")
print(f"Detected PSF radius : {report.psf_radius} px (S_2 = {report.s2_factor:.3f})")
print(f"Quality reliable    : {report.quality.is_reliable} (warnings: {report.quality.warnings})")

# Save an 8-panel diagnostic summary figure
plot_detector_stats_report(
    report,
    output_path="diagnostic_report.png",
    title="3D ED Timepix Rotation Scan",
    true_detector=det,
    true_flux=sim.incident_flux_scale,
)

Command-Line Interface (tsaad)

After installing the package, run tsaad on a .npy / .npz 3D stack, a dxtbx .expt file, or diffraction image files/directories:

tsaad /path/to/scan_or_stack \
    --max-frames 80 \
    --json summary.json \
    --plot diagnostic_report.png

Running Tests

Run the test suite with pytest:

# Using a standard Python environment:
pytest

# Or using DIALS's conda_base pytest (includes dxtbx & DIALS dispersion baseline tests):
/path/to/dials/conda_base/bin/pytest -v

License

tsaad is distributed under the terms of the BSD 3-Clause License.

Metadata

Release files for tsaad 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for tsaad 0.1.0
File Size Uploaded Provenance Actions
tsaad-0.1.0.tar.gz 79.7 kB With provenance Details for tsaad-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for tsaad 0.1.0
File Interpreter ABI Platform Actions
tsaad-0.1.0-py3-none-any.whl Python 3 none any Details for tsaad-0.1.0-py3-none-any.whl

Total release size: 145.3 kB

Release files / tsaad-0.1.0.tar.gz

Download URL tsaad-0.1.0.tar.gz
Size 79.7 kB
Tags Source
SHA-256 checksum
How to use checksums
cce7cad171603528bf861a266d881b09e10e890a37d0e8a11906e363e06c43e5
BLAKE2b-256 checksum
How to use checksums
467b52484ad1d7ba760960f42701431447dca8790fa6c2b4c758bcedfbfa0afd
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 12, 2026.

Transparency log

Release files / tsaad-0.1.0-py3-none-any.whl

Download URL tsaad-0.1.0-py3-none-any.whl
Size 65.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
1745fac99452dba7c4dc200d820d08d53d73100322dd5342f1aacbf6f79b417c
BLAKE2b-256 checksum
How to use checksums
47bc656f69974fb4aa822fbe56a37cb7ec82a3cd1f1c644069b0003c1ffb52cc
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 12, 2026.

Transparency log

Security for tsaad 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for tsaad-0.1.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 12, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page