tsaad — Time Series Analysis of Area Detectors
Calibration-free estimation of detector response statistics—absorbed quantum gain ($G$), cascade factor ($\gamma$), and pixel noise factor ($\psi$)—directly from diffraction rotation-scan datasets.
Overview
Following the detector response model of Waterman & Evans (2010), J. Appl. Cryst. 43, 1356–1371, the signal variance on an integrating or charge-sharing detector is governed by:
$$\sigma_p^2 = \bar{p} , G \gamma + N \psi$$
Local single-frame dispersion estimators (such as dials.estimate_gain) underestimate the true response product $G\gamma$ whenever a point-spread function (PSF) or charge-sharing kernel $K(\mathbf{u})$ introduces spatial correlations between neighboring pixels, damping single-pixel variance by the zero-lag PSF autocorrelation factor $S_2 = \alpha(\mathbf{0}) = \sum_{\mathbf{v}} K(\mathbf{v})^2 < 1$.
tsaad (Time Series Analysis of Area Detectors) treats a rotation scan as an ensemble of coupled pixel time series to separate smooth spatiotemporal background trends and transient outliers (Bragg reflections, cosmic-ray zingers) from stochastic detector fluctuations, recovering $(G, \gamma, \psi)$ and the spatial autocorrelation kernel $\alpha(\mathbf{u})$ via lag-summed spatial cross-covariance across intensity regimes.
Environment, Architecture & Long-Term Packaging
Standalone Design (No Dependency on DIALS)
tsaad is designed as a standalone package independent of DIALS so that users and external pipelines without a full DIALS installation can install and use it directly:
- Core package (
tsaad): Pure Python, depending only onnumpyandscipy(withmatplotliboptional for diagnostic plotting). dxtbxintegration (tsaad.dxtbx_loader): Optionally depends ondxtbxto read diffraction image formats,.exptfiles,ImageSets, and detectorPanelmodels, and replicates the single-frame Kabsch local dispersion baseline (dials.estimate_gain) in pure NumPy without importingdials.- Future
conda-forge& DIALS Integration: Becausetsaaddepends only onnumpy,scipy,matplotlib, and (optionally)dxtbx—and never ondials—it can be packaged onconda-forgewithout circular dependencies. Downstream DIALS builds can then installtsaadfromconda-forgeand import it directly inside a replacement implementation ofdials.estimate_gain.
Installing into a Standard Python Environment
python -m venv .venv
source .venv/bin/activate
pip install -e ".[plot,dev]"
Installing into an Existing DIALS Environment
If you have a DIALS installation (for example, a DIALS conda_base environment or developer build):
# Using the pip inside DIALS's conda_base environment:
/path/to/dials/conda_base/bin/pip install -e .
# Or, with a DIALS environment sourced (source /path/to/dials/dials_env.sh):
libtbx.pip install -e .
Simulator Usage (tsaad.simulator)
The tsaad.simulator module generates synthetic diffraction rotation-scan image stacks $\mathbf{Y}(\mathbf{x}, t)$ with exact ground-truth detector response statistics $(G, \gamma, \psi)$, 2D point-spread kernels $K(\mathbf{u})$, non-stationary incident beam flux $I_0(t)$, radially varying and rotating background scatter, transient 3D Bragg reflections, single-frame cosmic-ray zingers, and static bad/shadowed pixels.
1. Simulating a Rotation Scan with Built-in Detector Presets
Three benchmark detector presets are provided out of the box:
DetectorConfig.ideal_pad(): Ideal photon-counting Pixel Array Detector ($G = 1.0$, $\gamma = 1.0$, $\psi = 0.0$, $\delta$-PSF with $S_2 = 1.0$).DetectorConfig.pt_ccd_waterman_evans(): Phosphor-taper-CCD parameterized after Waterman & Evans (2010) ($G = 1.44\text{ ADU}$, $\gamma = 1.54$, $\psi = 9.08\text{ ADU}^2$, Lorentzian PSF with $\text{FWHM} = 1.3\text{ px}$ and $S_2 \approx 0.177$).DetectorConfig.ed_pad_timepix(): Hybrid PAD operated in 200 keV 3D electron diffraction mode after Vypritskaia et al. (2025) ($G = 2.9$, $\gamma = 1.35$, $\psi = 0.0$, discrete multi-pixel charge-sharing cluster PSF with $S_2 \approx 0.455$).
from tsaad.simulator import (
DetectorConfig,
SceneConfig,
simulate_rotation_scan,
)
# 1. Choose a detector preset (or customize parameters)
detector = DetectorConfig.pt_ccd_waterman_evans(
gain=1.44,
cascade_factor=1.54,
pedestal=50.0,
)
# 2. Configure the rotation-scan scene
scene = SceneConfig(
n_frames=60,
height=128,
width=128,
bg_min_quanta=2.0, # Minimum background (absorbed quanta / pixel / frame)
bg_max_quanta=60.0, # Maximum background near beam center / solvent ring
beam_drift_amplitude=0.05, # 5% slow sinusoidal drift across scan
beam_topup_period=25, # Synchrotron top-up sawtooth period (frames)
beam_topup_jump=0.04, # 4% jump at top-up injection
beam_jitter_sigma=0.01, # 1% frame-to-frame random source jitter
n_bragg_spots=80, # Number of 3D Gaussian Bragg reflections
n_zingers=15, # Single-frame cosmic-ray spikes
n_hot_pixels=10,
n_dead_pixels=10,
shadow_margin_px=4,
)
# 3. Run the simulation
result = simulate_rotation_scan(detector, scene, seed=42)
print("Image stack shape (T, H, W):", result.images.shape)
print("True G * gamma:", detector.g_gamma)
print("Zero-lag PSF factor S_2:", detector.s2_factor)
print("Apparent single-pixel dispersion slope:", detector.apparent_single_pixel_gain)
2. Inspecting the SimulationResult
simulate_rotation_scan returns a SimulationResult dataclass containing both the observed image stack and the exact ground-truth components for benchmarking estimators:
| Attribute | Shape / Type | Description |
|---|---|---|
result.images |
(n_frames, height, width) float64 |
Observed pixel intensities $Y(y, x, t)$ in ADU (including pedestal, noise, spots, zingers, and bad pixels). |
result.expected_background_adu |
(n_frames, height, width) float64 |
True noiseless expected background signal $\mu(y, x, t)$ in ADU above pedestal (after PSF blurring). |
result.incident_flux_scale |
(n_frames,) float64 |
True frame-by-frame global incident beam scale factor $I_0(t)$ (normalized to unit mean). |
result.spot_mask |
(n_frames, height, width) bool |
Ground-truth 3D mask marking pixels contaminated by Bragg reflections or cosmic-ray zingers. |
result.static_bad_mask |
(height, width) bool |
Ground-truth 2D mask marking static hot pixels, dead pixels, and shadowed detector borders. |
3. Customizing Detector Response & PSF Kernels
You can construct custom detectors with analytical or empirical 2D PSF kernels (delta_psf, gaussian_psf, lorentzian_psf, timepix_cluster_psf, or any normalized 2D NumPy array with odd dimensions) and query their theoretical spatial autocovariance:
from tsaad.simulator import DetectorConfig, gaussian_psf
custom_detector = DetectorConfig(
name="custom_cmos",
gain=2.2,
cascade_factor=1.25,
pixel_factor=3.5,
pedestal=100.0,
psf_kernel=gaussian_psf(sigma_px=0.85, radius=3),
psf_mode="continuous", # or "discrete_cluster" for integer cluster splitting
)
# Normalized 2D PSF autocorrelation alpha(u) = (K * K)(u)
alpha = custom_detector.psf_autocorrelation
# Exact theoretical 2D autocovariance matrix C(u; mu) at expected signal mu = 50 ADU
cov_matrix = custom_detector.theoretical_covariance_kernel(mu_adu=50.0)
Estimating Detector Response Statistics (tsaad.pipeline)
Use estimate_detector_statistics to run the complete four-stage pipeline (anomaly masking $\rightarrow$ spatiotemporal detrending $\rightarrow$ lagged cross-covariance regression $\rightarrow$ $G$/$\gamma$ separation) on any 3D rotation-scan image stack (n_frames, height, width):
from tsaad import (
DetectorConfig,
SceneConfig,
estimate_detector_statistics,
plot_detector_stats_report,
simulate_rotation_scan,
)
# Simulate a 3D ED Timepix rotation scan with a low-flux corner
det = DetectorConfig.ed_pad_timepix(gain=2.9, cascade_factor=1.35)
scene = SceneConfig(n_frames=60, height=128, width=128, include_sparse_corner=True)
sim = simulate_rotation_scan(det, scene, seed=42)
# Estimate (G, gamma, psi) directly from the image stack
report = estimate_detector_statistics(
sim.images,
pedestal=det.pedestal,
trend_method="hollow_stencil_kriging", # or "low_rank_spline", "temporal_difference"
)
print(f"Estimated G * gamma : {report.g_gamma:.3f} +- {report.covariance.g_gamma_se:.3f}")
print(f"Estimated gain G : {report.gain:.3f} (method: {report.separation.separation_method})")
print(f"Estimated cascade γ : {report.cascade_factor:.3f}")
print(f"Estimated noise ψ : {report.psi:.3f} +- {report.covariance.psi_se:.3f}")
print(f"Detected PSF radius : {report.psf_radius} px (S_2 = {report.s2_factor:.3f})")
print(f"Quality reliable : {report.quality.is_reliable} (warnings: {report.quality.warnings})")
# Save an 8-panel diagnostic summary figure
plot_detector_stats_report(
report,
output_path="diagnostic_report.png",
title="3D ED Timepix Rotation Scan",
true_detector=det,
true_flux=sim.incident_flux_scale,
)
Command-Line Interface (tsaad)
After installing the package, run tsaad on a .npy / .npz 3D stack, a dxtbx .expt file, or diffraction image files/directories:
tsaad /path/to/scan_or_stack \
--max-frames 80 \
--json summary.json \
--plot diagnostic_report.png
Running Tests
Run the test suite with pytest:
# Using a standard Python environment:
pytest
# Or using DIALS's conda_base pytest (includes dxtbx & DIALS dispersion baseline tests):
/path/to/dials/conda_base/bin/pytest -v
License
tsaad is distributed under the terms of the BSD 3-Clause License.
Metadata
Release files for tsaad 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| tsaad-0.1.0.tar.gz | 79.7 kB | With provenance | Details for tsaad-0.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| tsaad-0.1.0-py3-none-any.whl | Python 3 | none | any | Details for tsaad-0.1.0-py3-none-any.whl |
Total release size: 145.3 kB
Release files / tsaad-0.1.0.tar.gz
| Download URL | tsaad-0.1.0.tar.gz |
|---|---|
| Size | 79.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
cce7cad171603528bf861a266d881b09e10e890a37d0e8a11906e363e06c43e5
|
|
BLAKE2b-256 checksum How to use checksums |
467b52484ad1d7ba760960f42701431447dca8790fa6c2b4c758bcedfbfa0afd
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 12, 2026.
Transparency logRelease files / tsaad-0.1.0-py3-none-any.whl
| Download URL | tsaad-0.1.0-py3-none-any.whl |
|---|---|
| Size | 65.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
1745fac99452dba7c4dc200d820d08d53d73100322dd5342f1aacbf6f79b417c
|
|
BLAKE2b-256 checksum How to use checksums |
47bc656f69974fb4aa822fbe56a37cb7ec82a3cd1f1c644069b0003c1ffb52cc
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 12, 2026.
Transparency logSecurity for tsaad 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for tsaad-0.1.0.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 12, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.