Chita 🐆 — a fast cat(alog) maker

Catalogs from Hierarchical, Image-calibrated, Tuning-free Analysis. Pronounced like cheetah.

Chita is an ultra-fast, multi-scale source extractor for large astronomical images (Roman, Euclid, Rubin era). One run finds point sources, compact and extended galaxies, and low-surface-brightness (LSB) objects. You don't run it twice with "hot" and "cold" settings, and you don't tune it per field.

• Self-calibrating thresholds. The noise of every filtered map is measured, so 5 sigma means 5 sigma on every image, including drizzled or resampled coadds with correlated noise.
• Multi-scale in one pass. A PSF-matched detection is combined with a Gaussian pyramid of coarser significance maps. Coarse-scale light that finer scales don't explain becomes a new (LSB) source; envelopes of known sources are attached to them.
• Statistical deblending. A component tree (dendrogram) with noise-unit tests replaces SExtractor's 32 fixed levels plus a global contrast. Leftover pixels are assigned by a deterministic watershed.
• Built-in purity estimate. The same detection runs on the inverted image at every scale.
• Fast. Rust, multi-threaded at every stage, with O(N) or O(N log N) kernels. Results are identical for any thread count.

The documentation has the guides, the API and the technical notes: the design describes the algorithm, and the review covers the existing codes it learns from. python scripts/build_site.py api and html build it, with the website, into site/build/.

Install

pip install chita

This installs the Python package and the chita command, ready to run on Linux, macOS and Windows with Python 3.9 or newer. Nothing is compiled.

To install the latest code from GitHub instead, which compiles it, first install Rust:

pip install git+https://github.com/lmytime/chita

Command line

chita image.fits                          # -> image.cat.fits (FITS binary table)
chita image.fits -o cat.csv --seg seg.fits --bkg bkg.fits --rms rms.fits
chita image.fits --weight wht.fits --weight-type weight --mask bad.fits --gain 2.0

chita --help lists all options. The defaults are meant to be left alone:

While it runs, chita prints each stage as it finishes, with what it found and how long it took. Under them, a live line shows the running stage: a spinner, a progress bar for the long stages (reading, deblending, coarse scales, measurement) and the elapsed time. A summary closes the run:

  🐆 chita v0.0.2 · fast multi-scale source extraction
     bench/out/real/primer_f444w_sci.fits · 10 threads

  ✓ Read image + weight  8192 × 8192 · 537 MB           299 ms
  ✓ Background (masked)  sky σ 2.82e-3                  398 ms
  ✓ Significance map     correlated noise ×2.50         536 ms
  ✓ Deblending           9,793 sources                  685 ms
  ✓ Coarse scales        +68 sources, 9 envelopes       259 ms
  ⠹ Measurement          ━━━━━━━━━╸──── 6,992/9,861     227 ms

  ╭──────────────────────────────────────────────────────────╮
  │ 9,861 sources in 3.73 s                                  │
  │ 9,793 at the PSF scale · 68 on coarse scales             │
  │ spurious estimate 82 (0.8%), from the inverted image     │
  ╰──────────────────────────────────────────────────────────╯

(Some stage lines are left out here.) When stderr is not a terminal, such as a pipe or a log file, the same lines are printed as the stages finish, without colour or the live line. --color never (or NO_COLOR=1) turns the colour off, and -q prints only errors.

It reads any BITPIX, BSCALE/BZERO/BLANK, and primary or IMAGE-extension FITS. For tile-compressed (.fz) images, use chita.extract_file in Python (it reads them through astropy).

Python

import chita

res = chita.extract(image)                    # 2-D numpy array (any dtype)
res = chita.extract(image, weight=wht, mask=bad, gain=2.0, threads=8)
res = chita.extract_file("image.fits", hdu=1)  # via astropy; RA/Dec from its header
res = chita.extract(image, header=hdr)          # RA/Dec from a FITS header or astropy WCS

cat = res.catalog          # numpy structured array (as_table=True -> astropy Table)
seg = res.segmentation     # 0 = sky, else source id
res.background, res.rms    # full-resolution maps
res.info                   # FWHM used, noise correlation, spurious estimate, timings

Several bands

Images of several bands on one pixel grid are detected jointly and then measured band by band with the same apertures. Detection is not diluted for sources seen in only one or two bands (docs/multiband.md). Inside large objects, compact sources and clumps whose colour differs from the object's are found as sources of their own (flag 512; subsources=False turns this off):

mb = chita.MultiBand(width, height)
for name, sci, err in bands:                    # in wavelength order
    mb.add_band(name, sci, err, weight_type="rms", fwhm=psf_fwhm[name])
det = mb.detect()                               # catalog, segmentation, detection map
det.catalog["det_window"]                       # e.g. "F356W-F444W": the bands that found it
det.info["n_sub"]                               # compact sources and clumps found inside large objects

phot = chita.forced_photometry(sci, det.segmentation, det.catalog, err,
                                 aper_radii=[3.3, 5.0], kron_factors=(2.5, 1.2), fwhm=psf_fwhm[name])

MultiBand(width, height, header=hdr) gives the catalog RA/Dec.

Images on other pixel grids

Images from other telescopes (HST, for example) can be measured at the detection's positions. Each is converted to nJy, resampled onto the detection grid and aligned to it:

scale = chita.flux_scale(hst_hdr)           # nJy per pixel unit: MJy/sr, e-/s + PHOTFNU, nanomaggies, zeropoints
sci, err = chita.resample(hst * scale, hst_hdr, det_hdr, error=hst_err * scale)  # Lanczos-3, flux conserving
off = chita.measure_offset(sci, err, det.catalog, det.segmentation)              # on compact sources
fixed_hdr = chita.shift_header(hst_hdr, off["dx"], off["dy"], det_hdr)           # then resample again
stars, sseg = chita.psf.find_stars(sci, err, near=detected_xy)                     # stellar locus, real sources only
psf, frac = chita.psf.build_epsf(sci, np.c_[stars["x_win"], stars["y_win"]], segmentation=sseg, labels=stars["id"])
kernel = chita.psf.matching_kernel(psf, reference_psf)

The catalogue script does all of this with --other BAND=SCI:WEIGHT (docs/multiband.md).

Seeing-limited images (DECam, HSC, ...) are fitted instead, in their own pixels, with each detection's light in a sharp band as its template (chita.templatefit). The kernel is the image's PSF from listed stars (Gaia), or a Moffat profile fitted on the brightest sources:

from chita import templatefit as tf

kernel, fwhm, beta = tf.fit_kernel(hr, seg, cat, hr_wcs, lr, lr_err, lr_wcs, hr_scale=0.03)   # without stars
a = tf.templates(hr, seg, cat, kernel, hr_wcs, lr_wcs, lr.shape)   # sparse: pixels x sources
flux, err, cov_index, model = tf.solve(a, lr, 1 / lr_err**2)        # all fluxes at once

In the catalogue script this is --lowres BAND=SCI:WEIGHT --lowres-stars gaia.csv. On Legacy Surveys DECam images of RXCJ2211 it agrees with the Tractor catalogue to 1–4% in g and r; i and z come out about 10% higher, not yet understood (docs/multiband.md).

scripts/multiband_catalog.py builds a complete catalog this way:

• native and PSF-matched photometry;
• aperture corrections to the total flux from empirical PSFs;
• random-aperture errors;
• nJy fluxes;
• quality columns;
• images on other pixel grids, such as HST (--other);
• seeing-limited images, fitted with templates (--lowres).

Photometric redshifts

chita.photoz fits templates at every redshift of a grid, with EAZY's conventions and files (filters, template lists, template error, IGM), but exactly, in parallel and about ten times faster per core. On EAZY's own inputs it reproduces EAZY's χ²(z) to 10⁻⁷ (docs/photoz.md). Its defaults go further: NIRCam's in-flight throughputs (JDox v7), a template list of its own (Hainline et al.'s, with redshift ranges, and a β = −3.2 template), a cap on the luminosity, calibrated p(z), and star and little-red-dot fits next to the galaxies'. On 14,000 NIRSpec redshifts they cut the outliers from 21.6% to 17.7% with eight NIRCam bands:

python scripts/photoz_catalog.py catalog_phot.fits --eazy-dir /path/to/eazy/data --mw-ebv 0.0851
python scripts/highz_candidates.py catalog_phot.fits --zmin 7      # classed z > 7 candidates

Visual inspection

scripts/vi_figures.py draws one figure per source: the SED with the redshift fit (AB magnitudes on the right axis), p(z) and χ²(z), the source's numbers, and north-up stamps in every band with an RGB composite and the segmentation (guide):

python scripts/vi_figures.py catalog_phot.fits --eazy-dir /path/to/eazy/data --workers 8

Catalogue columns

Coordinates are 0-based with pixel centres at integers (numpy convention). Add 1 to compare with SExtractor/FITS conventions.

Benchmarks

bench/simulate.py makes GalSim images with known truth: stars, Sérsic galaxies (some with star-forming knots), LSB dwarfs, background structure, and white or correlated noise. bench/validate.py compares Chita with SEP, SExtractor and photutils using one-to-one matching and threshold sweeps. bench/speed.py measures throughput.

python bench/simulate.py --size 4096 --fwhm 2.5 --out bench/out/sim_ground
python bench/validate.py bench/out/sim_ground --sweep --photutils
python bench/speed.py bench/out/sim_ground.fits

Results are in docs/results.md.

On real images, bench/real_eval.py injects sources into a mosaic and recovers them. It covers blank sky and the wings of bright objects, and uses the inverted image as a purity proxy. bench/real_report.py turns its output into tables and a trade-off figure. docs/real_images.md covers JWST/NIRCam mosaics of Abell 2744 and PRIMER-COSMOS: the failure modes they exposed, the fixes, and the comparison with the other codes.

Repository layout

crates/chita-core   Rust library (background, filters, labeling, deblending,
                    measurement, pipeline, FITS I/O)
crates/chita-cli    `chita` command
crates/chita-py     Python bindings (PyO3) + `chita` package + tests
bench/              simulations, validation and speed benchmarks
scripts/            multi-band catalogues, photometric redshifts, VI figures, site build
docs/               guides, reference and technical notes (the documentation's source)
site/               the website's home page and the documentation's style
pyproject.toml      Python packaging (maturin)
.github/workflows/  tests on every push; packages and releases on version tags

Development

Building from a clone needs Rust ≥ 1.75 and maturin:

maturin develop --release        # the Python package and `chita`, into the active environment
cargo build --release            # the command alone: target/release/chita
cargo test --release             # Rust tests
python -m pytest crates/chita-py/tests

environment.yml creates a conda environment with the benchmark codes as well (SEP, photutils, GalSim, SExtractor):

conda env create -f environment.yml && conda activate chita

Releasing

1. Set the new version in Cargo.toml ([workspace.package]) and commit.
2. Tag and push: git tag v0.2.0 && git push origin v0.2.0.

The Release workflow then builds and tests the packages for every platform, publishes them to PyPI and creates the GitHub release. PyPI accepts them because the project trusts this repository's release.yml workflow in the pypi environment (trusted publishing); no token is stored.

Building inside a conda shell

Some conda setups export CFLAGS for a different architecture (for example -march=core2 from an Intel anaconda on Apple Silicon). Chita itself has no C code. If you add C-based dependencies, build with env -u CFLAGS -u CPPFLAGS cargo build --release.