Metadata-Version: 2.4
Name: chita
Version: 0.0.2
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: MacOS
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Rust
Classifier: Topic :: Scientific/Engineering :: Astronomy
Requires-Dist: numpy>=1.21
Requires-Dist: scipy>=1.7
Requires-Dist: astropy>=5
Requires-Dist: pytest ; extra == 'test'
Provides-Extra: test
License-File: LICENSE
Summary: Fast, tuning-free, multi-scale source extraction for astronomical images
Keywords: astronomy,source extraction,source detection,deblending,photometry,JWST
Author: Mingyu Li
License-Expression: BSD-3-Clause
Requires-Python: >=3.9
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Homepage, https://github.com/lmytime/chita
Project-URL: Issues, https://github.com/lmytime/chita/issues

# Chita 🐆 — a fast cat(alog) maker

*Catalogs from Hierarchical, Image-calibrated, Tuning-free Analysis.*
Pronounced like *cheetah*.

Chita is an ultra-fast, multi-scale source extractor for large astronomical
images (Roman, Euclid, Rubin era). One run finds point sources, compact and
extended galaxies, and low-surface-brightness (LSB) objects. You don't run it
twice with "hot" and "cold" settings, and you don't tune it per field.

* **Self-calibrating thresholds.** The noise of every filtered map is
  *measured*, so `5 sigma` means 5 sigma on every image, including
  drizzled or resampled coadds with correlated noise.
* **Multi-scale in one pass.** A PSF-matched detection is combined with a
  Gaussian pyramid of coarser significance maps. Coarse-scale light that
  finer scales don't explain becomes a new (LSB) source; envelopes of known
  sources are attached to them.
* **Statistical deblending.** A component tree (dendrogram) with
  noise-unit tests replaces SExtractor's 32 fixed levels plus a global
  contrast. Leftover pixels are assigned by a deterministic watershed.
* **Built-in purity estimate.** The same detection runs on the inverted image
  at every scale.
* **Fast.** Rust, multi-threaded at every stage, with O(N) or O(N log N)
  kernels. Results are identical for any thread count.

The [documentation](docs/index.md) has the guides, the API and the
technical notes: the [design](docs/design.md) describes the algorithm, and
the [review](docs/review.md) covers the existing codes it learns from.
`python scripts/build_site.py api` and `html` build it, with the website, into
`site/build/`.

## Install

```bash
pip install chita
```

This installs the Python package and the `chita` command, ready to run on
Linux, macOS and Windows with Python 3.9 or newer. Nothing is compiled.

To install the latest code from GitHub instead, which compiles it, first
install [Rust](https://rustup.rs):

```bash
pip install git+https://github.com/lmytime/chita
```

## Command line

```bash
chita image.fits                          # -> image.cat.fits (FITS binary table)
chita image.fits -o cat.csv --seg seg.fits --bkg bkg.fits --rms rms.fits
chita image.fits --weight wht.fits --weight-type weight --mask bad.fits --gain 2.0
```

`chita --help` lists all options. The defaults are meant to be left
alone:

| option | default | meaning |
|---|---|---|
| `--thresh` | 5.0 | detection: peak significance, in measured sigma of the detection image |
| `--analysis-thresh` | 2.0 | footprint (isophote) significance |
| `--fwhm` | estimated | PSF FWHM (pixels) |
| `--back-size` | 64 | background mesh (pixels) |
| `--contrast` | 0.005 | deblending: minimum flux fraction of a split, applied only above the detection level |
| `--levels` | auto | number of coarse scales (0 disables multi-scale) |
| `--subsources` | off | search large objects for compact sources and clumps inside them (flag 512; experimental with one band) |
| `-j` | all cores | threads |

While it runs, `chita` prints each stage as it finishes, with what it
found and how long it took. Under them, a live line shows the running
stage: a spinner, a progress bar for the long stages (reading, deblending,
coarse scales, measurement) and the elapsed time. A summary closes the run:

```
  🐆 chita v0.0.2 · fast multi-scale source extraction
     bench/out/real/primer_f444w_sci.fits · 10 threads

  ✓ Read image + weight  8192 × 8192 · 537 MB           299 ms
  ✓ Background (masked)  sky σ 2.82e-3                  398 ms
  ✓ Significance map     correlated noise ×2.50         536 ms
  ✓ Deblending           9,793 sources                  685 ms
  ✓ Coarse scales        +68 sources, 9 envelopes       259 ms
  ⠹ Measurement          ━━━━━━━━━╸──── 6,992/9,861     227 ms

  ╭──────────────────────────────────────────────────────────╮
  │ 9,861 sources in 3.73 s                                  │
  │ 9,793 at the PSF scale · 68 on coarse scales             │
  │ spurious estimate 82 (0.8%), from the inverted image     │
  ╰──────────────────────────────────────────────────────────╯
```

(Some stage lines are left out here.) When stderr is not a terminal, such
as a pipe or a log file, the same lines are printed as the stages finish,
without colour or the live line. `--color never` (or `NO_COLOR=1`) turns the
colour off, and `-q` prints only errors.

It reads any BITPIX, BSCALE/BZERO/BLANK, and primary or IMAGE-extension
FITS. For tile-compressed (`.fz`) images, use `chita.extract_file` in
Python (it reads them through astropy).

## Python

```python
import chita

res = chita.extract(image)                    # 2-D numpy array (any dtype)
res = chita.extract(image, weight=wht, mask=bad, gain=2.0, threads=8)
res = chita.extract_file("image.fits", hdu=1)  # via astropy; RA/Dec from its header
res = chita.extract(image, header=hdr)          # RA/Dec from a FITS header or astropy WCS

cat = res.catalog          # numpy structured array (as_table=True -> astropy Table)
seg = res.segmentation     # 0 = sky, else source id
res.background, res.rms    # full-resolution maps
res.info                   # FWHM used, noise correlation, spurious estimate, timings
```

### Several bands

Images of several bands on one pixel grid are detected jointly and then
measured band by band with the same apertures. Detection is not diluted for
sources seen in only one or two bands
([docs/multiband.md](docs/multiband.md)). Inside large objects, compact
sources and clumps whose colour differs from the object's are found as
sources of their own (flag 512; `subsources=False` turns this off):

```python
mb = chita.MultiBand(width, height)
for name, sci, err in bands:                    # in wavelength order
    mb.add_band(name, sci, err, weight_type="rms", fwhm=psf_fwhm[name])
det = mb.detect()                               # catalog, segmentation, detection map
det.catalog["det_window"]                       # e.g. "F356W-F444W": the bands that found it
det.info["n_sub"]                               # compact sources and clumps found inside large objects

phot = chita.forced_photometry(sci, det.segmentation, det.catalog, err,
                                 aper_radii=[3.3, 5.0], kron_factors=(2.5, 1.2), fwhm=psf_fwhm[name])
```

`MultiBand(width, height, header=hdr)` gives the catalog RA/Dec.

### Images on other pixel grids

Images from other telescopes (HST, for example) can be measured at the
detection's positions. Each is converted to nJy, resampled onto the
detection grid and aligned to it:

```python
scale = chita.flux_scale(hst_hdr)           # nJy per pixel unit: MJy/sr, e-/s + PHOTFNU, nanomaggies, zeropoints
sci, err = chita.resample(hst * scale, hst_hdr, det_hdr, error=hst_err * scale)  # Lanczos-3, flux conserving
off = chita.measure_offset(sci, err, det.catalog, det.segmentation)              # on compact sources
fixed_hdr = chita.shift_header(hst_hdr, off["dx"], off["dy"], det_hdr)           # then resample again
stars, sseg = chita.psf.find_stars(sci, err, near=detected_xy)                     # stellar locus, real sources only
psf, frac = chita.psf.build_epsf(sci, np.c_[stars["x_win"], stars["y_win"]], segmentation=sseg, labels=stars["id"])
kernel = chita.psf.matching_kernel(psf, reference_psf)
```

The catalogue script does all of this with `--other BAND=SCI:WEIGHT`
([docs/multiband.md](docs/multiband.md)).

Seeing-limited images (DECam, HSC, ...) are fitted instead, in their own
pixels, with each detection's light in a sharp band as its template
(`chita.templatefit`). The kernel is the image's PSF from listed stars
(Gaia), or a Moffat profile fitted on the brightest sources:

```python
from chita import templatefit as tf

kernel, fwhm, beta = tf.fit_kernel(hr, seg, cat, hr_wcs, lr, lr_err, lr_wcs, hr_scale=0.03)   # without stars
a = tf.templates(hr, seg, cat, kernel, hr_wcs, lr_wcs, lr.shape)   # sparse: pixels x sources
flux, err, cov_index, model = tf.solve(a, lr, 1 / lr_err**2)        # all fluxes at once
```

In the catalogue script this is `--lowres BAND=SCI:WEIGHT --lowres-stars
gaia.csv`. On Legacy Surveys DECam images of RXCJ2211 it agrees with the
Tractor catalogue to 1–4% in g and r; i and z come out about 10% higher,
not yet understood ([docs/multiband.md](docs/multiband.md)).

`scripts/multiband_catalog.py` builds a complete catalog this way:

- native and PSF-matched photometry;
- aperture corrections to the total flux from empirical PSFs;
- random-aperture errors;
- nJy fluxes;
- quality columns;
- images on other pixel grids, such as HST (`--other`);
- seeing-limited images, fitted with templates (`--lowres`).

### Photometric redshifts

`chita.photoz` fits templates at every redshift of a grid, with EAZY's
conventions and files (filters, template lists, template error, IGM), but
exactly, in parallel and about ten times faster per core. On EAZY's own
inputs it reproduces EAZY's χ²(z) to 10⁻⁷ ([docs/photoz.md](docs/photoz.md)).
Its defaults go further: NIRCam's in-flight throughputs (JDox v7), a
template list of its own (Hainline et al.'s, with redshift ranges, and a
β = −3.2 template), a cap on the luminosity, calibrated p(z), and star and
little-red-dot fits next to the galaxies'. On 14,000 NIRSpec redshifts
they cut the outliers from 21.6% to 17.7% with eight NIRCam bands:

```bash
python scripts/photoz_catalog.py catalog_phot.fits --eazy-dir /path/to/eazy/data --mw-ebv 0.0851
python scripts/highz_candidates.py catalog_phot.fits --zmin 7      # classed z > 7 candidates
```

### Visual inspection

`scripts/vi_figures.py` draws one figure per source: the SED with the
redshift fit (AB magnitudes on the right axis), p(z) and χ²(z), the source's
numbers, and north-up stamps in every band with an RGB composite and the
segmentation ([guide](docs/guide-vi.md)):

```bash
python scripts/vi_figures.py catalog_phot.fits --eazy-dir /path/to/eazy/data --workers 8
```

## Catalogue columns

Coordinates are 0-based with pixel centres at integers (numpy convention).
Add 1 to compare with SExtractor/FITS conventions.

| column | meaning |
|---|---|
| `x`, `y` | isophotal barycentre |
| `x_win`, `y_win` | windowed (iterative Gaussian) centroid, as SExtractor's XWIN |
| `ra`, `dec`; `ra_win`, `dec_win` | sky coordinates (degrees) of the barycentre and the windowed centroid, from the image's TAN or TAN-SIP WCS; NaN without one |
| `x_peak`, `y_peak` | brightest pixel |
| `x2`, `y2`, `xy`, `a`, `b`, `theta` | second moments, RMS semi-axes, position angle (rad, CCW from +x) |
| `npix`, `flux_iso`, `flux_iso_err` | segment area and flux |
| `kron_radius`, `flux_auto`, `flux_auto_err` | Kron radius (units of a,b) and Kron-aperture (2.5, min 3.5) flux; neighbours replaced by mirror pixels |
| `flux_aper`, `flux_aper_err` | circular apertures (default radii 1 and 2 x FWHM), exact pixel overlap |
| `flux_radius`, `fwhm` | half-light radius; FWHM from the half-maximum area |
| `peak`, `det_sig`, `snr` | peak value, peak significance at the detection scale, AUTO S/N |
| `bkg` | local background (plus the subtracted halo level for `ON_HALO` objects) |
| `scale` | detection scale: 0 = PSF scale, k = pyramid level (~2^k px) |
| `parent`, `nblend` | footprint id and number of sources deblended from it |
| `host` | id of the brighter object on whose light (halo or body) this one sits, for `ON_HALO` objects; else 0 |
| `flags` | 1 blended, 2 edge, 4 masked pixels, 8 saturated, 16 AUTO aperture truncated, 32 windowed centroid failed, 64 neighbours in AUTO aperture, 128 grown with a coarse-scale envelope, 256 on a brighter object's halo or body (`host`; its light there was subtracted), 512 sub-source: a compact source or clump found inside its `host` (use circular apertures) |

## Benchmarks

`bench/simulate.py` makes GalSim images with known truth: stars, Sérsic
galaxies (some with star-forming knots), LSB dwarfs, background structure,
and white or correlated noise. `bench/validate.py` compares Chita with
SEP, SExtractor and photutils using one-to-one matching and threshold
sweeps. `bench/speed.py` measures throughput.

```bash
python bench/simulate.py --size 4096 --fwhm 2.5 --out bench/out/sim_ground
python bench/validate.py bench/out/sim_ground --sweep --photutils
python bench/speed.py bench/out/sim_ground.fits
```

Results are in [docs/results.md](docs/results.md).

On real images, `bench/real_eval.py` injects sources into a mosaic and
recovers them. It covers blank sky and the wings of bright objects, and uses
the inverted image as a purity proxy. `bench/real_report.py` turns its output
into tables and a trade-off figure. [docs/real_images.md](docs/real_images.md)
covers JWST/NIRCam mosaics of Abell 2744 and PRIMER-COSMOS: the failure modes
they exposed, the fixes, and the comparison with the other codes.

## Repository layout

```
crates/chita-core   Rust library (background, filters, labeling, deblending,
                    measurement, pipeline, FITS I/O)
crates/chita-cli    `chita` command
crates/chita-py     Python bindings (PyO3) + `chita` package + tests
bench/              simulations, validation and speed benchmarks
scripts/            multi-band catalogues, photometric redshifts, VI figures, site build
docs/               guides, reference and technical notes (the documentation's source)
site/               the website's home page and the documentation's style
pyproject.toml      Python packaging (maturin)
.github/workflows/  tests on every push; packages and releases on version tags
```

## Development

Building from a clone needs Rust ≥ 1.75 and [maturin](https://www.maturin.rs):

```bash
maturin develop --release        # the Python package and `chita`, into the active environment
cargo build --release            # the command alone: target/release/chita
cargo test --release             # Rust tests
python -m pytest crates/chita-py/tests
```

`environment.yml` creates a conda environment with the benchmark codes as
well (SEP, photutils, GalSim, SExtractor):

```bash
conda env create -f environment.yml && conda activate chita
```

### Releasing

1. Set the new version in `Cargo.toml` (`[workspace.package]`) and commit.
2. Tag and push: `git tag v0.2.0 && git push origin v0.2.0`.

The Release workflow then builds and tests the packages for every platform,
publishes them to PyPI and creates the GitHub release. PyPI accepts them
because the project trusts this repository's `release.yml` workflow in the
`pypi` environment (trusted publishing); no token is stored.

## Building inside a conda shell

Some conda setups export `CFLAGS` for a different architecture (for example
`-march=core2` from an Intel anaconda on Apple Silicon). Chita itself has no
C code. If you add C-based dependencies, build with
`env -u CFLAGS -u CPPFLAGS cargo build --release`.

