Live Stacking

Watch a capture folder, stack every new sub-exposure as it lands, and see a cumulative image build in real time — with on-the-fly calibration, star alignment, μ–σ clipping rejection, per-filter mono-to-colour composition, star-trail mode, live metrics plotting and a culling folder for bad frames. The during-session companion to the batch Stacking Suite.

SASpro-exclusive tool. Live Stacking is an original SetiAstro Suite Pro implementation by Franklin Marek. The watch-folder orchestration (file-stability probe, penalty window, three-strike retry), the three-phase linear↔μσ bootstrap-to-Welford transition, the FILTER-header-based mono-to-colour bucketing with seven narrowband mapping modes including the auto-Natural logic, the SNR proxy formula used for the live metrics plot, the per-filter flat support, and the red/blue culling display on the live metrics window are all built from the ground up for SASpro's streaming pipeline.
Workflow
Point your capture software at a folder → in SASpro, open Live Stacking and Select Folder → (optionally) load master dark, master flat, per-filter flats, and pick a cull folder → pick Mono→Color and/or Star-Trail if you want those modes → hit Process && Monitor — the stack rebuilds live as frames arrive, with metrics plotting alongside and bad frames culled automatically.

The core incremental update, applied per pixel for every good frame x arriving at count n+1:

S_{n+1} = \frac{n}{n+1}\,S_n + \frac{1}{n+1}\,x_{n+1}

During bootstrap this runs over raw normalized frames; after n reaches the bootstrap threshold, it runs over μ–σ-clipped frames with per-pixel Welford statistics (§13.9).

1. The main window

The window is a vertical stack: a top row of controls, a status line showing where files come from and go to, a zoom toolbar with a Settings wrench, the Show Metrics button, the large live-preview area, brightness/contrast sliders, and the Open in New View ▶ button at the bottom. A dynamic Mode label (“Linear Average” / “μ-σ Clipping Average” / “Star-Trail” / per-filter variants) and a Status label at the very bottom narrate what's happening.

The preview is a true QGraphicsView, so pan with click-drag and zoom with the toolbar. The dialog is non-modal — you can leave it running and keep working in SASpro.

2. Picking a watch folder

Click Select Folder… and pick the directory your capture software writes to. Live Stacking recurses into subfolders, so if your capture program organizes by filter or session into subdirectories, it still works.

By default every supported file type in the folder is scanned:

CategoryExtensions
FITS.fit, .fits
TIFF & XISF.tif, .tiff, .xisf
Camera RAW.cr2, .cr3, .nef, .arw, .dng, .raf, .orf, .rw2, .pef
Everyday.png, .jpg, .jpeg

If your capture program writes preview JPGs, autostretch TIFFs, debug thumbnails or other derived files alongside the real subs, tick Ignore non-FITS files in Settings (§9) — the scanner then limits itself to *.fit and *.fits only, which is also what you want for most dedicated astro cameras.

How Live Stacking knows a file is “done”

Capture software writes a file in chunks and often holds the handle open for a moment after the last byte lands. Reading it half-written would be a disaster. Live Stacking's file-readiness probe handles this:

  1. Every scan, each candidate's size and modification time are recorded.
  2. A file is “ready” only once size and mtime have stayed exactly the same for the Acquisition Delay window (default 3.0 s; see Settings).
  3. A final small open-and-read is attempted. If the OS still refuses (writer holds the handle), the file enters a 10 s penalty window and gets rechecked next cycle.
  4. If a file fails three times in a row during processing, it's marked permanently skipped so it stops blocking the queue; its basename appears in the status line.

The watch loop polls every 1.5 s. A re-entrancy guard prevents overlapping scans if a frame takes longer than one tick to process.

3. Calibration masters

Live Stacking supports a single master dark, a single master flat, and a per-filter flat dictionary for mono-to-colour sessions.

During processing (§13.3), per-filter flats take precedence over the global master flat when the incoming frame's FILTER header matches a key. Masters are automatically shape-coerced to match each incoming frame: a mono master applied to a colour frame broadcasts to all three channels; a colour master applied to a mono frame is collapsed to its channel mean. This means you can load a mono dark and still have it correctly subtract from debayered RGB frames.

Shape mismatches are rejected up front. If you try to load a 2D flat while a 3D dark is already loaded (or vice versa), the load is refused with a message rather than silently miscalibrating later. Reset or reload the mismatched master first.

Cull folder

Click Select Cull Folder… to pick a destination for rejected frames (frames whose star metrics fail the thresholds in §7). When set, flagged frames are moved there with shutil.move; when not set, they're simply logged in the status line and skipped. The folder is created on first use if it doesn't exist.

4. Running the stack

Four run buttons sit at the right of the top row:

The Frames counter and Total Exp label (format HH:MM:SS) in the status row tick up with each accepted frame; exposure is accumulated from the FITS EXPOSURE or EXPTIME header, or from EXIF on RAW.

5. Stack modes

Live Stacking has three top-level modes, two of them toggled by checkboxes next to the run buttons.

Linear Average → μ–σ Clipping (default)

With both checkboxes off, Live Stacking runs a classical two-phase integration: the first Bootstrap frames (default 24) go into a plain linear running mean while per-pixel statistics are collected; after the bootstrap, the stack switches to μ–σ clipping at Clip threshold σ (default 3.5σ) with Welford's incremental mean/variance updates (§13.9). The Mode label narrates the transition.

Mono → Color Stacking

Tick Mono → Color Stacking to bucket incoming mono frames by their FILTER header into separate per-filter stacks and composite them live into an RGB preview according to a Narrowband Mapping (§6). Each filter stack goes through its own independent bootstrap→Welford transition. The composite is rebuilt from scratch each new frame, so adding a late OIII frame updates the whole preview immediately.

Mono-to-colour mode works entirely from the FILTER header — frames with no FILTER are not routed to any channel. OSC frames with a BAYERPAT header are still debayered and stacked as colour, bypassing the mono buckets.

Star-Trail Mode

Tick ★★ Star-Trail Mode ★★ to replace the running-mean pipeline with an elementwise per-pixel maximum accumulator. No alignment is performed and no clipping is applied; each frame is calibrated, debayered if needed, stretched to target_median = 0.3, and then combined with the current stack as:

S_{n+1} = \max(S_n,\,x_{n+1})

— the classic star-trail blend, where every bright pixel wins. Enabling Star-Trail disables Mono→Color (they're mutually exclusive).

6. Narrowband mapping (Mono → Color)

When Mono→Color is on, the Narrowband Mapping setting controls how per-filter stacks are composited into the live RGB preview. Change it any time in Settings (§9) — the next preview refresh uses the new mapping.

FILTER-header matching

SASpro recognises these FILTER values (case-insensitive, with common synonyms):

KeyMatches
HH, Ha, HAlpha, H-Alpha
OO, O3, OIII
SS, S2, SII
RR, Red, Rd
GG, Green, Grn
BB, Blue, Bl
LL, Lum, Lumi, Luminance

Mapping options

Natural (default) — auto-selects the composite based on what's loaded:

SHO, HSO, OSH, SOH, HOS, OHS — direct three-letter mappings. The letters assign to R, G, B in order:

Missing channels in a direct mapping default to zero, so an SHO composition with only Hα and OIII loaded shows up red-less until the first SII frame lands.

7. Culling

Every frame is measured for star quality (§13.6) after calibration and alignment but before it gets combined into the stack. A frame is flagged and rejected if any of the three thresholds in Settings is exceeded:

\mathrm{flag} = (\mathrm{FWHM} > F_{\max})\;\vee\;(e > e_{\max})\;\vee\;(N < N_{\min})
ThresholdDefaultMeaning
Max FWHM15.0 pxReject frames with median star FWHM above this. Catches heavy defocus and bad seeing spikes.
Max Eccentricity0.9Reject frames with median eccentricity above this. Catches tracking slips, wind gusts, cable snags.
Min Star Count5Reject frames with fewer detected stars than this. Catches clouds, satellite streaks saturating detection, and dome lights.

Flagged frames appear as red dots on the live metrics plot. If a cull folder is set, the file is moved there; otherwise the status line notes it and continues. The frame counter and exposure accumulator don't tick up for culled frames — what you see in Frames: N is the integrated count only.

Picking thresholds. Run a session for a few minutes with wide thresholds (say Max FWHM = 50, Max Ecc = 1.0, Min Star = 0), look at the metrics plot's blue-dot cloud, and then set thresholds slightly outside the usual spread. Overly tight thresholds on a bad night will cull everything.

8. Live metrics window

Click Show Metrics to open a separate 2×2 scatter window that updates per frame:

Blue dots are kept frames; red dots are culled. Both colours stay plotted for the whole session so you can see when and why conditions changed. The window is independent of the main dialog — close it any time without stopping the stack; reopen with Show Metrics.

9. Settings dialog

Click the orange wrench button in the zoom toolbar to open the combined Live Stack & Culling settings dialog. All values persist across sessions via QSettings under the LiveStack/ key.

SettingDefaultRangeEffect
Switch to μ–σ clipping after241 – 100Bootstrap frame count. The linear running mean collects frames up to this count, then the stack switches to Welford-based clipping.
Clip threshold3.50.1 – 10.0μ–σ rejection multiplier (§13.9). Lower = more aggressive rejection.
Max FWHM (px)15.00.1 – 50.0Culling threshold on median star FWHM.
Max Eccentricity0.90.0 – 1.0Culling threshold on median star eccentricity.
Min Star Count50 – 5000Culling threshold on detected star count.
Acquisition Delay3.0 s0.0 – 60.0 sFile stability window. Size+mtime must stay unchanged for this long before a file is considered writeable-complete.
Star Detection σ7.01.0 – 20.0SEP detection threshold above background. Lower = more faint stars (and more false positives in a bright sky).
Ignore non-FITS filesoff—When on, only *.fit/*.fits are picked up. Use when the capture app writes derived files beside the raw subs.
Narrowband MappingNatural—Composition rule for Mono→Color (§6).

10. Opening the live result

Click Open in New View ▶ to push the current stack (or composite) into SASpro as a new document named LiveStack_<YYYYMMDD_HHMMSS>_<N>f where N is the integrated frame count. The result is always float32 clipped to [0, 1], with metadata recording the source and frame count.

Live Stacking tries a sequence of document-manager entry points (create_numpy_document, new_document_from_numpy, open_numpy, open_array, open_image_array, add_document_from_array) so it works across SASpro's variants; if none fit, it writes a temp 16-bit TIFF and asks the host window to open it as a fallback. If even that fails, the temp path is shown in a dialog so you can grab it manually.

11. Preview, brightness, contrast and zoom

The brightness and contrast sliders affect the preview only — the underlying stack is untouched. Both are pivoted at 0.3 (the normalization target median, so the preview midtone is where the sliders have most leverage):

y = (x - 0.3)\,c + 0.3 + b

with c the contrast slider (10–1000 mapped to 0.1–10.0) and b the brightness slider (-100–100 mapped to -1.0–+1.0). The result is clipped to [0, 1] for display.

Preview controls:
🖱️ Drag — pan (QGraphicsView scroll-hand drag).
Zoom-in / Zoom-out toolbar buttons — 1.2× steps centred on the cursor.
Fit to Window — rescale to fill the preview area with aspect preserved.
The first good frame auto-fits; later frames preserve your current zoom/pan.

12. Technical details

The streaming pipeline, in order of each frame's processing.

12.1 File readiness and polling

The scanner runs on a 1500 ms QTimer. Each tick, the folder is reglobbed recursively against the current extension set, candidate paths are normalised (case-insensitive, absolute, platform-normalised slashes), and those not yet in the processed set are tested for readiness. Each path accumulates a probe record:

A _poll_busy re-entrancy guard prevents a long-running frame from stacking up overlapping scans. A threading.Event (_stop_event) is checked at every major pipeline stage so Stop is responsive mid-frame rather than only between frames.

12.2 Image loading

Non-RAW files go through SASpro's shared load_image, which returns the pixel array, a FITS-style header, bit depth and a mono flag. XISF, TIFF, FITS, PNG and JPG all land in the same code path.

Camera-RAW files are decoded through rawpy (LibRaw) with use_camera_wb=True, no_auto_bright=True, output_bps=16, then normalised to float32 in [0, 1] by dividing by 65535. Exposure time, ISO and DateTimeOriginal are pulled from EXIF via exifread and planted into a synthetic FITS header so the rest of the pipeline sees a RAW frame as if it had come from a FITS-aware camera. If EXIF parsing fails the defaults "Unknown" are kept and processing continues.

12.3 Calibration

Calibration is a two-step process applied once per frame after loading:

I' = I - D
I'' = \frac{I'}{F}

Dark subtraction is a plain NumPy subtraction in float32. Flat division is dispatched through apply_flat_division_numba, a JIT-compiled helper shared with the Stacking Suite. The flat used is the per-filter flat F_k when (a) Mono→Color is active, (b) the frame's FILTER header matches a key in the per-filter dictionary; otherwise the global master flat is used.

Shape coercion is automatic:

Up-front shape compatibility between the master dark and master flat themselves is checked at load time, so you can't mix a 3D dark with a 2D flat without noticing.

12.4 Debayering

If a frame is marked mono by the loader but its header carries a BAYERPAT, Live Stacking debayers it with debayer_fits_fast (SASpro's shared vectorised debayerer, same one the Stacking Suite uses) before the alignment stage. Supports RGGB, GRBG, GBRG and BGGR patterns.

12.5 Star alignment

The first good frame becomes reference_image_2d — a 2D intensity plane used as the alignment reference for the rest of the session (reset clears it, so a new reference is picked on the next good frame after a Reset). Every subsequent frame is reduced to a 2D plane (channel mean for colour, or the mono plane in Mono→Color), aligned to the reference via Astroalign's star-triangle matching, and the resulting 2×3 affine transform:

\mathbf{p}' = A\,\mathbf{p} + \mathbf{t}

is applied to the full image (full RGB tensor if the frame is colour, else the mono plane). If Astroalign fails to find a transform (too few stars, catastrophic defocus), SASpro falls back to the identity transform rather than discarding the frame — so even badly-degraded frames at least land on top of the stack rather than vanishing, and will usually be culled by the FWHM/eccentricity thresholds anyway.

12.6 Per-frame normalization

Before stacking, each frame is pulled to a common median via SASpro's stretch_mono_image / stretch_color_image helpers (shared with Stacking Suite's normalization block), both with target_median = 0.3. Mono frames pass through the mono stretcher directly; colour frames use the colour stretcher with linked=False (each channel pulled independently) so a per-channel exposure imbalance doesn't skew the composite.

Target 0.3 is deliberately above the typical sky median so the running stack has useful contrast on the live preview from the first frame.

12.7 Star metrics (SEP)

Per-frame star metrics come from SASpro's measure_stars_sep, which wraps SEP (Barbary 2016; the Python port of Bertin & Arnouts' Source Extractor). The call uses:

This matches the metrics SASpro's Blink tool uses, so the FWHM you see live is directly comparable with the quality numbers you'd get from running Blink on the same subs later.

12.8 Global SNR proxy

The fourth metric plot — (μ–ν)/σ — is a cheap, scale-stable SNR proxy computed on the cumulative stack (the running stack for colour sessions; the composite for Mono→Color sessions; the current frame if nothing's integrated yet). The construction:

  1. Reduce the stack to a 2D intensity plane (OpenCV's cvtColor with COLOR_RGB2GRAY if available, else a channel mean).
  2. Isolate the middle 50% rectangle of the plane (quartered on both axes). Call this C.
  3. Centre a patch P of up to 200×200 pixels inside C.
  4. Compute:
    \mu_{\mathrm{patch}} = \mathrm{mean}(P)
    \mathrm{med}_{\mathrm{patch}} = \mathrm{median}(P)
    \sigma_{\mathrm{central}} = \mathrm{std}(C)
  5. Build a background offset
    \nu = \mathrm{med}_{\mathrm{patch}} - 3\,\sigma_{\mathrm{central}}
    which places ν roughly at the sky-noise floor estimate.
  6. Report:
    S = \frac{\mu_{\mathrm{patch}} - \nu}{\sigma_{\mathrm{central}}}

If σ_central is zero (empty stack, uniform fill), the proxy returns 0. The result isn't a calibrated SNR — it's a monotonic quantity that should grow roughly as √N for a stack of similar frames, giving you a clean visual indicator that integration is actually helping. Noisy nights with gradients show up as a slower rise; a sudden flattening usually means clouds or focus drift.

12.9 The bootstrap → Welford transition

Running mean and running variance of the normalized frames are built up in two phases.

Phase 1 — linear bootstrap. For frames 1 through B (the bootstrap frame count), the pixel-wise running mean uses the standard incremental form:

S_{n+1} = \frac{n}{n+1}\,S_n + \frac{1}{n+1}\,x_{n+1}

and every frame is also appended to an in-memory buffer. The stack is therefore available from the very first frame — no “warmup” delay before you see anything.

Phase 2 — Welford initialization. When n reaches B, the buffered frames are collapsed into the Welford summary:

\mu_0 = \frac{1}{B}\sum_{i=1}^{B} x_i
M_2^{(0)} = \sum_{i=1}^{B} (x_i - \mu_0)^2

and the frame buffer is discarded. The Mode label flips to μ-σ Clipping Average.

Phase 3 — incremental μ–σ clipping. From frame B+1 onwards, each new frame goes through the clipping update in §13.9. In Mono→Color mode every filter runs through this three-phase transition independently at its own frame count, so Hα might still be bootstrapping while OIII has already switched to clipping.

12.10 μ–σ clipping with Welford updates

For each incoming post-bootstrap frame, pixels further than k = clip_threshold standard deviations from the running mean are rejected and replaced with the running mean itself before being combined into the stack:

\sigma = \sqrt{\frac{M_2}{n - 1}}
m(x,y) = \mathbb{1}\!\left[\,|x - \mu| \leq k\,\sigma\,\right]
x' = m\,x + (1-m)\,\mu

The clipped frame x' is combined into the running mean with the same incremental formula as the bootstrap, and the Welford mean/variance summary is updated in place using the standard online algorithm (Welford 1962):

\delta_1 = x' - \mu_{\mathrm{old}}
\mu_{\mathrm{new}} = \mu_{\mathrm{old}} + \frac{\delta_1}{n+1}
\delta_2 = x' - \mu_{\mathrm{new}}
M_2^{\mathrm{new}} = M_2^{\mathrm{old}} + \delta_1\,\delta_2

This never requires revisiting past frames and uses O(H × W) memory regardless of integration count — critical when you're going to stack hundreds of frames over the course of a session. Replacing rejected pixels with the current μ rather than skipping them keeps the running mean's denominator consistent and prevents the variance estimate from drifting when hot pixels or satellite trails are rejected.

12.11 Mono → Color composition rules

Each incoming mono frame whose FILTER key is recognized (§6) goes into its own stack (filter_stacks[key]) with its own frame counter, bootstrap buffer, and Welford summary. The RGB preview is rebuilt from scratch after every new frame using the current mapping. For the Natural mode, the branch decision is made at composite time, not at frame-ingestion time, so adding the first SII frame to an already-running HOO session automatically promotes the composite to the SII palette on the next refresh.

For direct 3-letter mappings (SHO, HSO, …), missing filter stacks default to zeros in that channel, so a partial dataset simply shows the available channels and the missing one as black. If an invalid mapping string is somehow set, the composite falls back to Natural and the mapping is reset.

12.12 Star-trail max accumulator

Star-trail mode replaces steps 7–10 of the normal pipeline (align, normalize, measure, cull, integrate) with a direct per-pixel maximum:

S_{n+1} = \max(S_n,\,x_{n+1})

Each frame is still calibrated and debayered as usual and still stretched to target_median = 0.3 — stars across the frames may have different brightnesses but the max accumulator always keeps the brightest sample at each pixel. No alignment is performed (that's the whole point — tracking off, let the stars sweep), no metrics are collected and no culling is done. The frame counter still ticks and the exposure accumulator still accumulates, so you can see how long you've been building the trail.

12.13 Error handling and retry policy

Every pipeline stage is bracketed by _should_stop() checks so Stop is responsive mid-frame. Exceptions during process_frame are caught and reflected to the status line without crashing the dialog; the file's fail counter in the probe record is incremented and a 10 s penalty window is set. After three failures the file is marked permanently processed so the queue drains past it; this is the right call for e.g. a half-written file the capture app aborted and left behind.

QApplication.processEvents() is called at key points through the pipeline so the UI stays responsive during long frames (large debayer, big affine warp). The dialog connects to QApplication.aboutToQuit to call stop_live() at app shutdown, so you never leave a polling timer running past exit.

Credits. Live Stacking — the watch-folder orchestration, file-stability probe, three-phase linear↔μσ bootstrap-to-Welford transition in both single and per-filter variants, Mono→Color FILTER-header bucketing with seven narrowband mapping modes including the auto-Natural logic, the SNR proxy on the live metrics plot, per-filter flat support, red/blue culling display, star-trail max accumulator, and the whole UI and preview engine — is original work by Franklin Marek for SetiAstro Suite Pro. Standard components credited to community: incremental mean and variance follow Welford (1962, “Note on a method for calculating corrected sums of squares and products”), with the pairwise-formulation refinement attributed to Chan, Golub & LeVeque; μ–σ clipping (sigma-clipping) is classical outlier rejection in observational astronomy. Star detection and background estimation use SEP (Barbary 2016, J. Open Source Softw. 1(6):58; a Python port of Bertin & Arnouts 1996's Source Extractor). Star-based affine registration uses Astroalign (Beroiz, Cabral & Sanchez 2020, Astronomy and Computing 32:100384). Camera-RAW decoding uses rawpy (a Python binding to LibRaw); EXIF parsing uses exifread. FITS I/O and header handling use Astropy (Astropy Collaboration 2013, 2018, 2022). The affine-warp application and debayer kernels are SASpro's shared helpers (ported from the Stacking Suite). OpenCV is credited as an optional backend for grayscale conversion in the SNR proxy. PyQtGraph renders the live metrics plot, NumPy provides the array engine, and PyQt6 / Qt QGraphicsView the UI framework and interactive preview.