Metadata-Version: 2.4
Name: scar-calib
Version: 2.2.0
Summary: Satellite-based Calibration for Aerial Recordings
Author-email: Henry Hölzemann <henry.hoelzemann@fkie.fraunhofer.de>
License: MIT License
        
        Copyright (c) 2026 Henry Hölzemann, Michael Schleiss
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.24
Requires-Dist: scipy>=1.10
Requires-Dist: gtsam>=4.2
Requires-Dist: pyproj>=3.6
Requires-Dist: opencv-python>=4.8
Requires-Dist: pyyaml>=6.0
Provides-Extra: legacy
Requires-Dist: utm==0.8.1; extra == "legacy"
Provides-Extra: references
Requires-Dist: georay>=0.1; extra == "references"
Requires-Dist: rasterio>=1.3; extra == "references"
Provides-Extra: germany
Requires-Dist: georay[germany]>=0.1; extra == "germany"
Provides-Extra: matching
Requires-Dist: torch; extra == "matching"
Requires-Dist: vismatch>=1.3; extra == "matching"
Provides-Extra: viewer
Requires-Dist: rerun-sdk>=0.23; extra == "viewer"
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: ruff>=0.6; extra == "dev"
Dynamic: license-file

<table align="center"><tr><td align="center" width="9999">
<img src="assets/scar_icon.png" align="center" width="150" alt="SCAR icon">

# SCAR

**S**atellite-based **C**alibration for **A**erial **R**ecordings
</td></tr></table>

SCAR refines the intrinsics of an aerial camera and its extrinsics relative to a GNSS/INS from the
recording itself: it matches the images against georeferenced orthophotos, lifts the matches to 3D
anchors with a surface model and optimizes a factor graph (GTSAM) over cameras, anchors and the
calibration. No calibration target or ground control points are needed.

You provide images with timestamps, INS poses, an initial calibration and reference data:

- **Germany:** official orthophotos and DSMs of the federal states supported by
  [georaffer](https://github.com/michaelschleiss/georaffer) (NRW, RLP, BW, BY, BB, ...) are downloaded on
  demand.
- **Anywhere else:** your own orthophoto and surface model as GeoTIFF/COG/VRT in any CRS (e.g. NAIP +
  3DEP in the US, national mapping data, your own survey), or your own reference provider.

Frame cameras and event cameras (as reconstructed frames) are supported. The paper version is kept
frozen as the preset `cvprw2026` and the tag `v1.0.0-cvprw2026`.

## Installation

SCAR needs Python 3.11+.

```bash
git clone https://github.com/hlzmnhnry/scar.git && cd scar
pixi install            # environment with GTSAM, pyproj, rasterio, georay (CPU only)
pixi install -e gpu     # + PyTorch/CUDA, learned matchers (vismatch), georaffer, rerun viewer
```

With pip: `pip install -e ".[references,matching,viewer]"` (add `germany` for German state data).
[georay](https://github.com/hlzmnhnry/georay) renders the reference views. The `germany` extra adds
georaffer.

## Quick start

A complete run on a synthetic flight with local GeoTIFF reference data, CPU only and without downloads:

```bash
pixi run python examples/custom_region_geotiff/make_demo_data.py examples/custom_region_geotiff/demo
pixi run scar run examples/custom_region_geotiff/config.yaml
pixi run python examples/custom_region_geotiff/make_demo_data.py --compare examples/custom_region_geotiff/demo
```

For your own flight:

```bash
scar init my_flight.yaml --template csv     # or: ltc, event
scar check my_flight.yaml --render          # time coverage, CRS, reference availability, footprint overlays
scar run my_flight.yaml --viewer            # collect anchors and optimize, live view in rerun
scar reproject my_flight.yaml --every 50    # reference points projected with initial and calibrated result
scar export runs/my_flight/calibration/calibration.yaml --format kalibr --output camchain.yaml
```

`scar reproject` shows calibrations on the images: reference points, matched as during collection
(or `--source anchors`: the anchors of the run), projected with the pose priors, next to the reference
view, the legend gives the median error and the mean offset per calibration. By default it compares the
initial and the calibrated result, `--calibration` takes any others (also camera and extrinsics files,
`--calibration old=camera.json,extrinsics.json`), `--vector-scale` magnifies the offsets and `--save`
writes the images instead of opening a window.

`scar collect` and `scar optimize` run the two steps separately, `scar evaluate` scores a calibration on
anchors of an independent test sequence (reprojection error, optionally visual localization), and
`scar anchor-noise` estimates how precise the collected anchors are by matching tracked observations
against the reference data again. See [`examples/`](examples) for an LTC campaign;
`scar init --template event` starts an event camera setup.

## Data

- **LTC dataset** ([Leaving the City](https://hlzmnhnry.github.io/publications/ltc), ECCV 2026): the full
  aerial recordings with INS data and calibrations, licensed under CC BY 4.0. SCAR reads the campaign
  folders directly (`scar init my_ltc.yaml --template ltc`). Images are distributed in archives
  (`<campaign>_images/<index>.png`), `meta/image_shards.csv` lists which frames each archive contains.
- **SCAR evaluation data**: the subset used in the paper (per campaign every 10th image of the training and
  test segments, `camera.csv`, `ins.csv`) is available
  [here](https://owncloud.fraunhofer.de/index.php/s/Cf7U6xfx5YHMRSs). Put a campaign folder at
  `data/<campaign>/` and run, e.g. for 2022-10-19:

  ```bash
  pixi install -e gpu
  pixi run -e gpu scar run examples/ltc_quickstart/config.yaml
  ```

  The configuration needs the GCG2016 quasigeoid (see [examples/README.md](examples/README.md)), reference
  tiles are downloaded automatically. For the other campaigns copy the configuration and set the campaign
  and the time window from [`paper/cvprw2026/campaigns.yaml`](paper/cvprw2026/campaigns.yaml).

If you use LTC data, please also cite the
[LTC paper](https://hlzmnhnry.github.io/publications/ltc).

## Inputs

All inputs are described in one YAML file, the templates in
[`scar/config/templates`](scar/config/templates) document every option.

| Input | Format |
| --- | --- |
| Images | CSV with timestamp and path, a directory of files named by timestamp, or LTC `camera.csv` |
| Poses | CSV with geodetic (lat, lon, height) or projected positions in any CRS, quaternion or Euler orientation w.r.t. NED or ENU, optional per-sample accuracies |
| Calibration | SCAR YAML, Kalibr camchain or LTC `camera.json`/`extrinsics.json`, pinhole with radial-tangential distortion |
| Reference data | georaffer regions (Germany) or local orthophoto and DSM rasters |

Poses are interpolated to the image timestamps (linear position, SLERP rotation), optionally with a
time offset. The conventions for frames, heights, datums and the meridian convergence are described in
[docs/coordinate_systems.md](docs/coordinate_systems.md), please read it before using your own data.

Images are selected in one time window (`selection.start`/`end`) or in several `selection.segments`.
A few short segments with different headings (ideally opposite ones) and altitudes constrain the
calibration better than one long straight leg: on a single heading a boresight error cannot be told
apart from a world-fixed offset of positions or reference data. `scar check --matches N` matches a few
frames per segment and shows whether the terrain gives enough anchors, anchors are not tracked across
segments. With large image motion between frames, `collect.settings.tracking_config.forward_backward_threshold_px:
1.0` drops tracks that lag behind, for test anchors seconds apart, switch tracking off
(`tracking: false`, `collect_every_nth_frame: 1`). `scar evaluate` leaves out observations that project
behind the camera or far outside the image, and `--offsets` those with gross errors.

## Outputs

A run directory contains the anchors (`anchors.json`), the per-stage results (`calibration/stages.json`),
the final calibration with the marginal covariance of the intrinsics (`calibration/calibration.yaml`)
and, with `--viewer`, a rerun recording. `scar export` writes Kalibr, OpenCV and LTC formats.

## Optimization presets

| Preset | Description |
| --- | --- |
| `default` | Staged schedule of the paper (rotation, translation, landmarks, calibration, full) followed by a robust least-squares fit of the extrinsics (frames whose pose estimate failed do not pull the result), use with the corrected pose processing (default of all templates) |
| `cvprw2026` | Frozen paper configuration, together with `paper/cvprw2026` it reproduces the published results |
| `joint` | Experimental: extrinsics as a variable of the factor graph instead of the least-squares fit |

## Reproducing the paper

[`paper/cvprw2026`](paper/cvprw2026) contains the scripts, campaign definitions and initial calibrations
of the CVPRW 2026 evaluation, the reproduction log and an ablation of geodetic corrections that were not
part of the paper version (float32 coordinates, meridian convergence, heights, interpolation).
The meridian convergence correction alone lowers the test reprojection error by 21 % on average over the
six LTC campaigns. `anchor_noise_ltc.py` estimates the anchor noise of an LTC sequence, the empirical
counterpart of the landmark prior.

## Development

```bash
pixi run test           # unit, synthetic and system tests, regression tests need local LTC data
pixi run -e repro test  # bit-exact paper regression with GTSAM 4.2.0
```

## Citation

If you use SCAR in academic work, please cite:

```bibtex
@InProceedings{Holzemann_2026_CVPR,
    author    = {H\"olzemann, Henry and Schleiss, Michael},
    title     = {SCAR: Satellite Imagery-Based Calibration for Aerial Recordings},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
    month     = {June},
    year      = {2026},
    pages     = {987-996}
}
```

Reference data obtained through georaffer are subject to the licenses of the respective providers
(e.g. NRW: dl-de/zero-2.0, RLP: dl-de/by-2.0), SCAR does not redistribute them.
