Metadata-Version: 2.4
Name: atlas-fit
Version: 2.0.0
Summary: Tool to perform comparison and calibration of flat fielded flat images against a solar atlas.
Home-page: https://gitlab.gwdg.de/hoelken/atlas-fit
Author: Johannes Hölken
Author-email: hoelken@mps.mpg.de
Project-URL: Bug Tracker, https://gitlab.gwdg.de/hoelken/atlas-fit/issues
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: BSD License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: spectroflat>=0.5
Requires-Dist: matplotlib>=3.5
Requires-Dist: numpy>=1.23
Requires-Dist: scipy>=1.9
Requires-Dist: pyyaml
Requires-Dist: astropy
Requires-Dist: pandas
Requires-Dist: mpl-point-clicker
Requires-Dist: qollib
Dynamic: license-file

# Atlas-Fit

Python tools to fit solar spectra with known atlantes for 
 - **wavelength calibration**
 - **continuum correction**
 - stray-light analysis
 - effective FWHM computation
 - chi^2 error computation of the atlas fit

### Summary
This package provides two tools:

- `bin/setup`:  
    This script is used to test and fine-tune the configuration 
    and to inspect the fitting and analysis results with interactive 
    plots.
- `bin/run`:  
    This tool allows to amend the calibration results from the
    `spectroflat` library. The fitted dispersion can be applied 
    to the `OffsetMap` (to include wavelength calibration).  
    The straylight and continuum correction will be appended to 
    the calibrated illumination pattern (soft flat).

Both tools can (and should) be run with the same configuration. 
Run the `setup` first to find the right configuration values 
and use the `run` script to apply the results.

### Getting Started
#### Installation
We recommend the installation via `pip install altas_fit`.  
You can also pull from the repository, but be aware that the main branch is not frozen at the latest stable release.
Please check out a [git tag](https://gitlab.gwdg.de/hoelken/atlas_fit/-/tags) to be in a defined and tested state.

#### Example
Verify the installation and get a feeling for how to use it with the working example. 
It is provided in the `examples` folder. You can run it with
```bash
$ bin/setup examples/config.yml examples/fine_range.yml
``` 

#### With your own data
Please refer to the **[QUICKSTART GUIDE](QUICKSTART.md)** for a brief introduction
on how to create and test the config for your data.

### Input/Output data 
#### Input
This library works on the calibration data from `spectroflat`. It uses 
```yaml
input:
  corrected_frame: flat_fielded_flat.fits  # The calibrated flat field frame (state, slit, wavelength)
  soft_flat: illumination_pattern.fits     # The slowly varying illumination patter (or 'soft flat') (state, slit, wavelength)
  offset_map: offset_map.fits              # The measured shifts from the smile correction (i.e., spectroflat::OffsetMap)
```

#### Output
The generated output are amended versions of the offset map and illumination pattern. 

The offsets in the **amended OffsetMap** contain the smile and wavelength correction, such that 
the science data can be interpolated to a uniform linear wavelength grid in one go.
It further contains now a wavelength axis and wavelength keywords in the header.

The **amended Illumination Pattern** contains 
- HDU 0: the original illumination pattern shifted to the wavelength calibrated position
- HDU 1: 2D continuum correction surface 
- HDU 2: 2D gray straylight surface
- HDU 3: 1D wavelength axis
You need to scale the straylight surface with the intensity mean prior to subtraction.
To keep [DN] values after continuum correction divide the continuum correction surface by its mean value. 

THe header contains WCS ready wavelength information. 
However, you might need to re-process and adjust this within your pipeline


### Citation

When using this library please cite:

Hölken et al. "Spectroflat: A generic spectrum and flat-field calibration library for spectro-polarimetric data"
([DOI: 10.1051/0004-6361/202348877](https://doi.org/10.1051/0004-6361/202348877))
```bibtex
@ARTICLE{spectroflat,
       author = {{H{\"o}lken}, J. and {Doerr}, H.-P. and {Feller}, A. and {Iglesias}, F.~A.},
        title = "{Spectroflat: A generic spectrum and flat-field calibration library for spectro-polarimetric data}",
      journal = {\aap},
     keywords = {instrumentation: polarimeters, instrumentation: spectrographs, methods: data analysis, methods: numerical},
         year = 2024,
        month = jul,
       volume = {687},
          eid = {A22},
        pages = {A22},
          doi = {10.1051/0004-6361/202348877},
}
```

## Versioning
We follow the [Semantic Versioning](https://semver.org/) pattern. In short semantic versioning boils down to:
- PATCH 0.0.x level changes for implementation level detail changes, such as small bug fixes
- MINOR 0.x.0 level changes for any backwards compatible API changes, such as new functionality/features
- MAJOR x.0.0 level changes for backwards incompatible API and algorithm changes, such as changes that will
  break existing users code if they update and or change the outcome of the code notably.

Please find a list of relevant releases on the [git tags](https://gitlab.gwdg.de/hoelken/atlas_fit/-/tags)
and [releases](https://gitlab.gwdg.de/hoelken/atlas_fit/-/releases) pages. Upcoming versions are described as
[milestones](https://gitlab.gwdg.de/hoelken/atlas_fit/-/milestones).

### Original algorithm
The `atlas_fit` algorithm `1.X.X` is described and evaluated in
Hölken et al. (2024) "Spectroflat: A generic calibration library for spectro-polarimetric data". [see above]

### Changes in `2.X.X`:
We have re-implemented the line selection. Now it is no longer required to click on individual lines.
Instead, the user can now define the range in the atlas and fine tune the automated selection if needed. 

- [**Braking change**] The scripts have new name `prepare`->`setup` and `amend_spectroflat`->`run`. 
- [**Braking change**] We have simplified the configuration by large. 
   The following config entries are no longer suppoerted and must be removed from the config. Removed config entries are:
  - `input:roi` use `input:correction_roi` to define the to-be-corrected area 
  - `fitting:lowpass_filter` no longer needed as the new pre-straighten algorithm is more robust.
  - `fitting:high_quantile_continuum` no longer needed as the new pre-straighten algorithm is more robust.
  - `fitting:continuum_spline_smoothing` no longer needed as the continuum extraction is now robust
  - `fitting:continuum_points` no longer needed as the continuum extraction is now robust
  - `fitting:continuum_points` no longer needed as the continuum extraction is now robust
  - `fitting:dispersion_min_degree` no longer needed as dispersion is now measured, not fitted
  - `fitting:dispersion_max_degree` no longer needed as dispersion is now measured, not fitted
- **Line Selection** Lines are now pre-selected by their bisector. If deviation is too large (e.g. asymmetry) 
   the line is disregarded. This ensures better measurement of the wavelength solution 
- **Wavelength Solution** The wavelength solution is now directly measured from the discrepancy of the line positions
  in atlas and data space. From this discrepancy shifts to a linear wavelength scale are computed.
- **Continuum Correction** We now directly fit a high order polynomial through the ration of atlas and data.
   The fit is weighted to follow the continuum points and not the lines. 
- **Non-global straylight** The algorithm now supports gray straylight with a linear gradient or second order 
  curvature. By default, the penalty on these terms is high to favor global straylight terms.
- **WCS headers** We now provide WCS conform header information on the wavelength calibration  
- New config entries (A full description can be found in the API documentation):
  - `input:straighten` This was a command line switch before, and should have been a config item
  - `fitting:continuum_trend_degree` To be used if `input:straighten` is `True` 
  - `fitting:continuum_level` To tune automatic selection of lines
  - `fitting:max_bisector_nonuniformity` To tune automatic selection of lines
  - `fitting:straylight_gradient_penalty` Penalty to fit non-global straylight
  - `fitting:straylight_non_linearity_penalty` Penalty to fit non-global non-linear straylight
  


### Users
This routine is part of the **SUNRISE-III/SUSI** data reduction pipeline. 
It is actively used for the calibration of **GST/FISS-SP** data.

It has additionally been tested with **GREGOR/GRIS**, **SST/TRIPPEL**, **ZIMPOL**, and other datasets.

### Support
Developed and maintained by `hoelken@mps.mpg.de`

Contributions by `iglesias@mps.mpg.de` and `feller@mps.mpg.de`.


### Contributing
Any contribution that follows our [Code of Conduct](CODE_OF_CONDUCT.md) and that helps to advance the `atlas_fit`
library is highly welcome.

**Bug reports & feature requests** are welcome via gitlab issues or email.
Issues can either be opened via the web interface (needs access to the gitlab.gwdg.de)
or send to the maintainer(s) email address(es) directly.

To ease bug fixing please provide a reproducible description and/or relevant testing data in bug reports.
If provided data needs to be handled confidentially, please email the maintainer(s) directly.

**Code contributions** are welcome as merge or pull requests and will undergo a code review.
Any provided code must satisfy the python code style (i.e., the `stylecheck.bash` needs to pass) and
have at least 80 % test coverage.
Development forks of the repository are explicitly recommended.


## License
This code is developed and maintained at the Max Planck Institute for
Solar System Research (MPS) Göttingen.

License: BSD clause-2, see [LICENSE](LICENSE)
