Metadata-Version: 2.4
Name: ctao-lst-dl0-converter
Version: 0.1.0rc1
Summary: Convert LST-1 R0V data to DL0 format
Author-email: Maximilian Linhoff <maximilian.linhoff@cta-observatory.org>, Bastien Lacave <bastien.lacave@unige.ch>
License-Expression: BSD-3-Clause
Project-URL: repository, https://gitlab.cta-observatory.org/cta-computing/common/lst_dl0_converter
Project-URL: documentation, http://cta-computing.gitlab-pages.cta-observatory.org/documentation/lst_dl0_converter
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: ctapipe
Requires-Dist: ctapipe_io_lst
Requires-Dist: ctao-acada-mock~=0.1.0
Provides-Extra: test
Requires-Dist: pytest; extra == "test"
Requires-Dist: pytest-cov; extra == "test"
Requires-Dist: ctapipe_io_zfits; extra == "test"
Provides-Extra: doc
Requires-Dist: sphinx; extra == "doc"
Requires-Dist: numpydoc; extra == "doc"
Requires-Dist: ctao-sphinx-theme~=0.1.2; extra == "doc"
Requires-Dist: sphinx-changelog; extra == "doc"
Requires-Dist: sphinx-automodapi; extra == "doc"
Provides-Extra: dev
Requires-Dist: setuptools_scm; extra == "dev"
Requires-Dist: towncrier; extra == "dev"
Requires-Dist: pre-commit; extra == "dev"
Provides-Extra: all
Requires-Dist: lst_dl0_converter[dev,doc,test]; extra == "all"
Dynamic: license-file

# lst-dl0-converter

`lst-dl0-converter` is a tool designed to convert LST-1 R0 data files into CTAO DL0 format.
This makes it possible to analyze LST-1 data using standard CTAO software (DPPS).

Most LST commissioning data has been written as uncalibrated R0 data using
an older definition of the R1 event data structure (`ProtoR1`).

To convert this data to DL0 compliant with the data model and the current
ACADA to DPPS ICD, several steps are necessary:

- Calibrate the waveforms applying DRS4 corrections and calibration from adc counts to photo electrons
- Computing `pixel_timeshift` from DRS4 time correction and flatfielding
- For older data: compute best event timestamp from available sources.
- For older data: determine R1 event_type from lower level trigger bit information
- Optional: using lstchain muon analysis results to tag muon candidate events
- Optional: using lstchain DVR pixel selection to apply DVR. Most data still available
  in the onsite cluster is already data volume reduced, but not calibrated (R0V), so this step
  should not be necessary in most cases.
- Write out the calibrated, tagged events to DL0 files using the directory structure and
  filenames as prescribed by the ACADA-DPPS ICD.


Reading the R0 data and calibrating it to R1 is handled by [`ctapipe_io_lst`](https://github.com/cta-observatory/ctapipe_io_lst),
converting the R1 to DL0 is performed by code in this repository and most of the writing
is handled by the [`acada-mock`](https://gitlab.cta-observatory.org/cta-computing/dpps/acada-mock) package.

Applying the DRS4 corrections is compute intensive. For this reasons, the converter
will work best when converting LST subruns in parallel to equivalent chunks.
This however violates the chunking prescribed by the ACADA-DPPS ICD:

- Files with shower candidates will not follow the event_id based chunking and will contain roughly 13000 events,
  that's the LST chunk size of 13250 minus the calibration events of that chunk.
- Files with calibration events will be much smaller than the chunk size, as any given subrun will only contain hundreds of calibration events.
- Files with the muon event candidates will be very small.

Since all LST data that needs this conversion is monoscopic LST-1 data, this does not seem to be an issue.
The chunking could be made to follow the ICD in a second step, but this is currently not planned.

## Finding calibration files

The converter needs several files with calibration coefficients. While the converter itself
does not depend on `lstchain` and it is at least currently not possible to have `lstchain` in
the same environment due to different requirements on `ctapipe`, this repository provides a script
using `lstchain` to find the needed calibration files in the IT cluster and put them into a config file
for the converter.

Use a separate environment with `lstchain` and run the script to find the files for a given run:

```
$ python scripts/create_config_for_run.py 11961 > config_11961.yaml
```
