Metadata-Version: 2.4
Name: ftsdata
Version: 0.1.0
Summary: Fourier-transform spectrometer data containers, transforms, I/O and co-addition
Author-email: Alexandre Beelen <alexandre.beelen@lam.fr>
License: Copyright (c) 2019, Alexandre Beelen
        Laboratory. All rights reserved.
        
        Redistribution and use in source and binary forms, with or without
        modification, are permitted provided that the following conditions are met:
        
        * Redistributions of source code must retain the above copyright notice, this
          list of conditions and the following disclaimer.
        
        * Redistributions in binary form must reproduce the above copyright notice,
          this list of conditions and the following disclaimer in the documentation
          and/or other materials provided with the distribution.
        
        * Neither the name of the Brookhaven Science Associates, Brookhaven National
          Laboratory nor the names of its contributors may be used to endorse or promote
          products derived from this software without specific prior written permission.
        
        THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
        AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
        IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
        DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
        FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
        DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
        SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
        CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
        OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
        OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Project-URL: Homepage, https://gitlab.lam.fr/KISS/ftsdata
Project-URL: Repository, https://gitlab.lam.fr/KISS/ftsdata
Project-URL: Documentation, https://ftsdata-7333b4.docs.lam.fr
Project-URL: GitHub Pages, https://abeelen.github.io/ftsdata/
Project-URL: Read the Docs, https://ftsdata.readthedocs.io/
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: BSD License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Astronomy
Requires-Python: >=3.8
Description-Content-Type: text/x-rst
License-File: LICENSE
Requires-Dist: numpy>=1.24.4; python_version < "3.9"
Requires-Dist: numpy>=2.0.2; python_version >= "3.9" and python_version < "3.11"
Requires-Dist: numpy>=2.3.0; python_version >= "3.11"
Requires-Dist: scipy>=1.10.1; python_version < "3.9"
Requires-Dist: scipy>=1.13.1; python_version >= "3.9" and python_version < "3.11"
Requires-Dist: scipy>=1.15.0; python_version >= "3.11"
Requires-Dist: astropy>=5.2.2; python_version < "3.9"
Requires-Dist: astropy>=6.0.1; python_version >= "3.9" and python_version < "3.11"
Requires-Dist: astropy>=7.0.0; python_version >= "3.11"
Provides-Extra: test
Requires-Dist: pytest; extra == "test"
Requires-Dist: pytest-cov; extra == "test"
Provides-Extra: hdf5
Requires-Dist: h5py; extra == "hdf5"
Provides-Extra: dask
Requires-Dist: dask; extra == "dask"
Provides-Extra: plot
Requires-Dist: matplotlib; extra == "plot"
Provides-Extra: psutil
Requires-Dist: psutil; extra == "psutil"
Provides-Extra: all
Requires-Dist: h5py; extra == "all"
Requires-Dist: dask; extra == "all"
Requires-Dist: matplotlib; extra == "all"
Requires-Dist: psutil; extra == "all"
Provides-Extra: docs
Requires-Dist: sphinx>=7.0; extra == "docs"
Requires-Dist: sphinx-rtd-theme>=2.0; extra == "docs"
Requires-Dist: numpydoc>=1.5; extra == "docs"
Requires-Dist: sphinx-copybutton; extra == "docs"
Requires-Dist: myst-parser>=3.0; extra == "docs"
Requires-Dist: sphinxcontrib-mermaid>=0.9; extra == "docs"
Requires-Dist: matplotlib; extra == "docs"
Dynamic: license-file

ftsdata
=========

|GitLab pipeline status| |GitHub Actions status| |docs| |Read the Docs|
|SPEC 0 — Minimum Supported Dependencies|

``ftsdata`` is the Fourier-transform spectrometer (FTS) data toolkit for
the CONCERTO and KIDS reductions. It provides OPD- and frequency-aware
data containers, interferogram/spectrum transforms, FITS and IDIA HDF5
I/O, and inverse-variance co-addition of spectral cubes.

The package used to be the single ``kidsdata.ftsdata`` module. It is now
an independent distribution, and existing code keeps working through the
``kidsdata.ftsdata`` compatibility shim.

Features
--------

- **Data containers** — ``SingleSidedOpdData``, ``DoubleSidedOpdData``,
  ``OneSidedOpdData``, ``SpecData``, and the retro-compatible
  ``FTSData``. ``OpdData`` is the geometry-agnostic entry point
  (``from_array`` / ``read``).
- **Interferogram → spectrum** — Forman phase correction, apodisation,
  zero-padding and ITG binning correction.
- **Spectrum → interferogram** — ``to_itg`` for forward modelling.
- **I/O** — FITS products registered with Astropy’s unified I/O, plus
  export to the IDIA HDF5 schema used by CARTA.
- **Large cubes** — thread-based or dask-backed blockwise processing.
- **Co-addition** — ``ftsdata_average`` combines cubes with per-pixel
  inverse-variance weighting.
- **Optional kidsdata integration** — opacity correction can use
  ``kidsdata.forward_model.ATM`` when kidsdata is installed.

Support policy
--------------

``ftsdata`` follows `SPEC 0 — Minimum Supported
Dependencies <https://scientific-python.org/specs/spec-0000/>`__ for
Python and core dependency support, with one explicit exception:
**Python 3.8 remains supported** for the CONCERTO environment used by
``kidsdata``.

On Python 3.8 the package uses the last compatible NumPy, SciPy and
Astropy releases; newer Python versions use the current SPEC 0 floors.
Minimum versions are raised on minor or major releases, never on patch
releases. The current floors and supported versions are listed in
`docs/min_versions.md <docs/min_versions.md>`__.

Python 3.8 and the associated Astropy 5.2 stack are targeted for removal in
``ftsdata`` 0.2.0, or once the CONCERTO environment used by ``kidsdata`` moves
to Python 3.11+ and Astropy 7+.

Installation
------------

The package is not on PyPI yet; install it from this repository (Python
3.8+):

.. code:: bash

   git clone git@gitlab.lam.fr:KISS/ftsdata.git
   cd ftsdata
   python -m pip install -e .

Optional features are grouped in extras:

.. code:: bash

   python -m pip install -e ".[all]"     # h5py, dask, matplotlib, psutil
   python -m pip install -e ".[hdf5]"    # HDF5 / IDIA export
   python -m pip install -e ".[dask]"    # dask-backed blockwise transforms
   python -m pip install -e ".[plot]"    # cube previews
   python -m pip install -e ".[docs]"    # Sphinx documentation build
   python -m pip install -e ".[test]"    # pytest and pytest-cov

Only NumPy, SciPy and Astropy are required for the core containers and
transforms. ``h5py`` enables HDF5 I/O, ``dask`` enables dask-backed
transforms, ``matplotlib`` enables ``plot_cube_preview``, and ``psutil``
improves memory detection.

Quick start
-----------

.. code:: python

   import astropy.units as u
   import numpy as np
   from ftsdata import SingleSidedOpdData

   # 101 OPD samples with a 100 µm step, symmetric around the ZPD
   opd = np.arange(-50, 51) * 0.1 * u.mm
   data = np.random.default_rng(42).normal(size=(opd.size, 16, 16))

   # from_array inspects the OPD geometry and returns the matching subclass
   itg = SingleSidedOpdData.from_array(opd, data)
   print(type(itg).__name__)          # DoubleSidedOpdData

   # Apodise, zero-pad and FFT to a spectral cube
   spectrum = itg.to_spectra(apodization=np.hanning, n_pad=3)
   print(spectrum.data.shape)         # (nfreq, 16, 16)
   print(spectrum.spectral_axis.to("GHz")[:3])

   spectrum.write("spectrum.fits", overwrite=True)

Read a product back with a compatible class:

.. code:: python

   from ftsdata import FTSData

   cube = FTSData.read("spectrum.fits")
   print(cube.data.shape)

Data model
----------

All containers derive from ``astropy.nddata.NDDataArray`` and carry:

+-----------------+----------------------------------------------------+
| Attribute       | Description                                        |
+=================+====================================================+
| ``data``        | Data cube, normally ``(nopd, ny, nx)`` or          |
|                 | ``(nfreq, ny, nx)``                                |
+-----------------+----------------------------------------------------+
| ``uncertainty`` | Astropy uncertainty array                          |
+-----------------+----------------------------------------------------+
| ``mask``        | Boolean bad-pixel mask (``True`` = invalid)        |
+-----------------+----------------------------------------------------+
| ``wcs``         | Full WCS (sky axes plus OPD or frequency)          |
+-----------------+----------------------------------------------------+
| ``meta``        | Provenance and pipeline parameters                 |
+-----------------+----------------------------------------------------+
| ``hits``        | Number of detector samples per voxel               |
+-----------------+----------------------------------------------------+

+-------------------------------------+--------------------------------+
| Class                               | Role                           |
+=====================================+================================+
| ``SingleSidedOpdData``              | Base interferogram cube;       |
|                                     | ``from_array`` dispatches on   |
|                                     | OPD geometry                   |
+-------------------------------------+--------------------------------+
| ``DoubleSidedOpdData``              | Symmetric OPD grid with the    |
|                                     | ZPD centred                    |
+-------------------------------------+--------------------------------+
| ``OneSidedOpdData``                 | OPD grid starting at the ZPD;  |
|                                     | real inverse FFT               |
+-------------------------------------+--------------------------------+
| ``SpecData``                        | Spectrum cube                  |
+-------------------------------------+--------------------------------+
| ``FTSData``                         | Retro-compatible class         |
|                                     | inheriting from the            |
|                                     | interferogram and spectrum     |
|                                     | classes                        |
+-------------------------------------+--------------------------------+
| ``OpdData``                         | Generic entry point when the   |
|                                     | OPD geometry is unknown; alias |
|                                     | of ``SingleSidedOpdData``, so  |
|                                     | ``OpdData.from_array`` /       |
|                                     | ``OpdData.read`` still         |
|                                     | dispatch                       |
+-------------------------------------+--------------------------------+

I/O and co-addition
-------------------

FITS products use the Astropy unified I/O interface:

.. code:: python

   from ftsdata import FTSData

   spectrum.write("cube.fits", overwrite=True)
   spectrum.write("cube-compressed.fits", compress=True, overwrite=True)
   cube = FTSData.read("cube.fits")

For CARTA, write the IDIA HDF5 layout directly — the ``.hdf5`` extension
selects the registered writer and reader:

.. code:: python

   from ftsdata import OpdData

   cube.write("cube.hdf5", mip_levels=[2, 4, 8], dtype="float32")
   cube = OpdData.read("cube.hdf5")   # returns the matching geometry subclass

The writer creates the ``/0`` group and the ``VERSION`` attribute
required by the IDIA schema. Pass ``format="hdf5"`` explicitly when the
filename lacks an ``.hdf5`` or ``.h5`` extension.

Co-add cubes that share the same WCS grid:

.. code:: python

   from pathlib import Path
   from ftsdata import FTSData, ftsdata_average

   cubes = [FTSData.read(path) for path in sorted(Path("products").glob("*_T.fits"))]
   combined = ftsdata_average(cubes)
   combined.write("combined_T.fits", overwrite=True)

Relationship with kidsdata
--------------------------

``ftsdata`` does not import ``kidsdata`` at module import time, so it
can be used on its own. When the ``kidsdata`` distribution is installed,
two integrations become available lazily:

- ``available_memory()`` uses ``kidsdata.utils.available_memory`` when
  available, which knows about SLURM allocations.

- ``SpecData.correct_opacity()`` uses ``kidsdata.forward_model.ATM`` by
  default. Without kidsdata, pass a compatible model explicitly:

  .. code:: python

     cube.correct_opacity(elevation=60, pwv=1.0, atm=my_atm_model)

Existing code can keep using the old import path:

.. code:: python

   from kidsdata.ftsdata import FTSData

New code should import from ``ftsdata`` directly.

Documentation
-------------

The Sphinx sources live in ``docs/``:

.. code:: bash

   python -m pip install -e ".[docs]"
   python -m sphinx -b html docs docs/_build/html

The built pages are published in three places from the ``main`` branch:

- GitLab Pages: https://ftsdata-7333b4.docs.lam.fr
- GitHub Pages: https://abeelen.github.io/ftsdata/ (GitHub Actions)
- Read the Docs: https://ftsdata.readthedocs.io/ (configured by
  ``.readthedocs.yaml``)

Tests
-----

.. code:: bash

   python -m pip install -e ".[all,test]"
   python -m pytest --pyargs ftsdata -q

The GitLab and GitHub CI pipelines run the test-suite on several Python
versions and build the documentation with warnings treated as errors.

Repository layout
-----------------

::

   ftsdata/            package modules and pytest suite (ftsdata/tests)
   docs/               Sphinx sources
   licenses/           license files
   pyproject.toml      packaging and tool configuration
   tox.ini             test and documentation environments
   .gitlab-ci.yml      tests and GitLab Pages
   .github/workflows/  GitHub Actions tests and Pages

License
-------

BSD 3-clause. See `licenses/LICENSE.rst <licenses/LICENSE.rst>`__.

.. |GitLab pipeline status| image:: https://gitlab.lam.fr/KISS/ftsdata/badges/main/pipeline.svg
   :target: https://gitlab.lam.fr/KISS/ftsdata/-/pipelines
.. |GitHub Actions status| image:: https://github.com/abeelen/ftsdata/actions/workflows/ci.yml/badge.svg?branch=main
   :target: https://github.com/abeelen/ftsdata/actions/workflows/ci.yml
.. |docs| image:: https://img.shields.io/badge/docs-GitHub%20Pages-blue
   :target: https://abeelen.github.io/ftsdata/
.. |Read the Docs| image:: https://readthedocs.org/projects/ftsdata/badge/?version=latest
   :target: https://ftsdata.readthedocs.io/en/latest/
.. |SPEC 0 — Minimum Supported Dependencies| image:: https://img.shields.io/badge/SPEC-0-green?labelColor=%23004811&color=%235CA038
   :target: https://scientific-python.org/specs/spec-0000/
