Metadata-Version: 2.4
Name: sed-tools
Version: 0.5.0
Summary: Programmatic access to SED Tools stellar spectral grids and photometry helpers.
Author: Niall Miller
Project-URL: Homepage, https://github.com/nialljmiller/SED_Tools
Keywords: astronomy,spectra,photometry,sed
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Astronomy
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=2.0
Requires-Dist: pandas>=1.5
Requires-Dist: tqdm>=4.66
Requires-Dist: requests>=2.31
Requires-Dist: beautifulsoup4>=4.12
Requires-Dist: lxml>=4.9
Requires-Dist: h5py>=3.10
Requires-Dist: astropy>=6.1
Requires-Dist: astroquery>=0.4.7
Requires-Dist: matplotlib>=3.7
Requires-Dist: tomli>=2.0; python_version < "3.11"
Provides-Extra: full
Requires-Dist: scipy>=1.14; extra == "full"
Requires-Dist: xlrd>=2.0; extra == "full"
Requires-Dist: scikit-learn; extra == "full"
Requires-Dist: torch; extra == "full"
Requires-Dist: dust_extinction>=1.6; extra == "full"
Provides-Extra: extinction
Requires-Dist: dust_extinction>=1.6; extra == "extinction"
Provides-Extra: ml-tune
Requires-Dist: optuna>=3; extra == "ml-tune"
Dynamic: license-file

<p align="center">
  <img src="https://raw.githubusercontent.com/nialljmiller/SED_Tools/v0.5.0/docs/SED_Tools_Logo.jpeg" alt="SED_Tools Logo" width="200"/>
</p>

<h1 align="center">SED_Tools</h1>

<p align="center">
  <strong>Download, process, and standardize stellar spectral grids for SED_Model and MESA</strong>
</p>

<p align="center">
  <a href="https://github.com/nialljmiller/SED_Tools/actions/workflows/tests.yml"><img src="https://github.com/nialljmiller/SED_Tools/actions/workflows/tests.yml/badge.svg" alt="Tests"></a>
</p>

<p align="center">
  <a href="#installation">Installation</a> •
  <a href="#quick-start">Quick Start</a> •
  <a href="#cli-reference">CLI Reference</a> •
  <a href="#python-api">Python API</a> •
  <a href="#data-sources">Data Sources</a>
</p>

---

## Overview

SED_Tools is a Python package for working with stellar spectral energy distributions (SEDs). It provides unified access to multiple stellar spectral grids, standardizes spectral data to consistent units, and generates output files compatible with [MESA](https://docs.mesastar.org/)'s `colors` module, [SED_Model](https://github.com/nialljmiller/SED_Model) and other codes.

### Key Features

- **Multi-source downloads** — Fetch model spectra from SVO, MSG, MAST (BOSZ), and NJM mirror
- **Photometric filters** — Download transmission curves from the NJM mirror, the instrument teams' original files, or the SVO Filter Profile Service; NJM is optional
- **Optical path components** — Track which parts of the atmosphere, telescope/instrument optics, filter and detector QE are included in a passband; compose available factors for the same instrument and site
- **Detector-aware photometry** — Record photon- or energy-counting response conventions and use the corresponding integration weights for synthetic photometry
- **Unit standardization** — Convert all spectra to consistent units (wavelength in Å, flux in erg/cm²/s/Å)
- **External integration** — Generate binary flux cubes, HDF5 bundles, and lookup tables. This is for [SED_Model](https://github.com/nialljmiller/SED_Model) and [MESA](https://docs.mesastar.org/)'s `colors` module
- **Grid combination** — Merge multiple stellar spectral grids into unified "omni grids"
- **ML completion** — Extend incomplete SEDs to broader wavelength ranges using neural networks
- **ML generation** — Create complete SEDs from stellar parameters (Teff, logg, [M/H]) using neural networks

### Terminology

A **stellar spectral grid** is a collection of **model spectra** indexed by
parameters such as effective temperature, surface gravity and metallicity.
We use **spectral grids** in menus and commands. **Spectral libraries** is
the broader term when discussing collections that also include observed
templates, planetary spectra or stellar population spectra.

Use `sed-tools grids` and `--grids` to discover and download spectral grids.
The earlier `spectra` command and `--models` option remain supported aliases.
In Python, `SpectralGrid`, `SpectralGridInfo`, `sed.grid`, `sed.available_grids()`
and `find_grids` are the preferred names; `Catalog`, `CatalogInfo`, `sed.cat`,
`sed.available_models()` and `find_atmospheres` remain compatible. Existing keyword arguments such as
`catalog` and `model_root`, and the `stellar_models/` storage directory,
retain their established names.

Earth's **atmosphere transmission models**, used in ground-based optical
paths, are separate from stellar spectral grids.

---

## Installation

### From PyPI

```bash
pip install sed-tools
```

### From Source

```bash
git clone https://github.com/nialljmiller/SED_Tools.git
cd SED_Tools
pip install .
```

### Development Install

```bash
pip install -e .
```

### Requirements

- Python ≥ 3.10
- numpy, pandas, h5py, astropy, matplotlib
- PyTorch (required for ML completer and generator)

Optional extras: `pip install "sed-tools[full]"` (scipy, scikit-learn, torch, xlrd, dust_extinction), `"sed-tools[extinction]"` (dust_extinction only, for `AfAv`) and `"sed-tools[ml-tune]"` (optuna, for `ml_generator --auto`). psutil is not needed. See `pyproject.toml` for the complete dependency list.

---

## Quick Start

### Interactive Mode

Launch the interactive menu:

```bash
sed-tools
```

The menu has four filter options: 1 downloads filters, 2 combines filter sets, 3 shows what `sed-tools filters-info` shows for an instrument picked from the lists, and 4 builds a filter curve from an instrument's own parts (filter, optics, camera, atmosphere), with the questions asked after a download. Options 5-7 are the spectral grids, 8-10 machine learning and 11-14 the tools (14 is Config). In 0.4.1 the filter options were 1 and 2 only, so 0.4.1's options 3-12 are now 5-14.

### Direct Commands

```bash
# Download photometric filter transmission curves
sed-tools filters

# Combine installed filter sets
sed-tools filters-combine Gaia2MASS GAIA/GAIA 2MASS/2MASS

# Download model spectra
sed-tools grids

# Build flux cubes and lookup tables from downloaded spectra
sed-tools rebuild

# Combine multiple grids into a unified ensemble
sed-tools combine

# Train or apply the ML SED completer (extends existing spectra)
sed-tools ml_completer

# Train or apply the ML SED generator (creates SEDs from parameters)
sed-tools ml_generator

# Fill coarse Teff gaps in a flux cube to reduce interpolation stair-stepping
sed-tools densify --flux-cube data/stellar_models/tmap2/flux_cube.bin \
                  --output data/stellar_models/tmap2_dense/flux_cube.bin \
                  --teff-spacing 1000

# Ingest an external grid of .txt spectra into the pipeline
sed-tools import --path /path/to/spectra/ --name MyGrid

# Report parameter-space coverage of an installed grid
sed-tools coverage --grids Kurucz2003all

# Show or set the data directory
sed-tools config
sed-tools config --set /path/to/data --move
```

---

## CLI Reference

### Options added in 0.5.0

| Command | Option | What it does |
|---|---|---|
| `grids` (`spectra`), `rebuild`, `import` | `--flux-unit {flam,fnu,fnu_jy,flam_si,flam_si_nm,flam_si_um,normalized}` | Flux unit of spectra whose headers do not declare one (otherwise sed-tools asks in a terminal, and stops without one); spectra that declare theirs keep it |
| `grids` | `--teff LOW HIGH` | Download only spectra with Teff in this range (K); NJM, MSG and MAST cut downloads, SVO refuses cuts |
| `grids` | `--logg LOW HIGH` | Download only spectra with log g in this range |
| `grids` | `--meta LOW HIGH` | Download only spectra with [M/H] in this range |
| `grids` | `--wavelength LOW HIGH` | Keep only this wavelength range (Å) |
| `filters`, `filters-compose` | `--atmosphere-site SITE` | One of the instrument's own atmosphere sites (default: its default site) |
| `filters`, `filters-compose` | `--airmass unstated` | `--airmass` also accepts `unstated`, for a team atmosphere whose source gives no airmass |
| `filters-build-components` | `--archive DIR` | Keep every raw download here with its URL, time and SHA-256 (`skycalc`, `team_atmospheres`) |
| `combine` | `--grids NAME ...` | Names of the local spectral grids to combine (no prompts) |
| `ml_generator` | `--trials N` | With `--auto`: number of hyperparameter search trials (default 20) |
| `ml_generator` | `--epochs N` | With `--auto`: maximum epochs of the final training (default 100) |

Changed meaning:
- `combine --non-interactive` combines the `--grids` given, else every local spectral grid except earlier combined ones (and the output grid).
- `densify` now runs. It refuses, as usage errors, the options `densify_grid` cannot honour: `--method interp`, `--method ml` without `--ml-model`, `--method blackbody` with `--ml-model`, and `--ml-gap-threshold` other than its default.
- Usage errors print the subcommand's usage and exit 2. A failed command exits 1; the interactive menu exits 1 at quit if any workflow failed. Set `SED_TOOLS_DEBUG=1` for tracebacks.

### `sed-tools filters`

Download photometric filter transmission curves interactively. The pickers merge the instruments installed locally, the NJM mirror, the original-source builders and the SVO Filter Profile Service, and show each product once, by name, in numbered columns as in 0.4.1. Picking a facility first lists its instruments with their one-line descriptions. Picking an instrument shows a short summary before the download question: its description, where the download comes from (the installed copy, or NJM, the original source and SVO in the order a download tries them), its bands, what each curve contains, its separate components, its atmosphere sites and its licence. Names use NJM's spelling where NJM has the product; the old spellings (`PLANCK/HFI`, `Gemini/GMOS-N`, `CTIO/S-PLUS`, ...) still work as input. See "NJM is optional" below for the order of sources.

```bash
# Interactive facility/instrument/filter selection
sed-tools filters
```

Enter a single facility number to browse its instruments. Enter several numbers
or ranges, such as `1,3,5-7`, to download every instrument for those facilities
directly, without instrument selection or component prompts.

**Output structure:**

```
data/filters/Generic/Johnson/
├── B.dat           # Filter transmission curve
├── V.dat
├── R.dat
└── Johnson         # Index file for MESA
```

Downloaded filter sets can be combined without rebuilding spectra or flux cubes, because a filter set is only a directory of ``*.dat`` curves plus the MESA index file whose lines list those curves. The helper below copies the selected curves into a new instrument folder and writes the matching index file automatically.

```bash
# Create data/filters/Combined/Gaia2MASS/ with a Gaia2MASS index file
sed-tools filters-combine Gaia2MASS GAIA/GAIA 2MASS/2MASS

# Pick custom MESA-facing labels and fail on duplicate band names
sed-tools filters-combine optical_ir Generic/Johnson 2MASS/2MASS \
  --facility Custom --instrument optical_ir --on-conflict error
```

The same operation is available from Python:

```python
from sed_tools.api import Filters

combined = Filters.combine("Gaia2MASS", "GAIA/GAIA", "2MASS/2MASS")
print(combined)
```

#### How filters are stored

The filter root is `<data dir>/filters/` (see `sed-tools config`; `SED_FILTER_DIR` overrides it). An installed instrument looks like this:

```
filters/<Facility>/<Instrument>/
  <band>.dat            published curves: what MESA reads
  <Instrument>          MESA index file: the curve files MESA loads, one per line
  manifest.json         what the instrument provides, with provenance
  properties.json       band properties computed by NJM (NJM downloads only)
  info.json             descriptions, references, acknowledgements, licence (NJM downloads only)
  components/<role>/    separate factors: filter/, optics/, qe/, or a composite such as filter+optics+qe/
  products/<recipe>/    composed sets; each is a MESA instrument directory with its own <recipe> index file
atmospheres/<site>/     beside filters/: the site's manifest.json and its .dat curves
cache/njm/              beside filters/: NJM's index.json, properties.json and info.json, cached
```

- **Curves.** Each `<band>.dat` starts with a column row, `Wavelength,Transmission,Key=value;...`, whose third field holds the metadata (`FilterID`, `DetectorType`, `DetectorTypeSource`, `WavelengthUnit`, `Components`, ...); then one `wavelength,transmission` sample per line. MESA discards the column row.
- **MESA index file.** Named after the instrument directory. MESA loads only the curves it lists, so files beside it that it does not list (such as NJM's superseded `<band>_svo` copies) are not loaded.
- **`manifest.json`.** `bands`; `published` and `published_bands` (what each published curve contains, e.g. `filter+qe`); `components` (role, then band or `*`, then a file under `components/`); `detector_type`, `detector_types` and `detector_type_sources`; `variants` (alternative component curves, used only when selected with `variants=` in Python); `default_site`, `atmosphere_sites` and `ground_based`; `sources`, `notes` and `outside_range`; and `archived_bands` (superseded copies kept beside the set, never part of `bands` or the MESA index). `variants`, `atmosphere_sites` and `archived_bands` are written only when set.
- **Products.** A product directory's name records what was selected: `+<role>_<variant>` for a variant, `+atm_<label>` for an atmosphere, `+detector_<type>` for a detector override (e.g. `products/filter+qe+qe_single_layer/`).
- **Atmospheres.** `atmospheres/<site>/manifest.json` records the site's kind (`team` or `derived`), observing conditions, model, model version, citation and usage terms; `Filters.atmospheres()` lists them, and the composition prompt offers an instrument's own sites with their sources.
- **`properties.json` and `info.json`** beside the curves describe exactly those curves, so SED_Tools prefers them to NJM's current files.

#### NJM is optional

Filters are taken from the first of these sources that has a complete set: an installed copy, the NJM mirror, the instrument team's original files (the original-source builders, for the instruments registered in `sed_tools/component_sources/catalog.py`), and the SVO Filter Profile Service. NJM and the builders are asked under their own spelling of a name, matched without regard to case or punctuation, so `CTIO/S-PLUS` fetches NJM's `CTIO/S_PLUS`.

Facility and instrument names, their one-line descriptions, and which SVO entries are the same product as an NJM instrument come from a catalogue shipped with the package (`sed_tools/instrument_catalogue.json`). A reachable NJM adds to it, so the pickers show the same names with or without NJM. When NJM does not answer, its requests give up after a 5 s connect timeout, one warning per session names the sources used instead (installed copies, the original-source builders and SVO), and NJM is not asked again for five minutes. The summary shown after picking an instrument then says NJM is unreachable, and `filters-info` uses the cached catalogue and says so. `sed-tools grids` warns that NJM's spectral grids are not offered and lists the other sources' grids.

Without NJM, an NJM set that SVO holds under other names (NJM's `GAIA/GAIA` is SVO's `GAIA/GAIA3`) is installed from SVO under NJM's names. If SVO has only some of its bands, those are installed and a warning names the missing ones (`GAIA/GAIA`: 4 of 6, without `Gbp_bright` and `Gbp_faint`). `filters-info` and the summary in the browser mark the set "Incomplete", and in Python `Filters.available()` gives the installed `bands` and every `expected_bands`. Fetch it again when NJM is reachable to complete it.

#### Understanding the optical path and detector response

SED_Tools now describes a passband in terms of the elements along the light's path: the site's atmosphere (for ground-based observations), telescope and instrument optics, the bandpass filter, and the detector's quantum efficiency (QE). This makes it possible to inspect what a published transmission curve already includes before choosing which components to use in synthetic photometry.

Two separate pieces of metadata describe the response:

| Metadata | What it tells you |
|---|---|
| `Components` | Which physical throughput factors are already multiplied into the curve: `filter`, `optics`, `qe`, `atmosphere`, or a source's unresolved `instrument` contribution |
| `DetectorType` | Whether photometric integration uses the energy-counting weight `T(λ)` or photon-counting weight `T(λ)·λ` |

QE describes the detector's wavelength-dependent efficiency; detector type describes how the response is weighted during integration. Tracking both prevents counting a throughput factor twice or applying the wrong detector weighting. Component availability depends on what the source publishes: a total throughput does not imply that all of its factors can be recovered separately. Unknown contents and assumed detector types remain identified as such in the metadata.

#### Photon- vs energy-counting detectors

A transmission curve is defined either for an energy-counting detector (e.g. Bessell's Johnson–Cousins curves) or for a photon-counting one (CCDs, e.g. Gaia DR3, JWST, 2MASS). Synthetic photometry weights the integrand by `T(λ)` in the first case and by `T(λ)·λ` in the second ([Bessell & Murphy 2012](https://ui.adsabs.harvard.edu/abs/2012PASP..124..140B/abstract), eq. A9); using the wrong one shifts Gaia G by ~0.035 mag.

Every filter file records which applies on its column-name row, the one line MESA discards before reading samples:

```
Wavelength,Transmission,FilterID=GAIA/GAIA3.G;DetectorType=photon;DetectorTypeSource=svo;WavelengthUnit=Angstrom
3200.0,2.37366962e-08
...
```

The metadata is a third CSV field with no whitespace (`%`, spaces, tabs, line breaks, `,`, `;`, `"` and `#` in values are percent-escaped, e.g. `Atmosphere=site=ctio_des%20airmass=1.2`). Released MESA, pandas, `csv`, `np.loadtxt(skiprows=1)` and astropy's csv reader therefore see the same `Wavelength` and `Transmission` columns and samples as before; a curve without metadata keeps the bare `Wavelength,Transmission` row. Readers that require exactly two header names, such as `np.genfromtxt(names=True)`, see the extra field.

SED_Tools 0.3.0 wrote these fields as `#` lines above the column row instead, which released MESA (25.12.1, 26.4.1) misreads as extra samples. SED_Tools still reads such files, and moves their fields onto the column row whenever it fetches a cached set, downloads, tags or combines them. The NJM mirror has been converted.

A curve with no column-name row (its first line is a sample) has no free line, so SED_Tools leaves it as it is and stores no metadata in it.

`DetectorType` is `energy` or `photon`. `DetectorTypeSource` says how it was established. From the strongest evidence to the weakest:

1. `original`: stated by the curve's publisher (paper, instrument page or data file).
2. `inferred`: not stated by the publisher, but established from documented facts, such as a filter-only curve used with a documented photon-counting detector.
3. `svo`: taken from the SVO record for `FilterID`.
4. `svo-curve-match`: the file was identified by matching its curve point-by-point against SVO.
5. `assumed`: unverified. `filters-detector` writes `energy` with this source when a file has no `DetectorType` and cannot be identified; the original-source builders write the convention set for the instrument (`photon` by default) with it.

Downloads write the header automatically; existing files can be tagged in place:

```bash
sed-tools filters-detector                 # whole filter root
sed-tools filters-detector GAIA/GAIA --dry-run
sed-tools filters-detector Generic/Bessell --force --dry-run
```

```python
Filters.tag_detector_types()               # returns one row per file
```

By default `filters-detector` resolves untagged files, retries `assumed` ones (except curves built from original sources) and completes tags that lack `WavelengthUnit` or `Components`; `--force` re-resolves every file. In both modes an existing type is replaced only by a resolution of equal or higher precedence in the list above, so SVO's label never overrides an `original` or `inferred` type. A weaker resolution is not written: the file is left as it is, and the run lists it as kept, with the SVO answer and the reason (`kept photon [original]: original outranks svo; svo gives energy (Generic/Bessell.B)`). The returned rows carry `kept`, `resolved_type` and `resolved_source`. Types set by SED_Tools' composition tools (`user` overrides, `composed` products and `component` files) are outside this order and are never replaced.

`load_filter_curve` reads the header (`FilterCurve.detector_type`, `FilterCurve.weight`) and all band averages use it; untagged files are treated as energy-counting.

`WavelengthUnit` is read the same way. Files without it are assumed to be in Angstrom unless their reddest point is below 1000 (then nm, µm or m), with a logged warning; pass `load_filter_curve(path, wavelength_unit="nm")` when a file is ambiguous.

#### Filter properties

`sed_tools.filter_properties` computes the characteristics the SVO Filter Profile Service lists for a curve:
- mean, pivot, peak, central, minimum and maximum wavelengths;
- FWHM and effective width;
- the Vega-weighted effective and photon wavelengths;
- Vega, AB and ST zero points, in f_λ and in Jy;
- the AB−Vega and ST−Vega magnitude offsets (2MASS Ks: m_AB − m_Vega ≈ 1.85);
- `Fsun`, the Sun's band-averaged f_λ at 1 AU (erg s⁻¹ cm⁻² Å⁻¹), when you supply a solar spectrum;
- `AfAv`, the extinction ratio A(λ)/A(V) at the pivot wavelength, from the [Fitzpatrick (1999)](https://ui.adsabs.harvard.edu/abs/1999PASP..111...63F) law with R_V = 3.1.

The pivot, zero points and `Fsun` use the same detector-aware weights as SED_Tools photometry, so band-averaged f_λ and f_ν convert exactly. Definitions are in the module docstring. The NJM mirror serves these values precomputed in each instrument's `properties.json`, listed in `index.json`. It uses MESA's `vega_flam.csv` for the Vega quantities, so they match MESA colors. For `Fsun` it uses CALSPEC's solar spectrum (`sun_reference_stis_002` below 9600 Å, `sun_mod_001` above), the composite the SVO Filter Profile Service uses.

`AfAv` is computed by the `F99` class of [dust_extinction](https://github.com/karllark/dust_extinction) (Gordon 2024, [JOSS 9(100), 7023](https://doi.org/10.21105/joss.07023)), an optional dependency: `pip install "sed-tools[extinction]"`. Without it, `AfAv` is `None` and a warning says why. It is also `None` where the pivot lies outside the range dust_extinction defines for the law (1000 Å to 3.33 µm); the law is never extrapolated. A(V) is the law's own normalisation (V-band averaged), so `AfAv` is 1 at 5414 Å, not at 5500 Å. SVO's `Af/AV` uses a different law, so the two differ.

```python
from sed_tools.filter_properties import instrument_properties, load_sun, load_vega

vega = load_vega("vega_flam.csv")                 # wavelength in Å, f_λ in erg/s/cm²/Å
sun = load_sun("sun_flam.csv")                    # the Sun at 1 AU, same units
props = instrument_properties("data/filters/2MASS/2MASS", vega, sun)
props["Ks"]["WavelengthPivot"], props["Ks"]["ZeroPointVega_Jy"], props["Ks"]["Fsun"], props["Ks"]["AfAv"]
```

`sed_tools.filter_colors` gives each band a colour for plotting, computed from its curve rather than guessed from its name:
- **Mostly-visible bands** (at least half the throughput between 380 and 750 nm) get their true colour: daylight (CIE D65) seen through the curve, using the CIE 1931 colour-matching functions shipped with the package.
- **Ultraviolet and infrared bands** get false colours in wavelength order. Next to visible bands they continue bluer or redder; in an all-infrared or all-ultraviolet instrument they span a rainbow shifted toward that regime.

NJM serves the colours in `properties.json` (`DisplayColor`), and its website plots each instrument's curves with them.

```python
from pathlib import Path
from sed_tools.filter_colors import instrument_colors

colors = instrument_colors(sorted(Path("data/filters/2MASS/2MASS").glob("*.dat")))
colors["J"]["DisplayColor"]                       # "#6cee2b"
```

#### Searching, describing and citing filters without SVO

NJM serves, beside the curves, a catalogue that SED_Tools reads directly, so SVO is not needed to find a band, look up its characteristics or cite it:
- `index.json` lists every facility, instrument and band;
- each instrument's `properties.json` holds the values above, computed from its curves;
- each instrument's `info.json` holds descriptions, the references to cite (with BibTeX where a service supplied it), the providers' requested acknowledgements, the licence and what SVO recorded about each band.

```bash
# Description, licence and a per-band table, after the component report
sed-tools filters-info 2MASS/2MASS
```

Searching and citing are done in Python:

```python
from sed_tools.api import Filters

bands = Filters.search(wavelength=21500, fwhm=(1000, 5000))     # pandas DataFrame, one row per band
bands[["facility", "instrument", "band", "WavelengthPivot", "FWHM", "DetectorType"]]
Filters.search(facility="HST", pivot=(5000, 6000), where={"ABminusVega": (None, 0.2)})

Filters.properties("2MASS", "2MASS")["bands"]["Ks"]["ZeroPointVega_Jy"]
Filters.info("2MASS", "2MASS")["acknowledgements"]
print(Filters.cite("2MASS/2MASS", "GAIA/GAIA/G"))             # or format="bibtex"

ax = Filters.plot("2MASS/2MASS", normalize="peak", vega=True)   # matplotlib Axes
Filters.plot(bands)                                              # a search result
```

- **Search.** Name criteria are matched in `index.json` first. `properties.json` is then fetched only for instruments that can still match, at most six requests at a time, retrying when NJM briefly blocks a burst.
  - A single `wavelength` keeps bands whose span (`WavelengthMin`–`WavelengthMax`, where T is at least 1 % of its peak) covers it; a range keeps bands overlapping it.
  - Astropy quantities are accepted. A band with a missing value, such as a Vega quantity where the Vega spectrum does not cover it, never matches a criterion on that value.
- **Caching.** Catalogue files are cached in `cache/njm/` beside the filter root (next to `atmospheres/`). `index.json` is rechecked daily; the other files are revalidated, by ETag, only when `index.json` changes. Without the mirror the cached copies are used (`filters-info --offline` / `offline=True` never contacts it; `refresh=True` asks again).
- **Downloads.** A downloaded instrument keeps `properties.json` and `info.json` beside its curves, and these copies are preferred because they describe exactly those curves. The MESA index file still lists only the curves.
- **Citations.** References are merged across instruments, one per work (bibcode form). For chosen bands, the instrument-wide references plus those bands' own are given. Acknowledgement texts are quoted as the providers ask. References for which NJM has no BibTeX appear in the `.bib` output as `%` comments with what is known; nothing is made up.
- **Plots.** Curves use `sed_tools.filter_colors`: true colours for visible bands and ordered false colours otherwise. One instrument's bands keep its own colours; a selection across instruments is coloured as one set. Instruments not yet installed are downloaded first. `normalize` is `"peak"` or `"area"`; `vega=True` overlays NJM's `vega/vega_flam.csv`, scaled to the curves.

#### Filter, optics, QE and atmosphere as separate components

A published passband is usually a product of several throughputs, and which ones varies: Gaia DR3 is filter + CCD + instrument, SDSS/2MASS/PS1 include the atmosphere, Johnson is the filter alone. Every filter file records what its published curve contains (`Components=filter+qe+instrument`). Where an instrument's team publishes the factors separately, SED_Tools stores them alongside and generates that instrument's throughput with and without each one. Components are never mixed between instruments or sites: an instrument is only ever composed from its own filter, optics and QE, and only its own site's atmosphere.

```
filters/<Facility>/<Instrument>/
  <Instrument>, <band>.dat     published curves — unchanged, what MESA reads by default
  manifest.json                what is available, with provenance
  components/<role>/*.dat      filter / optics / qe (or composites, e.g. filter+optics+qe)
  products/<recipe>/           composed sets; each is itself a MESA instrument directory
atmospheres/<site>/            telluric sites: team standards (lsst_std, ...) or derived SkyCalc grids
```

```bash
sed-tools filters-info LSST/LSST
#   Published curves contain : filter+instrument+atmosphere [svo]
#   Separate components      : filter, optics, qe
#   Can be built             : filter, filter+optics, filter+qe, filter+optics+qe
#   Atmosphere               : can be added (own site: lsst_std)
sed-tools filters-compose LSST/LSST --components filter                      # bare filters
sed-tools filters-compose LSST/LSST --components filter,optics,qe --airmass 1.2   # full system at its site
```

```python
Filters.available("LSST", "LSST")
Filters.fetch("LSST", "LSST", components=["filter"])                        # filter only
Filters.fetch("LSST", "LSST", components=["filter", "optics", "qe"],
              atmosphere={"airmass": 1.2})                                   # + its own atmosphere
```

Requests that cannot be met are refused with an explanation rather than approximated: components the team never published (e.g. Gaia's bare filters), an atmosphere from another site, adding an atmosphere to a curve that already includes one, or to a space/airborne instrument. A published curve that is the bare filter counts as the `filter` component automatically. Products containing QE are photon-counting; otherwise they inherit the instrument's detector type. A band whose manifest records a `user` detector type (Argus, whose team states its sensors are energy counting) keeps that type in every product.

An instrument's atmosphere is one at its own location, with the user choosing only airmass and PWV. Each site directory has a `manifest.json` recording its kind, observing conditions, model, model version, parameters, citation and any usage terms:

- **Team** sites (`kind: team`) hold the standard atmosphere an instrument team published, stored as published (wavelengths converted to Å) unless the entry records a `derivation`; such curves carry `ComponentsSource=derived`.
- **Derived** sites (`kind: derived`) are grids SED_Tools computes with the [ESO SkyCalc](https://www.eso.org/observing/etc/skycalc/) sky model (`lasilla`, `paranal`, `armazones`, `highanddry`), fetched at R=200000 and bin-averaged to R=20000 so unresolved lines are integrated correctly. SkyCalc models Cerro Paranal, so it is used only for ESO facilities there and at La Silla (FORS2, HAWK-I, SPHERE, ERIS, VISTA, La Silla instruments). Other observatories are offered only their team's own atmospheres.

The default is the manifest's `default_site`. Further sites at the same location are listed in the manifest's `atmosphere_sites` or registered for the instrument in `sed_tools/filters/sites.py`, and are chosen by name. Builders register a team atmosphere with `component_sources._common.SiteAtmosphere` (written by `write_site`; `SimpleSource.site`/`sites`, or `Instrument.sites` for original-source recipes); `component_sources/team_atmospheres.py` holds the registered ones.

| Site | Kind | Conditions | Model and source | Instruments |
|---|---|---|---|---|
| `lsst_std` | team | X = 1.0, 1.2 | Rubin MODTRAN, `lsst/throughputs` v1.9.2 | LSST |
| `ctio_des` | team | X = 1.2 | DES standard bandpass table | DECam |
| `maunakea_hsc` | team | X = 1.2, PWV 1.5 mm | MODTRAN (N. Yasuda), `lsst/obs_subaru` | HSC |
| `haleakala_ps1` | team | X = 1.2 | MODTRAN; Aero × Ray × Mol of Tonry et al. 2012, table 3 | PS1 |
| `paranal_vircam` | team | X = 1.0 / PWV 1 mm; X = 2.0 / PWV 3 mm | unnamed model in ESO's VIRCAM ETC curves (0.9–5.6 µm) | VISTA (beside SkyCalc `paranal`) |
| `hopkins_2mass`, `ctio_2mass` | team | airmass not stated | PLEXUS (Cohen et al. 2003; Explanatory Supplement III.1.b1, tables 15/16) | 2MASS |
| `apo_sdss` | team | X = 1.0; X = 1.3 derived (power 1.3) | DR7 imager tables, column 5 | SDSS |
| `lco_csp` | team | X = 1.2, derived | CSP filter package: one-airmass CTIO extinction ^1.2 × telluric lines | LCO/CSP |
| `kpno_bass` | team | X = 1.0, derived, in-band only | BASS `bass-g/r.txt` column 4 / column 3 | BOK/BASS |
| `maunakea_gemini` | team | X = 1.0, 1.5, 2.0 × PWV 1.0, 1.6, 3.0, 5.0 mm | Gemini ATRAN (Lord 1992), 0.9–5.6 µm | NIFS |
| `cpachon_gemini` | team | X = 1.0, 1.5, 2.0 × PWV 2.3, 4.3, 7.6, 10 mm | Gemini ATRAN (Lord 1992), 0.9–5.6 µm | GSAOI, Flamingos-2 |
| `paranal`, `lasilla` | derived | X = 1.0–3.0 × PWV 0.5–20 mm (9 × 9) | ESO SkyCalc 2.0.9 (Noll et al. 2012; Jones et al. 2013) | ESO instruments |

Some sources do not state their airmass (2MASS's PLEXUS models). Those atmospheres are kept as published and selected explicitly with `airmass=None`, or `--airmass unstated` on the command line; they are never relabelled with an assumed airmass. A site may set its own `outside_range` and `coverage_tolerance`. Most team sites use `error` with a tolerance of 0.1%: composition is refused when more than 0.1% of a band's throughput lies beyond the atmosphere's table, and the edge values are held only over fainter leak wings.

```bash
sed-tools filters-compose Paranal/VISTA --components filter,optics,qe --airmass 1.2 --pwv 2.5   # SkyCalc paranal
sed-tools filters-compose Paranal/VISTA --components filter,optics,qe \
  --atmosphere-site paranal_vircam --airmass 1.0                                            # ESO's own model
sed-tools filters-build-components team_atmospheres --sites haleakala_ps1 --archive evidence/  # curation
```

Where components come from:

| Instruments | Source of the separate factors |
|---|---|
| HST ACS (WFC/HRC/SBC), WFC3 (UVIS1/2, IR), WFPC2, NICMOS 1–3 | STScI synphot component tables (graph-table walk per obsmode); ACS/SBC's time-dependent sensitivity is pinned to the release (2021-04-25) its published curves were computed with |
| JWST NIRCam, NIRISS, MIRI imaging | STScI Pandeia reference data 2026.7 (in-flight; a 21.6 MB download without the PSF library) |
| Euclid NISP Y, J, H | ESA NISP-PHOTO-PASSBANDS V1.0: filter, telescope × NISP optics, and QE, which multiply to ESA's T_TOTAL, the published Y/J/H (SVO's earlier copies are kept on NJM as `Y_svo`, `J_svo`, `H_svo`) |
| Roman WFI | Detector QE only: the mean of the 18 flight detectors (Roman technical information repository) |
| Rubin/LSST | `lsst/throughputs` release 1.5 for `LSST/LSST` (SVO's curves) and release 1.9 for `LSST/LSSTv19` (+ Rubin's standard atmosphere `lsst_std`) |
| CFHT MegaPrime/MegaCam | CFHT filter-database scans of each band's own filter (2002, 2007 and 2014–2015 sets kept apart), primary mirror × corrector, simulated E2V QE |
| ERIS/NIX | ESO `nix-filters.zip`: AO dichroic × the band's default NIX camera (no QE or telescope curve published) |
| 2MASS | Explanatory Supplement III.1.b1 filter, mirror, window, lens and dichroic tables and NICMOS3 QE (the published RSRs also include PLEXUS atmosphere and a λ factor) |
| Subaru HSC | NAOJ/Subaru curves (+ HSC pipeline's Maunakea atmosphere `maunakea_hsc`) |
| CTIO DECam | DES standard bandpasses ÷ DES atmosphere (+ `ctio_des`); the i and z quotients exceed 1 and are kept unclipped, flagged in the manifest notes |
| VISTA/VIRCAM | ESO VIRCAM ETC curves (+ SkyCalc `paranal`, ESO's own `paranal_vircam`) |
| HAWK-I, NACO, SPHERE/IRDIS | ESO filter scans (11, 33 and 24 independently named curves; filter-only, with optics and QE still unavailable) |
| Gemini GSAOI, NIFS | Gemini filter scans (22 GSAOI bands and six NIFS temperature variants; filter-only) |
| CAHA PANIC, WHT INGRID, IRTF NSFCam | Independently sourced in-band filter scans (7 PANIC, 20 INGRID warm/cold variants, 26 historical NSFCam); filter-only |
| Liverpool IO:O | Seven filter scans plus the separately published CCD QE and cryostat-window transmission from the IO:O efficiency model; other telescope optics remain unavailable |
| ZTF | `ztf_information` filters and CCD QE |
| SLOAN/SDSS | Response without atmosphere from Gunn's 2001 `filter_curves.fits` (default `filter+optics+qe`), Doi et al. (2010) camera-column responses as variants (+ `apo_sdss`) |
| Swift/UVOT | HEASARC CALDB: current ARFs (published), `ftrans`, mirror and instrumental effective areas (÷ 659 cm²) |
| LCO CSP, CSP2 | CSP filter package: final passbands, optical scans, telescope and camera curves (+ `lco_csp`); CSP2 totals only |
| Cassini ISS NAC/WAC | PDS: system transmission (published), CCD QE, optics, derived filter pairs |
| Gemini GMOS-N/S | Detector QE by epoch (variants, no default) |
| BOK/BASS, KPNO/MzLS | Legacy Surveys curves: BASS filter+qe and qe (+ `kpno_bass`); MzLS corrected z with the filter |
| Argus/DR280 | Chroma filters (variants `production`, `addem`), DR280 optics, IMX455 QE |
| SVO instrument groups with filter-only sibling curves (UKIRT/WFCAM, OmegaCAM, Suprime-Cam, ...) | SVO's own filter-only sibling curves (`g_filter`, `g_fil`) |
| Pan-STARRS1 | not yet built: PS1's Table 3 has the open (no filter) cross-section and the atmosphere |
| Gaia, WISE, Spitzer, GALEX, TESS, Kepler, Hipparcos/Tycho | none — only totals are published (`filters-info` says so) |

The NJM mirror runs the automatic builders ahead of time (`sed-tools filters-build-components all`). Component requests reuse installed files, try NJM, then run the instrument's original-source builder if needed. For these table sets, a curator can run `sed-tools filters-build-components tabular --instrument PANIC` (or `INGRID`, `NSFCam`, `IOO`, `HAWKI`, `NACO`, `SPHERE`, `GSAOI`, `NIFS`); `--raw /path/to/evidence` reads the saved official files when a source is unavailable. These filters can be discovered and downloaded without SVO. If both NJM and the original source are unavailable, the normal SVO fallback remains. Their root curves are measured filters, not full system throughput. Builders marked as large downloads require `allow_heavy=True` in Python or `--allow-heavy` on the command line; none is marked at present (JWST's reference data is now a 21.6 MB download).

#### Inspecting availability and requesting products

The CLI and Python API use the same acquisition and composition workflow. Availability distinguishes installed component files from registered source support. Missing files are reported, and `recipes_by_band` identifies combinations available for only part of an instrument's band set. Instruments whose separate components have not been reviewed are marked `unreviewed`; they remain available through the generic SVO passband downloader.

The [component-source research inventory](https://github.com/nialljmiller/SED_Tools/blob/v0.5.0/docs/component_sources/README.md)
tracks every SVO facility/instrument entry, research ownership and the evidence
needed to add original component sources. Research findings are reviewed before
they become supported downloaders or NJM mirror data.

```bash
# Discover all SVO and NJM instruments, plus local and registered extra sources
sed-tools filters-inventory --json

# Inspect installed files without contacting any service
sed-tools filters-info LSST/LSST --offline --json

# Fetch an instrument directly and compose a subset of its bands
sed-tools filters LSST/LSST --components filter,optics,qe --bands g r

# Compare an energy-counting response in a separate derived product
sed-tools filters-compose LSST/LSST --components filter,optics,qe \
  --bands g r --detector-type energy
```

```python
inventory = Filters.inventory()  # instruments and discovery errors; no bandpass downloads
info = Filters.available("LSST", "LSST", include_remote=False)
print(info["recipes_by_band"], info["missing_files"])

product = Filters.fetch("LSST", "LSST", components="filter+optics+qe",
                        bands=["g", "r"], detector_type="energy")
```

The detector override changes the integration convention applied to the selected throughput; replacing a physical detector's QE requires its actual QE component. An override records `DetectorTypeSource: user` and `DetectorTypeOriginal` in a separate `+detector_energy` or `+detector_photon` product directory. Source curves retain their original metadata. Without an override, each band's convention is preserved, including instruments with mixed detector conventions.

Atmosphere requests use one of the instrument's own sites and exact observing conditions. Installed grids are used first, followed by NJM. During an NJM outage, team sites are rebuilt from their original files (`team_atmospheres`, or the instrument's own builder for `lsst_std` and `ctio_des`), and SkyCalc sites generate the requested grid point (through ESO's API, or its web form when the API host does not answer). SkyCalc generation requires PWV; an existing grid with several PWV values requires an explicit choice. SED_Tools does not substitute another site's atmosphere or interpolate between grid points.

Source failures remain visible in inventory and availability results. Published downloads check the expected bands recorded in manifests and index files; older installations without a catalogue declaration can only be checked against their existing contents. Product construction finishes every selected band before replacing a previous product, so a failed composition preserves the previous result.

#### Updating an existing filter collection

Run `sed-tools filters-detector --dry-run` to inspect proposed detector tags, then `sed-tools filters-detector` to write them. Review entries marked `assumed`: their type is unverified (files that had none keep the historical energy-counting convention until identified). `--force` re-resolves tagged files too, but never replaces an `original` or `inferred` type; review the kept list it prints. Use `sed-tools filters-info Facility/Instrument` to inspect published contents, available components and the instrument's atmosphere before composing a new set.

Published curves remain the default. Composed products have their own directories and MESA index files; select a product directory explicitly to use that recipe. Automatic photometry lookup skips `components/` and `products/`, so adding component data does not silently replace a published passband. The filter-set combination menu includes composed products as selectable inputs. Existing flux cubes do not need rebuilding to tag filters or compose throughputs; recompute photometry to apply corrected detector weights.

---


### `sed-tools grids`

Download model spectra from remote spectral grids.

```bash
# Interactive source and spectral grid selection
sed-tools grids

# Download specific spectral grids
sed-tools grids --grids Kurucz2003all

# Force a specific source
sed-tools grids --source svo --grids Kurucz2003all

# Parallel downloads
sed-tools grids --grids Kurucz2003all --workers 8

# Cut the download
sed-tools grids --source njm --grids Kurucz2003all --teff 3500 8000 --logg 3 5

# Give the flux unit of spectra that do not declare one
sed-tools grids --grids MyGrid --flux-unit flam
```

**What it does:**

1. Queries the selected source for available spectral grids
2. Downloads spectrum files matching your criteria
3. Standardizes units (wavelength → Å, flux → erg/cm²/s/Å). If the spectra do not declare their flux unit, it asks for it in a terminal, or stops unless `--flux-unit` gives it; spectra that declare theirs keep it
4. Generates `lookup_table.csv` with stellar parameters
5. Automatically runs `rebuild` to create binary files

**Sources:**

| Source | Description |
|--------|-------------|
| `njm` | NJM server (default, fastest) |
| `svo` | Spanish Virtual Observatory |
| `msg` | MSG grids (Townsend) |
| `mast` | MAST BOSZ library |

---

### `sed-tools rebuild`

Build MESA-compatible binary files from downloaded text spectra.

```bash
# Rebuild all local spectral grids
sed-tools rebuild

# Rebuild specific spectral grids
sed-tools rebuild --grids Kurucz2003all
```

**Generated files:**

| File | Description |
|------|-------------|
| `flux_cube.bin` | Binary flux cube (required by [SED_Model](https://github.com/nialljmiller/SED_Model) and other codes, such as MESA)|
| `lookup_table.csv` | Parameter lookup table |
| `*.h5` | HDF5 bundle with all spectra |

---

### `sed-tools combine`

Merge multiple stellar spectral grids into a unified ensemble.

```bash
# Interactive spectral grid selection
sed-tools combine

# Combine every local spectral grid except earlier combine outputs
sed-tools combine --non-interactive

# Combine named grids without prompts
sed-tools combine --grids Kurucz2003all bt-settl --output my_combined_grid
```

**Use cases:**

- Extend temperature coverage by combining hot and cool stellar spectral grids
- Fill gaps in parameter space using complementary libraries
- Create comprehensive grids for population synthesis

---

### `sed-tools ml_completer`

<p>
  <img src="https://raw.githubusercontent.com/nialljmiller/SED_Tools/v0.5.0/docs/prediction_examples.png" alt="Prediction examples" width="200"/>
</p>


Train and apply neural networks to extend incomplete SEDs to broader wavelength ranges.

**Use case:** You have spectra with limited wavelength coverage (e.g., optical-only) and need to extend them into UV or IR.

```bash
# Interactive mode
sed-tools ml_completer
```

**How it works:**

1. Trains on complete SED libraries with full wavelength coverage
2. Uses black body radiation as a physics-based baseline
3. Neural network learns corrections to the black body approximation
4. Masked training handles heterogeneous wavelength grids
5. Blends ML predictions with black body at extrapolation boundaries

---

### `sed-tools ml_generator`


<p>
  <img src="https://raw.githubusercontent.com/nialljmiller/SED_Tools/v0.5.0/docs/prediction_examples.png" alt="Prediction examples" width="200"/>
</p>

Train and apply neural networks to generate complete SEDs from stellar parameters alone.

**Use case:** You need SEDs for arbitrary stellar parameters but don't have an input spectrum — just Teff, logg, and [M/H].

```bash
# Interactive mode
sed-tools ml_generator

# Hyperparameter search, then training of the best model (needs: pip install "sed-tools[full,ml-tune]")
sed-tools ml_generator --auto --library data/stellar_models/Kurucz2003all --output models/auto \
                       --trials 20 --epochs 100
```

**How it works:**

1. Trains on flux cubes mapping (Teff, logg, [M/H]) → full SED
2. Network learns the complete spectral shape from 3 parameters
3. Log-scaling and normalization handle flux dynamic range
4. Generates diagnostic plots showing parameter space coverage

<p>
  <img src="https://raw.githubusercontent.com/nialljmiller/SED_Tools/v0.5.0/docs/sed_T6969_g4.20_m%2B0.02_params.png" alt="Prediction examples" width="200"/>
</p>

<p>
  <img src="https://raw.githubusercontent.com/nialljmiller/SED_Tools/v0.5.0/docs/sed_T6969_g4.20_m%2B0.02_spectrum.png" alt="Prediction examples" width="200"/>
</p>

---

### `sed-tools densify`

Fill coarse Teff gaps in an existing flux cube to eliminate stair-stepping artefacts in MESA photometric outputs. New Teff nodes are filled with ML predictions (when a generator model is provided and the point is within its training range) or a Planck blackbody scaled to the nearest real SED's bolometric flux. The densifier reads the source cube through `np.memmap` and builds the dense cube one block of wavelengths at a time, with blocks sized to half the available RAM, so cubes larger than memory are handled.

```bash
# Densify with blackbody fallback (no ML model)
sed-tools densify \
  --flux-cube data/stellar_models/tmap2/flux_cube.bin \
  --output data/stellar_models/tmap2_dense/flux_cube.bin \
  --teff-spacing 1000

# Densify with ML generator for in-range points, blackbody elsewhere
sed-tools densify \
  --flux-cube data/stellar_models/tmap2/flux_cube.bin \
  --output data/stellar_models/tmap2_dense/flux_cube.bin \
  --teff-spacing 1000 \
  --ml-model data/stellar_models/models/sed_generator_Kurucz2003all
```

See [notebook 10](https://github.com/nialljmiller/SED_Tools/blob/v0.5.0/jupyter_notebooks/10_grid_densification.ipynb) for a worked example including before/after comparison plots.

---

### `sed-tools import`

Ingest an external directory of `.txt` spectra (with parameter headers) into the SED_Tools pipeline. Files are copied or moved into the configured spectral grid directory, then processed through the standard clean → lookup → HDF5 → flux-cube pipeline.

```bash
# Import spectra, build all outputs
sed-tools import --path /path/to/my_spectra/ --name MyGrid

# Preview parseable headers without importing
sed-tools import --path /path/to/my_spectra/ --name MyGrid --dry-run

# Move files instead of copying; skip HDF5 bundle
sed-tools import --path /path/to/my_spectra/ --name MyGrid --move --no-h5

# Give the flux unit of spectra whose headers do not declare one (--dry-run reports whether they do)
sed-tools import --path /path/to/my_spectra/ --name MyGrid --flux-unit flam
```

---

### `sed-tools coverage`

Report parameter-space coverage statistics for one or more installed grids. Prints per-axis ranges, unique grid values, fill fraction, and optionally writes a Teff–logg + 3D scatter plot.

```bash
# Report and plot coverage for a single spectral grid
sed-tools coverage --grids Kurucz2003all

# Report without generating a plot
sed-tools coverage --grids Kurucz2003all --no-plot
```

---

### `sed-tools mesa_prepare`

Export one extra-axis variant (e.g. one alpha-enhancement, microturbulence or mixing-length combination) of a spectral grid as a clean, self-contained MESA-ready folder holding only that variant's flux cube and lookup table. `sed-tools rebuild` splits a grid whose extra axes vary into one subgrid per variant (`<grid>/<variant path>/`, each with its own `flux_cube.bin` and `lookup_table.csv`) and lists them in `<grid>/variants_index.csv`; `mesa_prepare` reads that file first. A `fluxcube_library/` written by earlier SED_Tools versions is still read when present. A uniform grid with its `flux_cube.bin` is already MESA-ready, and the command says so. The provenance columns of combined and ML-extended grids are not variant axes.

```bash
# Interactive spectral grid and variant selection
sed-tools mesa_prepare
```

---

### `sed-tools config`

Show or change the root data directory where SED_Tools stores spectral grids and filters.

```bash
# Show current data directory
sed-tools config

# Change the data directory (and optionally move existing data)
sed-tools config --set /new/data/path --move
```

---

## Python API

The Python API exposes the core CLI capabilities together with additional building blocks for data pipelines.

### `SED` — Main Entry Point

#### Discovery

```python
from sed_tools.api import SED

# List all available spectral grids
grids = SED.query()

# Filter by source
grids = SED.query(source='svo')

# Filter by parameter coverage
grids = SED.query(
    teff_min=5000,
    teff_max=7000,
    logg_min=3.5,
    metallicity_min=-1.0,
)

# Local spectral grids only
grids = SED.query(include_remote=False)
```

#### Downloading

```python
# Basic fetch (tries NJM mirror first, falls back to other sources)
sed = SED.fetch('Kurucz2003all')

# Force specific source
sed = SED.fetch('Kurucz2003all', source='svo')

# Download only part of the grid
sed = SED.fetch(
    'Kurucz2003all',
    teff_min=4000,
    teff_max=8000,
    logg_min=3.0,
    logg_max=5.0,
    metallicity_min=-1.0,
    metallicity_max=0.5,
    workers=8,
)

# Write the returned spectra, with a lookup table, flux cube and HDF5 bundle of their own, to another folder
sed.grid.write('/custom/output/path')
```

**With `teff_min` … `metallicity_max`, only the spectra within them are downloaded**, as `sed-tools grids --teff … --logg … --meta …` does. The grid's folder (`Kurucz2003all` under the stellar model directory) then holds those spectra, their `lookup_table.csv`, HDF5 bundle and `flux_cube.bin`, so `SED.local('Kurucz2003all')` and MESA use that selection. NJM's flux cube and bundle of the whole grid are not downloaded. MSG downloads its grid's single HDF5 file whole and extracts only the spectra within the bounds. SVO cannot select spectra, so from SVO the whole grid is downloaded, with a warning, then cut.

- A folder that already holds every spectrum within the bounds, with a flux cube holding every spectrum there (the whole grid, or a wider selection), is cut as it stands: nothing is downloaded or rewritten. Whether the cube holds them is read from the cube and the lookup table, not from file times, so a grid copied without its times (`cp -R`, `scp`) counts as complete.
- A later fetch without bounds, or with wider ones, downloads the rest and rebuilds the lookup table, bundle and flux cube from every spectrum in the folder.
- Separate selections fetched into one folder (say Teff ≤ 5800 with log g ≤ 4.2, then Teff ≥ 6200 with log g ≥ 4.3) leave points of the flux cube's Teff × log g × [M/H] grid without a spectrum, whose flux is zero and pulls interpolation near them toward zero.
- A whole grid from NJM comes with NJM's flux cube. For a grid from SVO, or a whole grid from MSG, build it with `sed-tools rebuild --grids NAME`.
- `sed.grid.write()` without a path writes into the grid's own folder, replacing its spectra with copies whose header keeps only Teff, log g, [M/H] and the units; give it another folder.

#### Loading Local Data

```python
# Load an installed spectral grid
sed = SED.local('Kurucz2003all')

# Check parameter coverage
ranges = sed.parameter_ranges()
# {'teff': (3500.0, 50000.0), 'logg': (0.0, 5.0), 'metallicity': (-5.0, 1.0)}
```

#### Interpolation

```python
sed = SED.local('Kurucz2003all')

# Interpolate a spectrum at specific stellar parameters
spectrum = sed(teff=5777, logg=4.44, metallicity=0.0)

print(spectrum.wavelength)  # Array in Angstroms
print(spectrum.flux)        # Array in erg/cm²/s/Å
```

#### Synthetic Photometry

<!-- SED_TOOLS_FILTER_SET_PHOTOMETRY_DOC -->

`EvaluatedSED.photometry(...)` accepts individual filter names, filter files, or a filter-set / instrument directory name. For example, with the standard MESA colors-data layout, `"GAIA"` expands to the files in `filters/GAIA/GAIA/`.

```python
import os
from pathlib import Path
from sed_tools.api import SED

colors_data = Path(os.path.expandvars("$MESA_DIR")) / "data/colors_data"

sed = SED.local(
    "Kurucz2003all",
    model_root=colors_data / "stellar_models",
    filter_root=colors_data / "filters",
)

spec = sed(teff=6000, logg=2.0, metallicity=-1.0)
phot = spec.photometry("GAIA", system="AB")

mags = {res.filter_name: res.magnitude for res in phot.values()}
bp_rp = mags["Gbp"] - mags["Grp"]
print(bp_rp)
```

If a filter specification is ambiguous, pass a specific file path or a specific filter directory.

#### Combining Grids

```python
ensemble = SED.combine(
    ['Kurucz2003all', 'NextGen'],
    output='my_combined_grid',
)
```

#### ML Completion

```python
completer = SED.ml_completer()

# Train on a complete grid
completer.train(grid='combined_grid', epochs=200)

# Extend an incomplete spectral grid
extended = completer.extend(
    'sparse_model',
    wavelength_range=(100, 100000),
)
extended.write()
```

#### ML Generation

```python
generator = SED.ml_generator()

# Train on a stellar spectral grid
generator.train(grid='Kurucz2003all', epochs=200)

# Generate a single SED
wl, flux = generator.generate(teff=5777, logg=4.44, metallicity=0.0)

# Generate with diagnostic plots
wl, flux = generator.generate_with_outputs(
    teff=5777, 
    logg=4.44, 
    metallicity=0.0,
    output_dir='output/sun_sed',
)

# Or load a pre-trained model
generator = SED.ml_generator()
generator.load('sed_generator_Kurucz2003all')
wl, flux = generator.generate(teff=6000, logg=4.0, metallicity=-0.5)

# Check parameter ranges
ranges = generator.parameter_ranges()
# {'teff': (3500.0, 50000.0), 'logg': (0.0, 5.0), 'metallicity': (-5.0, 1.0)}
```

#### Grid Coverage

```python
# Report parameter-space coverage and write a diagnostic plot
report = SED.coverage('Kurucz2003all')
# Keys: n_spectra, n_distinct_nodes, fill_fraction, axes (per-axis min/max/unique)

# Skip the plot, or write it to a custom path
report = SED.coverage('Kurucz2003all', plot=False)
report = SED.coverage('Kurucz2003all', out_path='coverage.png')
```

#### Importing External Grids

```python
# Ingest a directory of .txt spectra into the pipeline
result = SED.import_grid('/path/to/my_spectra/', name='MyGrid')

# Preview parseable headers without actually importing
result = SED.import_grid('/path/to/my_spectra/', name='MyGrid', dry_run=True)

# Move files instead of copying; skip HDF5
result = SED.import_grid('/path/to/my_spectra/', name='MyGrid', move=True, build_h5=False)
```

---

### `SpectralGrid` — Model Spectra and Parameters

```python
grid = sed.grid

# Properties
len(grid)              # Number of spectra
grid.teff_grid         # Unique Teff values
grid.logg_grid         # Unique logg values
grid.parameters        # DataFrame of all parameters

# Iteration
for spec in grid:
    print(spec.teff, spec.logg, spec.metallicity)

# Filtering
cool_stars = grid.filter(teff_max=5000)

# Persistence
grid.write()
grid.write('/custom/path')
```

---

### `Spectrum` — Individual Model Spectrum

```python
spec = grid[0]

# Data arrays
spec.wavelength    # np.ndarray (Angstroms)
spec.flux          # np.ndarray (erg/cm²/s/Å)
spec.wl            # Alias for wavelength
spec.fl            # Alias for flux

# Stellar parameters
spec.teff          # Effective temperature (K)
spec.logg          # Surface gravity (log g)
spec.metallicity   # [M/H]

# Metadata
spec.filename      # Original filename

# Save individual spectrum
spec.save('/path/to/spectrum.txt')
```

---

### `Filters` — Photometric Filters

```python
from sed_tools.api import Filters

# Query available filters
all_filters = Filters.query()
hst_filters = Filters.query(facility='HST')

# Download a filter set
path = Filters.fetch(facility='Generic', instrument='Johnson')

# NJM's catalogue: search, properties, descriptions, citations and plots
bands = Filters.search(wavelength=5500, fwhm=(500, 1500))
props = Filters.properties('Generic', 'Johnson')
info = Filters.info('Generic', 'Johnson')
print(Filters.cite('Generic/Johnson'))
Filters.plot('Generic/Johnson', normalize='peak', vega=True)
```

---

### `SpectralGridInfo` — Spectral Grid Metadata

```python
info = SED.query()[0]

info.name              # 'Kurucz2003all'
info.source            # 'svo', 'njm', 'local', etc.
info.teff_range        # (3500.0, 50000.0)
info.logg_range        # (0.0, 5.0)
info.metallicity_range # (-5.0, 1.0)
info.n_spectra         # Number of spectra
info.is_local          # True if already installed

# Coverage checks
info.covers(teff=5777, logg=4.44)
info.covers_range(teff_min=5000, teff_max=6000)
```

---

### CLI to API Mapping

| CLI Command | Python API Equivalent |
|-------------|----------------------|
| `sed-tools filters` | `Filters.fetch(...)` |
| `sed-tools filters-combine X A B` | `Filters.combine('X', 'A', 'B')` |
| `sed-tools filters-info F/I` | `Filters.available('F', 'I')`, `Filters.info('F', 'I')`, `Filters.properties('F', 'I')` |
| `sed-tools grids` (list) | `SED.query()` |
| `sed-tools grids --grids X` | `SED.fetch('X')` |
| `sed-tools rebuild --grids X` | `sed.grid.write()` |
| `sed-tools combine` | `SED.combine(['A', 'B'], output='...')` |
| `sed-tools densify --flux-cube F --output O` | `sed_tools.grid_densifier.densify_grid(src, dst)` |
| `sed-tools import --path P --name N` | `SED.import_grid('/path/', name='N')` |
| `sed-tools coverage --grids X` | `SED.coverage('X')` |
| `sed-tools ml_completer train` | `SED.ml_completer().train(...)` |
| `sed-tools ml_generator train` | `SED.ml_generator().train(...)` |
| `sed-tools ml_generator generate` | `SED.ml_generator().generate(...)` |
| `sed-tools grids --grids X --flux-unit flam` | `SED.fetch('X', flux_unit='flam')` |

---

## MESA Integration

### Installing Downloaded Data

Copy or symlink the generated data into your MESA installation:

```bash
# Copy
cp -r data/stellar_models/Kurucz2003all $MESA_DIR/data/colors_data/stellar_models/

# Or symlink (recommended for development)
ln -s $(pwd)/data/stellar_models/Kurucz2003all $MESA_DIR/data/colors_data/stellar_models/
```

### MESA Inlist Configuration

```fortran
&controls
    ! Stellar spectral grid
    stellar_atm = '/data/colors_data/stellar_models/Kurucz2003all/'
    
    ! Photometric filter set
    instrument = '/data/colors_data/filters/Generic/Johnson'
/
```

### Filter Specifications

When referencing filters in MESA, use the filename stem only:

- File: `data/filters/GAIA/GAIA/G.dat`
- Reference: `"G"`

---

## Directory Structure

```
SED_Tools/
├── sed_tools/              # Package source
│   ├── __init__.py
│   ├── api.py              # Python API
│   ├── cli.py              # CLI entry point
│   └── ...
├── data/                   # Downloaded data (created at runtime)
│   ├── stellar_models/
│   │   └── Kurucz2003all/
│   │       ├── flux_cube.bin
│   │       ├── lookup_table.csv
│   │       ├── spectra.h5
│   │       └── *.txt
│   └── filters/
│       └── Generic/
│           └── Johnson/
│               ├── B.dat
│               ├── V.dat
│               └── Johnson
├── docs/
├── tests/
├── pyproject.toml
└── README.md
```

---

## Data Sources

| Source | URL | Description |
|--------|-----|-------------|
| NJM Server | [data.nialljmiller.com/sed_tools/](https://data.nialljmiller.com/sed_tools/) | Pre-processed data host (fastest) |
| SVO | [svo2.cab.inta-csic.es](http://svo2.cab.inta-csic.es/theory/fps/) | Spanish Virtual Observatory |
| MSG | [astro.wisc.edu/~townsend](http://user.astro.wisc.edu/~townsend/static.php?ref=msg-grids) | MSG Stellar spectral grids |
| MAST | [archive.stsci.edu/prepds/bosz](https://archive.stsci.edu/prepds/bosz/) | BOSZ Spectral Library, including BOSZ 2024 fixed-resolution and original-resolution grids |

---

#### MAST BOSZ 2024 wavelength grids

<!-- SED_TOOLS_BOSZ_WAVEGRID_DOC -->

The BOSZ 2024 fixed-resolution products (`BOSZ-2024-r500` through `BOSZ-2024-r50000`) are resampled spectra. SED_Tools uses the official STScI wavelength grids distributed with the archive under `bosz2024/wavelength_grids/`; it does not synthesize or guess wavelength axes from row counts. The original-resolution product (`BOSZ-2024-rorig`) carries its wavelength column inline.


## Examples

### Batch Processing a Star Catalog

```python
from sed_tools.api import SED

sed = SED.local('Kurucz2003all')

stars = [
    {'name': 'Sun',         'teff': 5777, 'logg': 4.44, 'met':  0.0},
    {'name': 'Vega',        'teff': 9940, 'logg': 4.30, 'met':  0.0},
    {'name': 'Proxima Cen', 'teff': 3050, 'logg': 5.20, 'met': -0.1},
]

for star in stars:
    spectrum = sed(star['teff'], star['logg'], star['met'])
    spectrum.save(f"output/{star['name']}.txt")
```

### Building a Custom Temperature Grid

```python
from sed_tools.api import SED

models = '/path/to/stellar_models'

# Hot stars from Kurucz, cool stars from BT-Settl. From NJM or MSG only these Teff
# ranges are downloaded; a grid that comes from SVO is downloaded whole, then cut
# (see "Downloading"). Each selection is then written to a grid folder of its own.
hot = SED.fetch('Kurucz2003all', teff_min=7000, teff_max=50000, model_root=models)
hot.grid.write(f'{models}/Kurucz_hot')

cool = SED.fetch('bt-settl', teff_min=2500, teff_max=7000, model_root=models)
cool.grid.write(f'{models}/BT-Settl_cool')

# Combine the two selections into a unified grid
combined = SED.combine(
    ['Kurucz_hot', 'BT-Settl_cool'],
    output='full_temperature_grid',
    model_root=models,
)
```

### Extending UV Coverage with ML

```python
from sed_tools.api import SED

# Train on a grid with complete wavelength coverage
completer = SED.ml_completer()
completer.train('BOSZ', epochs=200)

# Extend a spectral grid with limited UV coverage
extended = completer.extend(
    'optical_only_model',
    wavelength_range=(912, 100000),  # Extend into UV
)
extended.write()
```

### Generating SEDs for Arbitrary Parameters

```python
from sed_tools.api import SED

# Train a generator on a comprehensive grid
generator = SED.ml_generator()
generator.train('Kurucz2003all', epochs=200)

# Generate SEDs for a list of stars
stars = [
    {'name': 'Sun',    'teff': 5777, 'logg': 4.44, 'met':  0.0},
    {'name': 'Vega',   'teff': 9940, 'logg': 4.30, 'met':  0.0},
    {'name': 'Sirius', 'teff': 9940, 'logg': 4.30, 'met':  0.5},
]

for star in stars:
    wl, flux = generator.generate(
        teff=star['teff'],
        logg=star['logg'],
        metallicity=star['met'],
    )
    # Save or process the SED
    import numpy as np
    np.savetxt(f"output/{star['name']}.txt", np.column_stack([wl, flux]))
```

---

## Troubleshooting

### Common Issues

**Downloads fail or timeout**

```bash
# Use the NJM mirror (faster, more reliable)
sed-tools grids --source njm --grids Kurucz2003all

# Or reduce parallel workers
sed-tools grids --grids Kurucz2003all --workers 2
```

**SVO Filter Profile Service unreachable**

SVO filter requests give up after a 10 s connect timeout. Use the NJM mirror
or the original-source builders (`sed-tools filters-build-components`), or
point SED_Tools at another FPS endpoint:

```bash
export SED_SVO_FPS_URL=https://svo2.cab.inta-csic.es/theory/fps/fps.php
```

**Missing PyTorch for ML tools**

```bash
pip install torch
```

**MESA cannot find flux cube**

Verify the directory structure matches MESA expectations:

```
$MESA_DIR/data/colors_data/stellar_models/Kurucz2003all/
├── flux_cube.bin       # Must exist
└── lookup_table.csv    # Must exist
```


**Filter set cannot be resolved**

<!-- SED_TOOLS_FILTER_SET_TROUBLESHOOTING -->

For installed filter sets, pass either an individual filter stem (`"Gbp"`), a full file path, or a filter-set name/directory such as `"GAIA"` when the data are arranged as:

```
filters/GAIA/GAIA/G.dat
filters/GAIA/GAIA/Gbp.dat
filters/GAIA/GAIA/Grp.dat
```

If a short filter name matches more than one file, use a full path or a specific filter directory.

**MAST BOSZ fixed-resolution download finds no wavelength grid**

BOSZ 2024 fixed-resolution spectra require the official wavelength files in the archive's `wavelength_grids/` directory. If those files cannot be reached, SED_Tools should fail clearly rather than invent a wavelength axis. You can also try `BOSZ-2024-rorig`, whose files include wavelength, flux, and continuum columns directly.

---

## Citation

If you use SED_Tools in your research, please cite the archived software release
via its Zenodo DOI:

All versions (resolves to the latest release): [10.5281/zenodo.18321943](https://doi.org/10.5281/zenodo.18321943)

Version 0.4.1: [10.5281/zenodo.23634323](https://doi.org/10.5281/zenodo.23634323)

Machine-readable citation metadata is available in [`CITATION.cff`](https://github.com/nialljmiller/SED_Tools/blob/v0.5.0/CITATION.cff).

SED_Tools is also the designated data-preparation pipeline for the
[MESA](https://docs.mesastar.org/) `colors` module — see
[`colors/README.rst`](https://github.com/MESAHub/mesa/blob/main/colors/README.rst)
in the MESA repository.

---

## Contributing

Bug reports, feature requests, and pull requests are welcome — see [CONTRIBUTING.md](https://github.com/nialljmiller/SED_Tools/blob/v0.5.0/CONTRIBUTING.md). Please also review our [Code of Conduct](https://github.com/nialljmiller/SED_Tools/blob/v0.5.0/CODE_OF_CONDUCT.md) and [Security Policy](https://github.com/nialljmiller/SED_Tools/blob/v0.5.0/SECURITY.md) (for reporting vulnerabilities).

---

## License

[MIT License](https://github.com/nialljmiller/SED_Tools/blob/v0.5.0/LICENSE)

---

## Acknowledgments

- [MESA](https://docs.mesastar.org/) — Modules for Experiments in Stellar Astrophysics
- [SVO Filter Profile Service](http://svo2.cab.inta-csic.es/theory/fps/) — Filter transmission curves
- [MAST](https://archive.stsci.edu/) — Mikulski Archive for Space Telescopes
- [MSG Grids](http://user.astro.wisc.edu/~townsend/static.php?ref=msg-grids) — Rich Townsend's stellar spectral grids

## Community Contributions

Thanks to everyone who has filed issues, suggested features, or reported bugs:

- [@andysantarelli](https://github.com/andysantarelli) — feature suggestions (grid coverage view, custom spectral grids)
- [@tianzhijia](https://github.com/tianzhijia) — bug report (BOSZ-2024 download issue)
- [@rhdtownsend](https://github.com/rhdtownsend) — issue report (Jupyter notebook results)
