Metadata-Version: 2.4
Name: cosmic-metrics
Version: 0.1.2
Summary: Per-block quality metrics of NenuFAR Cosmic Dawn data, for block selection and combination weights
Author: Eric Le Saux
Author-email: eric.ar.saos@gmail.com
Requires-Python: >=3.10
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Provides-Extra: docs
Provides-Extra: iono
Provides-Extra: report
Provides-Extra: test
Requires-Dist: aoquality (>=0.6)
Requires-Dist: astropy
Requires-Dist: bokeh (>=3.8) ; extra == "report"
Requires-Dist: click (>=8.0)
Requires-Dist: h5py
Requires-Dist: libpipe (>=0.9)
Requires-Dist: matplotlib
Requires-Dist: myst-parser (>=3.0) ; extra == "docs"
Requires-Dist: numpy
Requires-Dist: pandas
Requires-Dist: ps_eor (>=1.0)
Requires-Dist: pspipe (>=0.14)
Requires-Dist: pydata-sphinx-theme (>=0.16) ; extra == "docs"
Requires-Dist: pytest ; extra == "test"
Requires-Dist: ruff ; extra == "test"
Requires-Dist: sphinx (>=7.0) ; extra == "docs"
Requires-Dist: spinifex (>=2.0) ; extra == "iono"
Requires-Dist: threadpoolctl (>=3.1)
Requires-Dist: tomli ; python_version < "3.11"
Project-URL: Documentation, https://cosmic-metrics.readthedocs.io
Project-URL: Homepage, https://gitlab.com/nenufar-cosmic-dawn/cosmic-metrics
Project-URL: Issues, https://gitlab.com/nenufar-cosmic-dawn/cosmic-metrics/-/issues
Project-URL: Repository, https://gitlab.com/nenufar-cosmic-dawn/cosmic-metrics
Description-Content-Type: text/markdown

# cosmic-metrics

Per-observation quality metrics for NenuFAR Cosmic Dawn data. Read QUALITY
statistics, calibration solutions, power spectra, horizon-filtered variance and
optional GNSS ionosphere maps; inspect the resulting table, reject poor blocks
and compute pspipe weights.

**Documentation:** [Read the Docs](https://cosmic-metrics.readthedocs.io/en/latest/)

![Cosmic Metrics object and data flow](https://gitlab.com/nenufar-cosmic-dawn/cosmic-metrics/-/raw/main/docs/_static/workflow.svg)

[How the objects work together](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/overview.html)
explains the diagram.

## Install

Requires Python 3.10+ and the NenuFAR processing stack (`aoquality`, `ps_eor`,
`pspipe`), installed with it from PyPI:

```console
python -m pip install cosmic-metrics
python -m pip install 'cosmic-metrics[report]'  # Optional interactive HTML report
python -m pip install 'cosmic-metrics[iono]'  # Optional spinifex map reader
```

From a checkout, use `python -m pip install -e .` with the same extras.

## Python workflow

A `Dataset` contains observation blocks, named source readers and a managed raw
cache. Metric objects define calculations. `compute_metrics()` returns an
independent `MetricTable` containing scalars and observation metadata.

```python
import cosmic_metrics as cm
from cosmic_metrics import metrics as m

dataset = cm.Dataset.from_toml('metrics.toml')
noise = m.RegionPower('dt_V', **m.REG_B)
phase = m.PhaseChange(120, source='solutions_l2a')
table = dataset.compute_metrics([noise, phase], n_workers=1)
print(table.df)
print(table.missing_reasons)

# Example threshold only; choose science cuts for your dataset.
selection = table.select_blocks([
    cm.Rule(phase.name, '>', 3, scale='mad'),
], missing='reject')
weights = table.compute_weights(noise.name, selection=selection)
table.save('metrics.csv')
selection.save('selection.json', weights=weights)

print(dataset.cache.inspect())
preview = dataset.cache.clear(stale_only=True, dry_run=True)
```

Adapt [examples/nt04_l3.toml](https://gitlab.com/nenufar-cosmic-dawn/cosmic-metrics/-/blob/main/examples/nt04_l3.toml) to your input paths and save it
as `metrics.toml`. QUALITY and calibration templates are resolved per block;
source names choose which reader a metric uses. All block times are UTC MJD
seconds. Cache validity uses exact time bounds and dependency signatures.
For scripts with `n_workers > 1`, use a main guard.

Prepare the expensive raw arrays using only the dataset TOML, without defining
metrics:

```console
cosmic-metrics cache populate metrics.toml --dry-run
cosmic-metrics cache populate metrics.toml
cosmic-metrics cache status metrics.toml
```

The example uses Slurm/libpipe; set `[worker].nodes="localhost"` for local
population, or use SSH host names/ranges. The CLI accepts the same host and worker
overrides as nenucal, for example `--hosts localhost --max-concurrent 4`. It runs
in the foreground; use a persistent terminal session for unattended work.
Use `--slurm-nodes 4` to override the Slurm allocation size for one run while
retaining the account, partition and other Slurm options from the TOML.
Valid cache entries are skipped; use `--force` to rebuild them explicitly.
`[cache].require=true` makes later notebook computations raise on
missing/stale PS or variance entries. QUALITY and calibration remain uncached.
Run population before the Python example when using that configuration. See
[cache preparation](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/cache_population.html)
for worker settings, progress logs and resuming interrupted jobs.

## Command-line workflow

The dataset TOML can also define the metrics (`[[metrics]]`), selection stages
(`[[selection]]`) and an output directory, so that no Python is needed:

```console
cosmic-metrics compute metrics.toml --dry-run   # list blocks and metric columns
cosmic-metrics compute metrics.toml             # metrics.csv
cosmic-metrics select metrics.toml              # selection.json
cosmic-metrics report metrics.toml              # interactive report.html
```

The report shows every metric against LST or date, with linked selection across
metric panels, the profile of a selected block in all metrics and a night × LST
grid. See the
[command-line guide](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/command_line.html).

For RFI checks in a specific frequency band, configure a `variance` source and
compute `m.BandVariance((67, 68))` or `m.BandVarianceRatio((67, 68))`. These reduce
the horizon-filtered channel variance around 67–68 MHz; the ratio uses `dt_V`
as its noise reference. See the
[band-variance guide](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/metrics.html#rfi-diagnostics-in-a-frequency-band)
for units, filtering and interpretation.

## Documentation

Start with the
[complete synthetic example](https://cosmic-metrics.readthedocs.io/en/latest/getting_started.html),
which needs no input data. The guide and generated API reference cover the current
implementation:

- [Dataset configuration, blocks and paths](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/datasets.html)
- [Metric definitions, array shapes and physical units](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/metrics.html)
- [Selection, persistence and pspipe weights](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/selection_weights.html)
- [Cache inspection, invalidation and cleanup](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/cache.html)
- [Cache preparation locally and on Slurm](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/cache_population.html)
- [Command-line analysis and report](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/command_line.html)
- [Custom readers and metrics](https://cosmic-metrics.readthedocs.io/en/latest/user_guide/extending.html)
- [Python API reference](https://cosmic-metrics.readthedocs.io/en/latest/api/index.html)
- [Development and documentation builds](https://cosmic-metrics.readthedocs.io/en/latest/development.html)

The earlier metrics prototype is retained on the `eric-prototype` branch.

## Validate and build docs

```console
python -m pip install -e '.[test,docs,report]'
python -m pytest
python -m ruff check src tests docs/conf.py
python -m sphinx -W --keep-going -b html docs docs/_build/html
```

