Metadata-Version: 2.4
Name: scontrol
Version: 0.1.0
Summary: Machine learning toolkit for hyperspectral image segmentation
Author: Daniel Pérez-Rodríguez
License: MIT
Keywords: machine-learning,deep-learning,hyperspectral,segmentation,pytorch
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE.txt
Requires-Dist: torch>=2.0.0
Requires-Dist: torchvision>=0.15.0
Requires-Dist: numpy>=1.24.0
Requires-Dist: spectral-datamaker==0.6.3
Requires-Dist: spectral>=0.23.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: torchmetrics>=0.9.0
Requires-Dist: typer>=0.9.0
Requires-Dist: rich>=13.0.0
Provides-Extra: dev
Requires-Dist: pytest>=7.4.0; extra == "dev"
Requires-Dist: pytest-cov>=4.1.0; extra == "dev"
Requires-Dist: black>=23.0.0; extra == "dev"
Requires-Dist: flake8>=6.0.0; extra == "dev"
Requires-Dist: isort>=5.12.0; extra == "dev"
Requires-Dist: mypy>=1.0.0; extra == "dev"
Dynamic: license-file

# Smart Control
<p align="center">
  <img src="https://img.shields.io/badge/python-3.10+-blue" alt="Python 3.10+">
  <img src="https://img.shields.io/badge/pytorch-2.0+-orange" alt="PyTorch 2.0+">
  <img src="https://img.shields.io/badge/license-MIT-green" alt="MIT License">
</p>

Machine learning toolkit for hyperspectral image segmentation.


## Table of Contents

- [Smart Control](#smart-control)
  - [Table of Contents](#table-of-contents)
  - [Installation](#installation)
  - [Quick Start](#quick-start)
    - [1. Prepare a dataset](#1-prepare-a-dataset)
    - [2. Generate configuration templates](#2-generate-configuration-templates)
    - [3. Train a model](#3-train-a-model)
    - [4. Evaluate the trained model](#4-evaluate-the-trained-model)
  - [CLI Reference](#cli-reference)
    - [`smart-control train`](#smart-control-train)
    - [`smart-control evaluate`](#smart-control-evaluate)
    - [`smart-control template`](#smart-control-template)
    - [`smart-control info`](#smart-control-info)
  - [Configuration Files](#configuration-files)
    - [`augmentation.yaml`](#augmentationyaml)
    - [`model.yaml`](#modelyaml)
    - [`training.yaml`](#trainingyaml)
    - [`evaluation.yaml`](#evaluationyaml)
  - [Dataset Format](#dataset-format)
    - [Required structure](#required-structure)
    - [Creating a dataset with SpectralDatamaker](#creating-a-dataset-with-spectraldatamaker)
  - [Extending the Library](#extending-the-library)
  - [License](#license)

## Installation

```bash
pip install scontrol
```

## Quick Start

### 1. Prepare a dataset

Smart Control uses datasets created with [SpectralDatamaker v0.6.3](https://pypi.org/project/spectral-datamaker), which produces a directory
containing hyperspectral images, segmentation masks, and a `splits.csv` file.

A minimal dataset directory looks like:

```
my-dataset/
  ├── splits.csv          # Maps each image to "dev" or "test"
  ├── metadata.json       # Dataset-level metadata
  ├── images/
  │   ├── scene_01.hdr    # ENVI header
  │   ├── scene_01.img    # ENVI data
  │   └── scene_02.npy    # Alternative: NumPy format
  └── masks/
      ├── mask_01.npy
      └── mask_02.npy
```

### 2. Generate configuration templates

Create the configuration files you'll need. These define your model, training
hyperparameters, data augmentation, and evaluation settings.

```bash
smart-control template --output-dir ./config/
```

Four well-commented YAML files are created in `./config/`.  Edit them to match
your dataset and model requirements.

### 3. Train a model

```bash
smart-control train \
  --dataset ./my-dataset \
  --config-dir ./config \
  --output-dir ./output
```

The best checkpoint is saved to `./output/checkpoints/best.pt`.  A progress bar
shows live loss and mIoU/Dice values during training.

### 4. Evaluate the trained model

```bash
smart-control evaluate \
  --dataset ./my-dataset \
  --checkpoint ./output/checkpoints/best.pt \
  --config-dir ./config \
  --output-dir ./output/evaluation
```

Metrics, visualizations, and predictions are written to the output directory.

## CLI Reference

All commands use a common pattern for configuration files: you can point to a
directory with `--config-dir` (looks for `augmentation.yaml`, `model.yaml`,
`training.yaml`, and `evaluation.yaml` by name) or pass individual files with
`--augmentation-config`, `--model-config`, etc. Individual flags always take
precedence.

### `smart-control train`

Train a segmentation model on a dataset.

| Flag | Required | Default | Description |
|---|---|---|---|
| `--dataset`, `-d` | Yes | — | Path to the dataset directory (must contain `splits.csv`). |
| `--config-dir` | No | — | Directory with YAML config files. |
| `--augmentation-config` | No | — | Augmentation YAML path (overrides `--config-dir`). |
| `--model-config` | No | — | Model YAML path (overrides `--config-dir`). |
| `--training-config` | No | — | Training YAML path (overrides `--config-dir`). |
| `--checkpoint-dir` | No | `<output-dir>/checkpoints` | Where the best model checkpoint is saved. |
| `--output-dir`, `-o` | No | `./output/` | Base output directory. |
| `--device` | No | `auto` | `cuda`, `cpu`, or `auto` (auto-detects CUDA). |

At least one configuration source must be provided (via `--config-dir` or
individual `--*-config` flags).

### `smart-control evaluate`

Evaluate a trained segmentation model on a test dataset.

| Flag | Required | Default | Description |
|---|---|---|---|
| `--dataset`, `-d` | Yes | — | Path to the dataset directory. |
| `--checkpoint` | Yes | — | Path to the `.pt` checkpoint file. |
| `--config-dir` | No | — | Directory with YAML config files. |
| `--augmentation-config` | No | — | Augmentation YAML path (overrides `--config-dir`). |
| `--model-config` | No | — | Model YAML path. Only needed for legacy checkpoints that lack model metadata. |
| `--evaluation-config` | No | — | Evaluation YAML path (overrides `--config-dir`). |
| `--output-dir`, `-o` | No | `./output/evaluation/` | Base output directory. |
| `--device` | No | `auto` | `cuda`, `cpu`, or `auto`. |
| `--max-samples` | No | All | Limit the number of test samples to evaluate. |

Checkpoints saved by Smart Control contain model metadata, so `--model-config`
is not needed — the model architecture is reconstructed automatically.  You
only need `--model-config` when loading checkpoints from older versions of
Smart Control.

### `smart-control template`

Generate template YAML configuration files with all available options
documented inline.

| Flag | Default | Description |
|---|---|---|
| `--output-dir`, `-o` | `./` | Where to write the template files. |
| `--all` | *(if no specific flag)* | Generate all four templates. |
| `--augmentation` | — | Generate only `augmentation.yaml`. |
| `--model` | — | Generate only `model.yaml`. |
| `--training` | — | Generate only `training.yaml`. |
| `--evaluation` | — | Generate only `evaluation.yaml`. |
| `--force`, `-f` | — | Overwrite existing files. |

```bash
# Generate all four templates
smart-control template --output-dir ./my-config/

# Generate only the model config
smart-control template --model --output-dir ./my-config/
```

### `smart-control info`

Display the available models, transforms, optimizers, metrics, and other
components registered in Smart Control.  Useful when filling out configuration
files.

| Flag | Description |
|---|---|
| `--models` | List registered model architectures. |
| `--transforms` | List available augmentation transforms. |
| `--optimizers` | List allowed optimizers. |
| `--loss-functions` | List allowed loss functions. |
| `--metrics` | List available evaluation metrics. |
| `--visualizers` | List available visualizers. |
| `--exporters` | List available exporters. |
| `--callbacks` | List registered training callbacks. |
| `--schedulers` | List available LR schedulers. |

Without flags, a unified table showing all categories is displayed.

```bash
# Show everything
smart-control info

# Show only models and transforms
smart-control info --models --transforms
```

## Configuration Files

Configuration files are YAML documents.  Run `smart-control template` to
generate commented templates with the latest list of available options.

### `augmentation.yaml`

Defines data augmentation transforms applied during training, validation, and
testing.  Each section can be enabled or disabled independently.  Transform
parameters are passed directly to `torchvision.transforms.v2`.

```yaml
augmentations:
  train:
    enabled: true
    seed: 28
    transforms:
      - RandomResizedCrop:
            size: [64, 64]
            scale: [0.8, 1.0]
      - RandomRotation:
            degrees: 180
      - RandomHorizontalFlip:
            p: 0.5

  validation:
    enabled: false

  test:
    enabled: true
    transforms:
      - CenterCrop:
            size: [128, 128]
```

| Key | Description |
|---|---|
| `train` / `validation` / `test` | Section for each data split. |
| `.enabled` | Whether to apply transforms to this split. |
| `.seed` | Optional RNG seed for this section (fallback: dataset global seed). |
| `.transforms` | List of transform name → parameter dict pairs. |

Available transforms: `CenterCrop`, `ColorJitter`, `GaussianBlur`, `Normalize`,
`RandomCrop`, `RandomErasing`, `RandomGrayscale`, `RandomHorizontalFlip`,
`RandomResizedCrop`, `RandomRotation`, `RandomVerticalFlip`, `Resize`.

### `model.yaml`

Defines the neural network architecture.  The `name` must match a registered
model (run `smart-control info --models` to see available ones).  The `params`
dict is passed directly to the model constructor.

```yaml
model:
  name: "unet"
  params:
    in_channels: 224
    n_classes: 3
    depth: 6
    wf: 5
    padding: true
    batch_norm: false
    up_mode: upconv
```

| Key | Description |
|---|---|
| `name` | Registered model name. |
| `params` | Model-specific constructor arguments. |

### `training.yaml`

Sets hyperparameters for the training and validation loops.

```yaml
train:
  batch_size: 16
  shuffle: true
  num_epochs: 50
  learning_rate: 0.001
  optimizer: Adam
  loss_function: CrossEntropyLoss

validation:
  batch_size: 16
  shuffle: false
  learning_rate: 0.001
  optimizer: Adam
  loss_function: CrossEntropyLoss

runtime:
  seed: 28
  deterministic: false
  num_workers: 0
  validation_split: 0.2
```

| Key | Description |
|---|---|
| `train` / `validation` | Per-phase settings. |
| `.batch_size` | Mini-batch size. |
| `.shuffle` | Shuffle the training set each epoch. |
| `.num_epochs` | Number of training epochs (training section only). |
| `.learning_rate` | Learning rate for the optimizer. |
| `.optimizer` | Optimizer class name (`Adam`, `SGD`, `AdamW`, `RMSprop`, `Adagrad`). |
| `.loss_function` | Loss function name (`CrossEntropyLoss`, `MSELoss`, etc.). |
| `runtime.seed` | Global random seed for reproducibility (optional). |
| `runtime.deterministic` | Enable cuDNN deterministic mode (slower but reproducible). |
| `runtime.num_workers` | DataLoader worker processes. |
| `runtime.validation_split` | Fraction of dev images used for validation (0 = disabled). |

### `evaluation.yaml`

Configures the evaluation pipeline: which metrics to compute, what
visualizations to generate, and how to export results.

```yaml
metrics:
  enabled: true
  segmentation: [iou, dice, accuracy]
  classification: []

visualization:
  enabled: true
  segmentation:
    - segmentation_overlay
  classification: []
  save_format: png
  output_dir: ./evaluation/visualizations/
  max_samples: 8

export:
  enabled: true
  metrics:
    format: json
    output_dir: ./evaluation/results/
  predictions:
    format: npy
    output_dir: ./evaluation/predictions/

report:
  generate_html: false
  output_path: ./evaluation/report.html
  include_timestamp: true

runtime:
  retention_mode: auto
  retention_budget_mb: 512
  sample_size: 16
  seed: 28
  deterministic: false
```

| Section | Key | Description |
|---|---|---|
| `metrics` | `enabled` | Enable metric computation. |
| | `segmentation` | List of metric names (`iou`, `dice`, `accuracy`). |
| `visualization` | `enabled` | Enable visualization generation. |
| | `segmentation` | List of visualizer names (`segmentation_overlay`). |
| | `save_format` | Output image format (`png`, `jpg`, `pdf`, `svg`). |
| | `output_dir` | Where visualizations are saved. |
| | `max_samples` | Maximum number of samples to visualize. |
| `export` | `enabled` | Enable result export. |
| | `metrics.format` | Export format for metrics (`json`, `csv`, `yaml`). |
| | `predictions.format` | Export format for predictions (`npy`, `png`). |
| `runtime` | `retention_mode` | How to retain predictions in memory (`auto`, `full`, `sampled`). |
| | `retention_budget_mb` | Max memory (MB) for in-memory predictions. |
| | `sample_size` | Samples kept in `sampled` retention mode. |

## Dataset Format

Smart Control is compatible with datasets produced by
[SpectralDatamaker v0.6.3](https://pypi.org/project/spectral-datamaker/).

### Required structure

```
dataset-root/
  ├── splits.csv          # Maps each image to "dev" or "test" split
  ├── metadata.json       # Dataset-level metadata
  ├── images/             # Hyperspectral images (ENVI or .npy)
  └── masks/              # Segmentation masks (.npy)
```

### Creating a dataset with SpectralDatamaker

Install SpectralDatamaker and follow its documentation to create a dataset
from raw hyperspectral data:

```bash
pip install spectral-datamaker==0.6.3
```

Key requirements:
- Every dataset **must** contain a `splits.csv` file generated by
  SpectralDatamaker (via `--dev-split` during creation or the `splits`
  command).
- `splits.csv` assigns each image to the `dev` split (used for training and
  validation) or the `test` split (used for final evaluation).

## Extending the Library

Smart Control is designed to be extensible.  You can add new model
architectures, metrics, visualizers, callbacks, and CLI commands.  See the
developer documentation for detailed guides:

| Document | Description |
|---|---|
| [Architecture Overview](docs/ARCHITECTURE.md) | Module map and data flow. |
| [Models](docs/MODELS.md) | Adding new model architectures. |
| [Evaluator](docs/EVALUATOR.md) | Adding metrics, visualizers, and exporters. |
| [Callbacks & Schedulers](docs/CALLBACKS_AND_SCHEDULERS.md) | Adding training callbacks and LR schedulers. |
| [Datasets](docs/DATASETS.md) | Dataset loading internals. |
| [Parsers](docs/PARSERS.md) | Configuration parsing system. |
| [Contributing](docs/CONTRIBUTING.md) | Development setup and guidelines. |

## License

This project is licensed under the terms of the MIT license, see the [LICENSE](LICENSE.txt) file for details.
