Metadata-Version: 2.3
Name: silver-doodle
Version: 0.1.6
Summary: Lightning training runs, locally or on Kubeflow Trainer
Author: OERI
Author-email: OERI <25751602+oddeirikigland@users.noreply.github.com>
Requires-Dist: kubeflow>=0.5.0
Requires-Dist: packaging>=24.0
Requires-Dist: python-dotenv>=1.2.3
Requires-Dist: pyyaml>=6.0
Requires-Dist: jsonargparse[signatures]>=4.27.7 ; extra == 'train'
Requires-Dist: lightning>=2.6.5 ; extra == 'train'
Requires-Dist: mlflow>=3.15.2 ; extra == 'train'
Requires-Dist: torch==2.2.2 ; platform_machine == 'x86_64' and sys_platform == 'darwin' and extra == 'train'
Requires-Dist: numpy<2 ; platform_machine == 'x86_64' and sys_platform == 'darwin' and extra == 'train'
Requires-Dist: torch>=2.9 ; (platform_machine != 'x86_64' and extra == 'train') or (sys_platform != 'darwin' and extra == 'train')
Requires-Dist: numpy>=2 ; (platform_machine != 'x86_64' and extra == 'train') or (sys_platform != 'darwin' and extra == 'train')
Requires-Python: >=3.12
Provides-Extra: train
Description-Content-Type: text/markdown

# silver-doodle

Lightning training runs with [LightningCLI](https://lightning.ai/docs/pytorch/stable/cli/lightning_cli.html),
run locally or on Kubeflow Trainer, with the config and metrics logged to MLflow.

## Setup

```bash
uv sync --extra train  # train locally, and resolve configs locally before submitting
uv sync                # or: only submit to Kubeflow
cp .env.example .env   # set MLFLOW_TRACKING_URI and credentials
```

## Run locally

```bash
uv run silver-doodle fit --config configs/example.yaml
uv run silver-doodle fit --config configs/example.yaml --trainer.max_epochs 2   # override anything
uv run silver-doodle fit --config configs/example.yaml --print_config           # show the full config
```

## Run on Kubeflow

`silver-doodle-submit` takes the same arguments as `silver-doodle`:

```bash
uv run silver-doodle-submit --dry-run fit --config configs/example.yaml   # prepare only
uv run silver-doodle-submit fit --config configs/example.yaml                            # 1 GPU
uv run silver-doodle-submit fit --config configs/example.yaml --trainer.accelerator cpu  # CPU only
uv run silver-doodle-submit fit --config configs/example.yaml \
    --trainer.accelerator gpu --trainer.devices 2 --trainer.num_nodes 2
```

1. With the `train` extra, the full config is resolved locally with `--print_config`, so mistakes
   fail before submitting. Without it, the config files and overrides are sent as they are and
   resolved in the pod, and the pod size is read from the `trainer` settings in them.
2. The project you run from (the nearest `pyproject.toml` at or above the working directory) is
   built into a wheel with `uv build`, uncommitted changes included. That's this repo, or your own
   project using silver-doodle for its models and data: then the pod also installs the project's
   dependencies and the same silver-doodle version from PyPI. Outside any project (`uvx` in a
   plain folder), the pod only installs that silver-doodle version.
3. The pod installs the `train` dependencies and silver-doodle, then runs
   `silver-doodle` with the shipped config files.

The progress bar is turned off in the pod unless you set `--trainer.enable_progress_bar`.
Pod resources come from the config: `trainer.num_nodes` nodes, each with `trainer.devices` GPUs
when `trainer.accelerator` is `auto` (Lightning's default), `gpu` or `cuda`. A pod only sees the
GPUs it requests, so `auto` requests them: 1 per node unless `trainer.devices` is a count. Use
`--trainer.accelerator cpu` for a CPU-only job. If no GPU node is free, the job waits in Pending.
The MLflow env vars are forwarded to the pod.
Logs, the resolved config and the wheel are saved in `logs/<timestamp>/`.

`MLFLOW_TRACKING_URI` must point to a tracking server, not `sqlite:///...`: the URI ends up in the
resolved config (it is `MLFlowLogger`'s default), and a local database in the pod is lost with it.

## Development

```bash
uv run pytest
uv run ruff check .
uv run ruff format --check .
uv run ty check .
```
