Metadata-Version: 2.4
Name: phasesweep
Version: 0.3.6
Summary: YAML-driven phase-chained hyperparameter sweeps with Optuna: each phase's winner is set for all following sweeps.
Author: Peter Szemraj
License-Expression: MIT
Project-URL: Homepage, https://github.com/pszemraj/phasesweep
Project-URL: Documentation, https://github.com/pszemraj/phasesweep/tree/main/docs
Project-URL: Repository, https://github.com/pszemraj/phasesweep.git
Project-URL: Issues, https://github.com/pszemraj/phasesweep/issues
Keywords: experiments,hyperparameter-optimization,machine-learning,optuna,sweeps
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: POSIX
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Typing :: Typed
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: optuna<4.10,>=4.0
Requires-Dist: cmaes>=0.10
Requires-Dist: pydantic>=2.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: click>=8.1
Provides-Extra: wandb
Requires-Dist: wandb<0.29,>=0.28; extra == "wandb"
Provides-Extra: mcp
Requires-Dist: mcp<1.28,>=1.27; extra == "mcp"
Provides-Extra: dev
Requires-Dist: hydra-core<1.4,>=1.3; extra == "dev"
Requires-Dist: pytest>=7; extra == "dev"
Requires-Dist: setuptools-scm[simple]>=9.2; extra == "dev"
Requires-Dist: ruff>=0.5; extra == "dev"
Requires-Dist: mypy>=1.10; extra == "dev"
Requires-Dist: types-PyYAML>=6.0; extra == "dev"
Requires-Dist: mcp<1.28,>=1.27; extra == "dev"
Requires-Dist: pre-commit>=3.2; extra == "dev"
Requires-Dist: pathlint>=0.2; extra == "dev"
Dynamic: license-file

# PhaseSweep

PhaseSweep runs phase-chained hyperparameter sweeps from one ordinary YAML file. That file contains your trainer configuration, the metric, and the search plan. PhaseSweep materializes the configured trainer input for every trial, decides what to try next, persists each phase winner, and carries selected values forward as fixed inputs to later phases.

This is useful when a full joint sweep is too expensive or hard to interpret. For example, choose architecture depth, then tune learning rate, then regularization. The [configuration guide](https://github.com/pszemraj/phasesweep/blob/main/docs/config.md#phase-composition) explains the inheritance model and its tradeoffs.

![PhaseSweep phase DAG](https://raw.githubusercontent.com/pszemraj/phasesweep/main/docs/images/diagramA_dag.png)

## How it works

The starter generated by `phasesweep init` searches depth before learning rate (abridged; the full file also sets storage, the working directory, and the metric extractor):

```yaml
experiment: phasesweep_starter

# The trainer receives one complete generated YAML per trial.
trial_command: "python -m phasesweep.examples.fake_train --config {config_path}"

trainer_config:
  model:
    n_layers: 8
    dropout: 0.1
  optimizer:
    lr: 0.0003
    weight_decay: 0.05

metric:
  name: eval_loss
  goal: minimize

phases:
  - name: depth
    n_trials: 2
    sampler: { type: grid }
    search_space:
      model.n_layers: { type: categorical, choices: [6, 8] }

  - name: learning_rate
    inherits: [depth] # the winning model.n_layers is fixed here
    n_trials: 2
    sampler: { type: grid }
    search_space:
      optimizer.lr: { type: categorical, choices: [0.0001, 0.0003] }
```

The packaged fake trainer uses the optional `report_objective(...)` envelope
helper. Your own trainer can report through W&B, ordinary JSON, or logs without
importing PhaseSweep.

## Install and try it

Requirements: Python 3.11+ and a POSIX host for real runs. GPUs are optional.

```bash
python -m pip install phasesweep
phasesweep --version

mkdir phasesweep-demo && cd phasesweep-demo
phasesweep init                       # writes the starter experiment.yaml
phasesweep validate experiment.yaml   # checks it without launching anything
phasesweep run experiment.yaml        # four tiny trials, a few seconds
```

`phasesweep init` never overwrites an existing file; use `-o PATH` for another
destination.

The run log shows the phase chaining directly (abridged):

```text
[depth/trial_0] python -m phasesweep.examples.fake_train --config .../trainer_config.yaml
[depth/trial_1] python -m phasesweep.examples.fake_train --config .../trainer_config.yaml
phase=depth WINNER trial=0 metric=0.3 params={'model.n_layers': 8}
[learning_rate/trial_0] python -m phasesweep.examples.fake_train --config .../trainer_config.yaml
[learning_rate/trial_1] python -m phasesweep.examples.fake_train --config .../trainer_config.yaml
phase=learning_rate WINNER trial=0 metric=0.3 params={'optimizer.lr': 0.0003}
```

The `depth` winner's `model.n_layers: 8` is injected into every `learning_rate` trial's complete YAML. Winners persist in the working directory, so you can inspect them any time (abridged):

```bash
phasesweep show-winners experiment.yaml
```

```yaml
phase: learning_rate
metric:
  eval_loss: 0.3
  goal: minimize
trial_number: 0
params:
  optimizer.lr: 0.0003
effective_overrides:
  model.n_layers: 8
  optimizer.lr: 0.0003
# ... completion state, fingerprints, objective provenance, and confound
# verdicts on the ranked population follow
```

The starter uses `storage: auto`, so its ledger and outputs live together under
`<workdir>/<experiment>/`. The ledger is an Optuna journal: `phasesweep status`
prints its path, and Optuna's own CLI and dashboard can
[read it](https://github.com/pszemraj/phasesweep/blob/main/docs/runtime.md#inspect-the-ledger-with-optuna). See
[runtime behavior](https://github.com/pszemraj/phasesweep/blob/main/docs/runtime.md) for state layout, current-format
requirements, locks, cleanup, GPU isolation, and resume.

## Use your own trainer

Choose the input interface your trainer already accepts:

| Input | Command placeholder |
| --- | --- |
| Complete base-plus-overrides YAML (`yaml_file`, default) | `--config {config_path}` |
| Command-line flags (`argparse`) | `{overrides}` produces `--key=value` |
| Native Hydra overrides (`hydra`) | `{overrides}` produces `key=value` |
| Nested overrides-only JSON (`json_file`) | `--overrides {overrides_path}` |

> [!TIP]
> `phasesweep run experiment.yaml --dry-run` prints one rendered command per
> phase without launching work. Use it to check that your placeholders render
> what the trainer expects.

For ordinary JSON output, pass an output path under `{trial_dir}` to your
trainer and select it with `extractor: {type: json, path: result.json, key: eval.loss}`.
For W&B, use `extractor: {type: wandb, entity: YOUR_ENTITY, project: YOUR_PROJECT, metric_key: eval/loss}`,
have the trainer honor the supplied W&B identity, and install the extra:

```bash
python -m pip install "phasesweep[wandb]"
```

`metric.goal` ranks each trial's selected scalar. Log matching selects the
last observation by default; W&B selects an exact finished-summary key.
The trainer owns evaluation, early stopping, checkpoint selection, and
aggregation. Inheritance transfers parameters, not model weights.

See the [configuration guide](https://github.com/pszemraj/phasesweep/blob/main/docs/config.md) for input types, reporting,
credentials, and selection details. The [Tiny Decoder Enwik8
example](https://github.com/pszemraj/phasesweep/blob/main/examples/tiny_decoder_enwik8/README.md) is a complete real-trainer
integration.

## Connect an agent

The optional MCP server connects an AI agent to experiments you have approved without exposing config or storage paths, trainer commands, environment values, or raw logs. Sampled winner values follow the catalog's `visible_params` policy. Follow the [MCP setup](https://github.com/pszemraj/phasesweep/blob/main/docs/mcp_setup.md) to install the extra, review the catalog authority boundary, configure a client-owned connection, and verify the result.

## Reference

- [Configuration guide](https://github.com/pszemraj/phasesweep/blob/main/docs/config.md): Experiment YAML, search spaces, inheritance, gates, and objective extractors.
- [Configuration reference](https://github.com/pszemraj/phasesweep/blob/main/docs/config_reference.yaml): per-key types, defaults, valid values, interactions, and lifecycle warnings.
- [Runtime behavior](https://github.com/pszemraj/phasesweep/blob/main/docs/runtime.md): filesystem layout, locks, GPU leases, process supervision, fingerprints, and resume.
- [MCP setup](https://github.com/pszemraj/phasesweep/blob/main/docs/mcp_setup.md): installed-package agent onboarding and client-owned setup.
- [MCP operator reference](https://github.com/pszemraj/phasesweep/blob/main/docs/mcp.md): catalog fields, tools, authorization, run state, and recovery.
- [Toy experiment and MCP catalog](https://github.com/pszemraj/phasesweep/blob/main/examples/experiment.yaml): a checkout-local CLI example backed by the packaged fake trainer, plus an [MCP catalog](https://github.com/pszemraj/phasesweep/blob/main/examples/catalog.yaml) whose detached-run state and experiment outputs use absolute scratch paths under `/tmp`.
- [Tiny Decoder Enwik8 example](https://github.com/pszemraj/phasesweep/blob/main/examples/tiny_decoder_enwik8/README.md): real-trainer integration.
- [Durability invariants](https://github.com/pszemraj/phasesweep/blob/main/docs/invariants.md): the ordering rules the ledger, publication, and recovery paths must hold, and the test that owns each one.
- [Development](https://github.com/pszemraj/phasesweep/blob/main/docs/development.md): source checkout, contributor setup, and quality gates.

[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/pszemraj/phasesweep)

## License

MIT. See [LICENSE](https://github.com/pszemraj/phasesweep/blob/main/LICENSE).
