Metadata-Version: 2.4
Name: slurm-sweep
Version: 0.2.1
Summary: Hyperparameter sweeps on slurm
Project-URL: Documentation, https://github.com/quadbio/slurm_sweep
Project-URL: Homepage, https://github.com/quadbio/slurm_sweep
Project-URL: Source, https://github.com/quadbio/slurm_sweep
Author: Marius Lange
Maintainer-email: Marius Lange <marius.lange@bsse.ethz.ch>
License: MIT License
        
        Copyright (c) 2025, QuadBioLab
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: LICENSE
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Python: >=3.11
Requires-Dist: numpy
Requires-Dist: session-info2
Requires-Dist: simple-slurm
Requires-Dist: typer
Requires-Dist: wandb
Provides-Extra: dev
Requires-Dist: pre-commit; extra == 'dev'
Requires-Dist: twine>=4.0.2; extra == 'dev'
Provides-Extra: doc
Requires-Dist: docutils!=0.18.*,!=0.19.*,>=0.8; extra == 'doc'
Requires-Dist: ipykernel; extra == 'doc'
Requires-Dist: ipython; extra == 'doc'
Requires-Dist: myst-nb>=1.1; extra == 'doc'
Requires-Dist: pandas; extra == 'doc'
Requires-Dist: setuptools; extra == 'doc'
Requires-Dist: sphinx-autodoc-typehints; extra == 'doc'
Requires-Dist: sphinx-book-theme>=1; extra == 'doc'
Requires-Dist: sphinx-copybutton; extra == 'doc'
Requires-Dist: sphinx-tabs; extra == 'doc'
Requires-Dist: sphinx>=8.1; extra == 'doc'
Requires-Dist: sphinxcontrib-bibtex>=1; extra == 'doc'
Requires-Dist: sphinxext-opengraph; extra == 'doc'
Provides-Extra: examples
Requires-Dist: scembed; extra == 'examples'
Provides-Extra: test
Requires-Dist: coverage>=7.10; extra == 'test'
Requires-Dist: pytest; extra == 'test'
Requires-Dist: pytest-cov; extra == 'test'
Description-Content-Type: text/markdown

# slurm_sweep: hyperparameter sweeps with W&B on SLURM clusters

[![Tests][badge-tests]][tests]
[![Coverage][badge-coverage]][coverage]
[![Pre-commit.ci][badge-pre-commit]][pre-commit]
[![PyPI][badge-pypi]][pypi]
[![Downloads][badge-downloads]][downloads]

[badge-tests]: https://github.com/quadbio/slurm_sweep/actions/workflows/test.yaml/badge.svg
[badge-coverage]: https://codecov.io/gh/quadbio/slurm_sweep/branch/main/graph/badge.svg
[badge-pre-commit]: https://results.pre-commit.ci/badge/github/quadbio/slurm_sweep/main.svg
[badge-pypi]: https://img.shields.io/pypi/v/slurm_sweep.svg
[badge-downloads]: https://static.pepy.tech/badge/slurm-sweep


`slurm_sweep` is the missing (small) piece to efficiently run hyperparameter sweeps on SLURM clusters by combining the power of weights and biases ([W&B][]) and [simple_slurm][]. It allows you to efficiently parallelize sweeps with job arrays, while tracking experiments and results on W&B. All you need is:

- [W&B][] account.
- `config.yaml` file that defines your sweep.
- `train.py` script, that specifies the actual training and evaluation.

## Getting started

Create an account on [W&B][] and take a look at our examples in the [examples][] folder. These contain both `config.yaml` and `train.py` scripts.

### The config file
You need config file in `yaml` format. This file should have three sections:
- `general`: you need to define at least the `project_name` and the `entity` for the sweep on wandB.
- `slurm`: any valid slurm option. Depends on your cluster, see the `simple_slurm` docs.
- `wandb`: [standard W&B config](https://docs.wandb.ai/guides/sweeps/define-sweep-configuration/) for a hyperparameter sweep.

### The training script
This needs to be a python script that defines the training and evaluation logic. It should call `wandb.init()` and retrieve parameters from `wandb.config`. It can log values using `wandb.log`. See the [W&B docs](https://docs.wandb.ai/guides/sweeps/).

### Submission
Once you're ready, you can test your config file using `slurm-sweep validate-config config.yaml`. If this passes, create a submission script using `slurm-sweep configure-sweep config.yaml`, and submit with `sbatch submit.sh`.

## Installation

You need to have Python 3.11 or newer installed on your system.
If you don't have Python installed, we recommend installing [uv][].

There are two alternative options to install `slurm_sweep`:

- **Install the latest release from [PyPI][]**:

  ```bash
  pip install slurm_sweep
  ```

- **Install the latest development version**:

  ```bash
  pip install git+https://github.com/quadbio/slurm_sweep.git@main
  ```

## Release notes

See the [changelog][].

## Contact
If you found a bug, please use the [issue tracker][].


[examples]: https://github.com/quadbio/slurm_sweep/tree/main/examples
[uv]: https://github.com/astral-sh/uv
[scverse discourse]: https://discourse.scverse.org/
[issue tracker]: https://github.com/quadbio/slurm_sweep/issues
[tests]: https://github.com/quadbio/slurm_sweep/actions/workflows/test.yaml
[changelog]: https://github.com/quadbio/slurm_sweep/blob/main/CHANGELOG.md
[pypi]: https://pypi.org/project/slurm_sweep
[simple_slurm]: https://github.com/amq92/simple_slurm
[W&B]:  https://wandb.ai/site/
[coverage]: https://codecov.io/gh/quadbio/slurm_sweep
[pre-commit]: https://results.pre-commit.ci/latest/github/quadbio/slurm_sweep/main
[pypi]: https://pypi.org/project/slurm_sweep/
[downloads]: https://pepy.tech/project/slurm-sweep
