Metadata-Version: 2.4
Name: studypype
Version: 3.0.0
Summary: Folder-based Python research workflows with explicit YAML parameter sweeps
Author-email: Johannes Maierhofer <j.maierhofer@tum.de>
License-Expression: MIT
Project-URL: Homepage, https://github.com/Maierhofer-Technology/studypype
Project-URL: Repository, https://github.com/Maierhofer-Technology/studypype.git
Project-URL: Issues, https://github.com/Maierhofer-Technology/studypype/issues
Keywords: data-pipeline,parameter-sweep,reproducibility,research-software,workflow,yaml
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering
Requires-Python: >=3.13
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: PyYAML>=6.0
Dynamic: license-file

<p align="center">
  <img src="https://raw.githubusercontent.com/Maierhofer-Technology/studypype/main/docs/studypype-icon.png" width="144" alt="StudyPype icon: a study branching into a result tree">
</p>

# StudyPype

StudyPype is a lightweight Python toolkit for traceable research and
data-processing workflows that are easiest to understand as folders.

You describe a study in YAML. StudyPype builds a tree, creates the folder
structure, and runs one Python job class per executable node. Intermediate
results stay next to the step that produced them, so it is easy to inspect a
run afterwards.

![StudyPype turns a YAML definition into a dependency-aware execution tree and matching result folders.](https://raw.githubusercontent.com/Maierhofer-Technology/studypype/main/docs/studypype-workflow.svg)

## Installation

Install the released package from PyPI:

```bash
pip install studypype
```

For editable development from a repository checkout:

```bash
uv sync
```

StudyPype requires Python 3.13 or newer.

## Working With LLMs

The repository includes a focused StudyPype skill at
`.agents/skills/studypype/SKILL.md`.
Codex discovers repository skills automatically when it works in the checkout;
you can also invoke this one explicitly as `$studypype`.
[`llms.txt`](llms.txt) provides a compact, tool-independent project map, and
[`CONTRIBUTING.md`](CONTRIBUTING.md) records the public development rules.

See the official [Codex skill documentation](https://developers.openai.com/codex/skills)
for skill discovery and invocation details.

## Parameter Sweeps

StudyPype uses the YAML tag `!sweep` for parameter sweeps. A value is swept only
when it carries this tag.

Plain YAML lists are regular values:

```yaml
parameters:
  - point: [0.0, 1.0, 2.0]
```

Explicit scalar sweep:

```yaml
parameters:
  - cutoff_hz: !sweep [5, 10, 20]
```

Explicit nested coordinate sweep:

```yaml
parameters:
  - plane:
      origin: !sweep
        - [0.0, 0.0, 0.0]
        - [0.0, 0.02, 0.0]
      normal: [0.0, 1.0, 0.0]
```

Explicit paired-object sweep:

```yaml
parameters:
  - plane: !sweep
      longitudinal:
        origin: [0.0, 0.0, 0.0]
        normal: [0.0, 1.0, 0.0]
      transverse:
        origin: [0.0, 0.0, 0.0]
        normal: [1.0, 0.0, 0.0]
```

Use the paired-object form when values must stay coupled. Independent `!sweep`
tags are combined as a Cartesian product.

Mapping keys become the folder labels for mapped sweeps. Sequence values are
formatted into stable folder labels automatically.

## Loading YAML

Because `!sweep` is a StudyPype-specific YAML tag, load study files with
StudyPype:

```python
import studypype as Pype

yaml_content = Pype.load_yaml_file("study.yaml")
```

Do not use `yaml.safe_load(...)` for StudyPype study files that may contain
`!sweep`; PyYAML does not know that tag by default.

## Minimal Example

```yaml
study:
  name: results/FilterStudy
  max_workers: 2
  jobs:
    - RawData:
        script: ImportData
        recalculate: true
        parameters:
          - source: Measurement1.csv

    - Filter:
        script: ClipData
        dependency: RawData
        recalculate: true
        parameters:
          - cutoff_hz: !sweep [5, 10]
          - window_s: 2

    - Plot:
        script: MultiPlot
        dependency: Filter
        parameters:
          - export: png
```

After expansion, the folder tree is:

```text
results/FilterStudy/
  RawData/
    Filter/
      cutoff_hz5/
        Plot/
      cutoff_hz10/
        Plot/
```

`Filter` is the implicit group folder. The executable jobs are `cutoff_hz5` and
`cutoff_hz10`. `Plot` is copied below each parameter combination.

## Running A Study

```python
import studypype as Pype

yaml_content = Pype.load_yaml_file("study.yaml")
study_name = yaml_content["study"]["name"]
study = Pype.Tree(study_name)
study.createfromYAML(yaml_content)
study.expand_sweeps()

Pype.job_runner.run(study, job_folder="tasks")
```

With `max_workers > 1`, the runner uses Python processes. On Windows, keep the
runner call inside an `if __name__ == "__main__":` block.

## Writing Jobs

Job classes live in Python files named `Job__Something.py`. The class name is
used in the YAML `script` field.

```python
from studypype import Job, register_job


@register_job
class ClipData(Job):
    def func(self, path, parset, **kwargs):
        print(path)
        print(parset)
        print(kwargs.get("datalist"))
        return True
```

`path` is the output folder for the current job. `parset` is a flat dictionary
made from the YAML `parameters`. `datalist` contains output folders from tree
ancestors and declared dependencies.

## Dependencies And Folders

A single dependency places the job below its predecessor:

```yaml
- Compute:
    script: ComputeResult
    parameters:
      - alpha: !sweep [10, 20]

- MakeVideo:
    script: MakeVideo
    dependency: Compute
```

If `Compute` has an explicit `!sweep`, `MakeVideo` is copied under every
expanded combination:

```text
ReadInput/
  Compute/
    alpha10/
      MakeVideo/
    alpha20/
      MakeVideo/
```

When a job depends on multiple previous jobs, StudyPype keeps the result tree
simple: the job is placed under the nearest common folder of those dependencies.

```yaml
- CompareMethods:
    script: CompareMethods
    dependency: [Linear, Riesz]
```

For a tree with `Linear` and `Riesz` below `ReadInputFrames`, the comparison
folder becomes:

```text
ReadInputFrames/
  Linear/
  Riesz/
  CompareMethods/
```

This keeps fan-out easy to navigate and gives fan-in jobs a predictable place
without turning the visible results into a full graph.

When a dependency ends with `*`, the job is placed once at the parameter sweep
group and receives every expanded result with that job name in `datalist`.

```yaml
- Filter:
    script: BandpassFilter
    parameters:
      - in_h5_path: !sweep [patient_a.h5, patient_b.h5]

- Align:
    script: AlignBeats
    dependency: Filter

- PlotAll:
    script: PlotAlignedBeats
    dependency: Align*
```

The folder tree becomes:

```text
Filter/
  in_h5_pathpatient_a.h5/
    Align/
  in_h5_pathpatient_b.h5/
    Align/
  PlotAll/
```

Use the `*` form for aggregate plots, summary tables, and reports that combine
all parameter or patient runs from the predecessor.

## Parallel Runs

StudyPype reads `study.max_workers` from the YAML:

```yaml
study:
  name: results/MyStudy
  max_workers: 3
```

With `max_workers > 1`, the runner uses Python processes. A job starts when all
of its executable ancestors and declared dependencies have finished. This means
parameter combinations can run at the same time after their shared input step is
done, while follow-up jobs still wait for their own parent result.

Choose `max_workers` conservatively for large image or video studies. Running
more jobs in parallel also multiplies memory use.

## Development

Run the test suite from the repository root:

```bash
uv run python -m unittest discover -s tests -v
```

Packaging and PyPI release instructions are documented in
[`docs/PUBLISHING.md`](docs/PUBLISHING.md). Pushing a stable semantic-version tag
such as `v3.0.0` builds, validates, and uploads the matching package to PyPI
through Trusted Publishing; no long-lived PyPI API token is stored in GitHub.

## Citation

If StudyPype supports your research, please cite the software using the
metadata in [`CITATION.cff`](CITATION.cff). GitHub can export this metadata in
common citation formats.

## License

StudyPype is authored by Johannes Maierhofer and released under the
[MIT License](LICENSE).
