Metadata-Version: 2.4
Name: nf-meta
Version: 0.2.3
Summary: nf-meta is a cli tool for building, validating and running meta-pipelines based on the Nextflow workflow language
Author-email: Julian Flesch <julian.flesch@qbic.uni-tuebingen.de>
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/bmds-tue/nf-meta
Project-URL: Issues, https://github.com/bmds-tue/nf-meta/issues
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: click>=8.3.1
Requires-Dist: fastapi>=0.128.0
Requires-Dist: flask>=3.1.2
Requires-Dist: networkx>=3.6.1
Requires-Dist: numpy>=2.4.2
Requires-Dist: packaging>=26.0
Requires-Dist: pydantic>=2.12.5
Requires-Dist: pyyaml>=6.0.3
Requires-Dist: requests>=2.32.5
Requires-Dist: rich>=14.3.3
Requires-Dist: uvicorn>=0.40.0
Dynamic: license-file

# nf-meta: metapipeline representation, editor, and runner

This project features a reproducible representation for meta-pipelines, featuring an interactive editor running locally in a browser, as well as a cli based validator and runner.

**This is a beta version: Things can be unstable and change**

Find a growing collection of examples in our dedicated [showcase repository](https://github.com/bmds-tue/nf-meta-showcases/tree/main).

## Representation

This project proposes a representation for how metapipeline can be generalized in a in `.yml` file format.
The most important keys for a metapipeline in this format are: 
* `config_version` - which allows the representation to change over time without breaking
* `workflows` - lists the nextflow / nf-core pipeline _runs_ 
* `transitions` - describe the flow through the metapipelines
* `globals` - defines settings that apply to all workflows

More sections could follow, potentially starting with a sort of `adapter` key,
where transitions or Groovy glue code is described.

```{yaml}
config_version: 0.1.0
globals:
  nextflow_version: 25
workflows:
  n01:
    name: nf-core/fetchngs
    version: dev
    params:
      input: fetchngs-ids.csv
      outdir: out_fetchngs
      nf_core_pipeline: rnaseq
      nf_core_rnaseq_strandedness: auto
  n02:
    name: nf-core/rnaseq
    version: dev
    params:
      input: ${n01:params:outdir}/samplesheet/samplesheet.csv
      genome: EB1
      outdir: out_rnaseq
transitions:
- source: n01
  target: n02
```

This project features at it core validation logic for these sort of `.yml` representations.


To enable referencing fields form other workflows, constructs like could be made possible:
```
    # write this:
    input: ${n01:params:outdir}/samplesheet/samplesheet.csv
    # instead of this:
    input: fetchngs_out/samplesheet/samplesheet.csv
```


## Editor

This project features a small editor which intends to ease the creation and updating
of these config files, by visualizing the config as a graph and 
offering user-firendly form for entering and validating values.

![editor](assets/nf-meta-screenshot.png)


## Metapipeline Runners

Currently:
* Python Runner: Wraps nextflow commands, exectutes in dag order

Planned:
1. [nf-cascade](https://github.com/mahesh-panchal/nf-cascade/) runner: Convert config into one nextflow daisy-chaining nextflow script
2. Seqera Platform runner:
Start and monitor locally, call and poll a Platform instance with run parameters via API to handle running Nextflow
3. Meta-Pipeline runner:
compile config into new monolithic nextflow project, that imports required workflows to achieve most efficient orchestration + graceful errors handling


## Install from PyPI

The released version of this package can be installed from the [Python Package Index](https://pypi.org/project/nf-meta/)
```
# with pip
pip install nf-meta

# with uv 
uv add nf-meta
```

## Install and run the Development Setup

Install and run the frontend
```
$ cd /src/nf_meta/editor/frontend
$ npm install
$ npm run dev
```

Run the cli in dev mode
```
NF_META_DEVMODE=1 uv run nf-meta editor [</path/to/metapipeline.yml>]
```

## Project Structure


* Engine: Config Validation, Editor Session with History and exposing API
* Runners
* Editor frontend for creating or editing a config

