Metadata-Version: 2.4
Name: flowrep
Version: 0.4.1
Summary: flowrep - Your premier tool for workflow representations
Project-URL: Homepage, https://pyiron.org/
Project-URL: Documentation, https://flowrep.readthedocs.io
Project-URL: Repository, https://github.com/pyiron/flowrep
Author-email: Sam Waseda <waseda@mpie.de>, Liam Huber <liamhuber@greyhavensolutions.com>
License: BSD 3-Clause License
        
        Copyright (c) 2024, Max-Planck-Institut für Nachhaltige Materialien GmbH - Computational Materials Design (CM) Department
        All rights reserved.
        
        Redistribution and use in source and binary forms, with or without
        modification, are permitted provided that the following conditions are met:
        
        * Redistributions of source code must retain the above copyright notice, this
          list of conditions and the following disclaimer.
        
        * Redistributions in binary form must reproduce the above copyright notice,
          this list of conditions and the following disclaimer in the documentation
          and/or other materials provided with the distribution.
        
        * Neither the name of the copyright holder nor the names of its
          contributors may be used to endorse or promote products derived from
          this software without specific prior written permission.
        
        THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
        AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
        IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
        DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
        FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
        DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
        SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
        CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
        OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
        OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
License-File: LICENSE
Keywords: pyiron
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: BSD License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering
Requires-Python: <3.15,>=3.11
Requires-Dist: pydantic<2.13.0,>=2.12.0
Requires-Dist: pyiron-snippets<2.0.0,>=1.2.1
Provides-Extra: notebooks
Requires-Dist: numpy==2.4.3; extra == 'notebooks'
Provides-Extra: pwd
Requires-Dist: python-workflow-definition==0.1.4; extra == 'pwd'
Provides-Extra: storage
Requires-Dist: bagofholding==0.1.10; extra == 'storage'
Provides-Extra: storage-widget
Requires-Dist: bagofholding==0.1.10; extra == 'storage-widget'
Requires-Dist: ipytree==0.2.2; extra == 'storage-widget'
Description-Content-Type: text/markdown

# flowrep

[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/pyiron/flowrep/HEAD?urlpath=%2Fdoc%2Ftree%2Fuser-guide.ipynb)
[![License](https://img.shields.io/badge/License-BSD_3--Clause-blue.svg)](https://opensource.org/licenses/BSD-3-Clause)
[![Coverage](https://codecov.io/gh/pyiron/flowrep/graph/badge.svg)](https://codecov.io/gh/pyiron/flowrep)
[![Documentation](https://readthedocs.org/projects/flowrep/badge/?version=latest)](https://flowrep.readthedocs.io/en/latest/?badge=latest)

[![Anaconda](https://anaconda.org/conda-forge/flowrep/badges/version.svg)](https://anaconda.org/conda-forge/flowrep)
[![Last Updated](https://anaconda.org/conda-forge/flowrep/badges/latest_release_date.svg)](https://anaconda.org/conda-forge/flowrep)
[![Platform](https://anaconda.org/conda-forge/flowrep/badges/platforms.svg)](https://anaconda.org/conda-forge/flowrep)
[![Downloads](https://anaconda.org/conda-forge/flowrep/badges/downloads.svg)](https://anaconda.org/conda-forge/flowrep)


## flowrep — Workflow Recipes from Python

**flowrep** turns plain Python functions into shareable, versionable *workflow
recipes* — JSON-serialisable graphs that describe *what* to compute (which
functions, how they connect) without doing the computation or holding any data.
Recipes are prospective blueprints that a Workflow Management System (WfMS) can
digest, visualise, and execute.

Flowchart-style representations are already the lingua franca for describing
processes in science and engineering. flowrep gives you a way to author them in
Python and pass them around as simple JSON, validated by
[Pydantic](https://docs.pydantic.dev/) and enriched with version provenance so
they stay robust through time.


## Installation

```bash
conda install -c conda-forge flowrep   # recommended
pip install flowrep                     # also available from PyPI
```


## Quick Start

Define a couple of simple functions — it will be helpful to make our return statements 
nicely named variables, but it's not a necessity:

```python
>>> import flowrep as fr

>>> def add(a, b):
...     return a + b

>>> def multiply(x, y):
...     product = x * y
...     return product

```

Compose them into a workflow with the `@workflow` decorator.  The body reads like 
normal Python — assign function calls to variables, wire outputs into inputs, 
return results:

```python
>>> @fr.workflow
... def linear(x, slope, intercept):
...     """y = slope * x + intercept"""
...     scaled = multiply(x, slope)
...     result = add(scaled, intercept)
...     return result

```

The decorated function is still just its usual callable self:

```python
>>> linear(3, 2, 1)
7

```

But it now carries a recipe on its `flowrep_recipe` attribute — a pure-data
Pydantic model whose structure you can inspect directly
(`model_dump(mode="json")` gives a Python dict, `model_dump_json()` gives a
JSON string):

```python
>>> recipe = linear.flowrep_recipe.model_dump(mode="json")
>>> recipe["type"]
'workflow'
>>> recipe["inputs"]
['x', 'slope', 'intercept']
>>> recipe["outputs"]
['result']
>>> sorted(recipe["nodes"])
['add_0', 'multiply_0']
>>> recipe["nodes"]["multiply_0"]["outputs"]
['product']
>>> recipe["edges"]
{'add_0.a': 'multiply_0.product'}
>>> recipe["output_edges"]
{'result': 'add_0.output_0'}
>>> recipe["nodes"]["add_0"]["reference"]["info"]["qualname"]
'add'

```

A key difference between a workflow graph and typical python is that for a graph we 
need named handles for all of our output values.
We can see in the `edges` that the recipe nicely used our return variable to give 
the "multiply" a nice output label.
For our "add" node, what we returned couldn't be parsed nicely as a label, so in 
`output_edges` we see it got assigned a default name `"output_0"`.

Every `"atomic"` node for running a Python function and every `"workflow"` not created 
by parsing a function defition carries a `reference` recording for the Python function
to which it maps (module, qualname, and package version when available), so a WfMS can resolve
and execute them later. The recipe also captures exactly how data flows: 
`input_edges` wire workflow inputs to child node ports, `edges` connect sibling
outputs to inputs, and `output_edges` name which child port produces each 
workflow output.

The recipe *is* the shareable artifact. It could equally have been authored in a
GUI editor or generated by another tool — flowrep doesn't care where it came
from, only that it validates. (The availability of referenced functions on your
machine, and their actual behaviour, lie outside flowrep's responsibility. We
simply check that the recipe is internally consistent.)


## Features

**Pure Python foundation.** Knowing basic Python is enough to get started. The
decorators `@atomic` and `@workflow` (plus their functional counterparts
`parse_atomic` and `parse_workflow`) handle the heavy lifting; the recipe format
is plain JSON.

**Pydantic validation.** Recipes are full Pydantic models — they validate on
construction, serialise with `model_dump_json()`, and deserialise with
`model_validate_json()`.

**Version provenance.** Every node reference carries module, qualname, and
(optionally) package version metadata. Constraints like `forbid_main`,
`forbid_locals`, and `require_version` can be enforced at parse time to ensure
recipes only reference published, versioned code.

**Composability.** Workflows nest: a `@workflow`-decorated function can call
another `@workflow`-decorated function, and the child's recipe is embedded as a
sub-graph. Build complex pipelines from small, testable pieces.

**Flow control.** Recipes go beyond simple DAGs. The `@workflow` parser
recognises `for`, `while`, `if`/`elif`/`else`, and `try`/`except` — the same
control structures you use in real code. Flow control nodes are inherently
dynamic: their exact execution path depends on data and cannot be known until
run-time, but their IO signature is always fully known a priori.

### Example: flow control and nesting

So far we've seen `"workflow"` nodes, and alluded to `"atomic"` nodes.
You can also explicitly attach an atomic recipe to a function definition, but most of 
the time you won't need to since the workflow parser auto-parses undecorated function 
calls as atomic nodes.

On the other hand, we can parse `@workflow`-decorated functions _as workflows_ to 
compose complex workflows with nested subgraphs.

Beyond simple `"atomic"` and DAG `"workflow"` nodes, there are also recipe formats 
for flow control structures like `while`. 
Just like the other nodes, these can be written directly as JSON, or parsed inside a 
workflow from a python function definition -- with some syntax restrictions.
This is covered in more detail in the [user guide](https://mybinder.org/v2/gh/pyiron/flowrep/HEAD?urlpath=%2Fdoc%2Ftree%2Fuser-guide.ipynb), but here let's see both of these features 
in use at once by nesting a while loop inside a sub-workflow.
Let's look at a `while` loop, where our main syntax restriction is that the condition 
must be a function call:

```python
>>> def is_less_than_target(value, target):
...     result = value < target
...     return result

>>> @fr.atomic  # This is optional, but doesn't hurt us
... def double(x):
...     doubled = x * 2
...     return doubled

>>> @fr.workflow
... def double_until(x, target):
...     """Repeatedly double `x` until it reaches `target`."""
...     while is_less_than_target(x, target):
...         x = double(x)
...     return x

>>> double_until(3, 40)
48

```

And then we can compose workflows by nesting:

```python
>>> @fr.workflow
... def double_and_add(a, b, target):
...     big_a = double_until(a, target)
...     result = add(big_a, b)
...     return result

>>> double_and_add(3, 100, 40)
148

```

The resulting recipe captures the full structure — the while-loop, the nested
workflow, and all the edges between them — as a single pydantic model that can be dumped 
to a JSON document.

We can see the layers of nested subgraphs:

```python
>>> recipe_json = double_and_add.flowrep_recipe.model_dump(mode="json")
>>> [child for child in recipe_json["nodes"]]
['double_until_0', 'add_0']

>>> [nested_child for nested_child in recipe_json["nodes"]["double_until_0"]["nodes"]]
['while_0']

```

Although the while-node is dynamic -- and therefore we _can't_ know its exact nodes 
until run-time, it must and does still have well-defined IO signature.
We can look at the template it will follow, e.g., by peeking at the part of the recipe 
used for the "while" condition:

```python
>>> recipe_json["nodes"]["double_until_0"]["nodes"]["while_0"]["case"]["condition"]["node"]["type"]
'atomic'

>>> recipe_json["nodes"]["double_until_0"]["nodes"]["while_0"]["case"]["condition"]["node"]["reference"]["info"]["qualname"]
'is_less_than_target'

```

And to see how it will forward it's inputs down into its prospective subgraph:

```python
>>> recipe_json["nodes"]["double_until_0"]["nodes"]["while_0"]["input_edges"]
{'condition.value': 'x', 'condition.target': 'target', 'body.x': 'x'}

```

We run these in the examples above to show two things: first, that even when nested, 
the decorated functions are still just python functions; second, to show in the following 
section that the recipe we parse from this are intended to give the same result as 
these underlying functions when we run the recipe with a WfMS.

## Beyond Recipes: Live Data and Execution

Recipes are *prospective* — they describe a computation template without holding
data. For retrospective analysis (inspecting what actually happened during a
run), flowrep provides two additional layers accessible through the API:

```python
>>> from flowrep.api import tools as frt

```

**`flowrep.api.tools.recipe2live`** converts a recipe into a *live* object — a mutable
data structure whose input and output ports can hold actual Python values. Live
objects mirror the recipe graph but trade JSON-serializability for the ability to
carry arbitrary data:

```python
>>> live_wf = frt.recipe2live(double_and_add.flowrep_recipe)

```

**`flowrep.api.tools.run_recipe`** goes one step further: it executes the recipe with
the provided inputs and returns a fully populated live object. This is powered
by a minimal, built-in WfMS intended as a reference implementation and for use
in tests and documentation (like this!):

```python
>>> retrospective = frt.run_recipe(
...     double_and_add.flowrep_recipe, a=3, b=100, target=40
... )
>>> retrospective.output_ports["result"].value
148

```

Because every child node's ports are populated too, the live graph gives you
full data provenance — you can walk the tree and inspect exactly what each node
received and produced.
For flow control nodes, which are _prospectively_ "black boxes", we find that 
_retrospectively_ they are simple DAGs.
For our while-loop, that means that we can retrospectively inspect the provenance of 
each loop iteration:

```python
>>> while_loop = retrospective.nodes["double_until_0"].nodes["while_0"]
>>> while_loop.nodes["body_0"].output_ports["x"].value
6
>>> while_loop.nodes["body_1"].output_ports["x"].value
12

```

For a deeper look at all available node types, edge semantics, version
provenance, and the live/WfMS layer, see the
[user guide](https://mybinder.org/v2/gh/pyiron/flowrep/HEAD?urlpath=%2Fdoc%2Ftree%2Fuser-guide.ipynb).


## Documentation

- The user guide notebook comprehensively covers all node types, edge models, flow control, versioning, 
  live/retrospective data formats, a demo WfMS implementation, and recipe format converters. 
  Launch it interactively on
  [mybinder](https://mybinder.org/v2/gh/pyiron/flowrep/HEAD?urlpath=%2Fdoc%2Ftree%2Fuser-guide.ipynb).
- [readthedocs](https://flowrep.readthedocs.io/en/latest/)


## Contributing

Contributions are welcome! Please see
[`CODE_OF_CONDUCT.md`](../CODE_OF_CONDUCT.md) for community guidelines.