Metadata-Version: 2.4
Name: vcti-ywf
Version: 3.0.1
Summary: YAML workflow engine — parses, validates, and executes step-based workflows with pluggable Action / Evaluator / Predicate / Iterable methods.
Author: Visual Collaboration Technologies Inc.
Requires-Python: <3.15,>=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pydantic>=2.0
Requires-Dist: PyYAML>=6.0
Requires-Dist: pyyaml-include>=2.0
Requires-Dist: numpy>=1.24
Requires-Dist: pandas>=2.0
Requires-Dist: jinja2>=3.0
Requires-Dist: packaging>=21.0
Requires-Dist: simpleeval>=1.0
Requires-Dist: vcti-array-tree>=1.0.0
Requires-Dist: vcti-config>=3.1.2
Requires-Dist: vcti-data-scope>=1.0.0
Requires-Dist: vcti-enum>=1.1.0
Requires-Dist: vcti-error>=1.0.0
Requires-Dist: vcti-fileloader>=1.0.0
Requires-Dist: vcti-logging>=1.0.0
Requires-Dist: vcti-lookup>=1.0.1
Requires-Dist: vcti-path>=1.0.3
Requires-Dist: vcti-path-format>=1.2.0
Requires-Dist: vcti-path-format-descriptors>=1.1.0
Requires-Dist: vcti-path-tree>=1.1.1
Provides-Extra: hdf5
Requires-Dist: vcti-fileloader-hdf5>=1.0.0; extra == "hdf5"
Provides-Extra: test
Requires-Dist: pytest; extra == "test"
Requires-Dist: pytest-cov; extra == "test"
Provides-Extra: lint
Requires-Dist: ruff; extra == "lint"
Provides-Extra: typecheck
Requires-Dist: mypy; extra == "typecheck"
Dynamic: license-file

# YAML Workflow Framework

YAML workflow engine — parses, validates, and executes step-based workflows
with pluggable Action / Evaluator / Predicate / Iterable methods.

## Overview

`vcti-ywf` is the workflow engine behind VCollab Insights. A workflow is a
directory of YAML files — `about.yaml`, `variables/main.yaml`,
`steps/main.yaml` — that describe an automated process declaratively. The
engine parses those files (with support for `!var`, `!include`, and `!module`
tags), resolves variables, evaluates control-flow blocks (`if-else`,
`for-each`, `try-except`), and dispatches each step to a registered method
plugin.

Method plugins come in four typed shapes — **Action** (side-effecting),
**Evaluator** (produces a value), **Predicate** (boolean test), and
**Iterable** (yields keys for `for-each`). A built-in `stdlib` package ships
common methods; downstream consumers add domain-specific methods (CAE data
extraction, plotting, report generation) by registering their own classes.

Workflow YAML is evaluated against a sandboxed `Context` by default —
`- evaluator: evaluate` expressions go through `simpleeval`, and
`{{ var }}` template interpolation goes through Jinja2's
`SandboxedEnvironment`. Engine-internal context variables
(`__runtime__` and friends) are filtered from author-facing scope.
Hosts that run fully-trusted YAML can opt into raw `eval` or the
full Jinja2 environment by constructing the `Context` themselves and
passing it to `Runner.run()`.

## Installation

```bash
pip install vcti-ywf>=3.0.1
```

### In `requirements.txt`

```
vcti-ywf>=3.0.1
```

### In `pyproject.toml` dependencies

```toml
dependencies = [
    "vcti-ywf>=3.0.1",
]
```

---

## Quick Start

Run a workflow from Python:

```python
from pathlib import Path

from vcti.ywf import Runner, Workflow

workflow = Workflow(Path("path/to/my-workflow"))
workflow.read_all()

result = Runner(workflow).run({"input_file": "data/model.h5"})

print(result.return_value)   # top-level block's value
print(result.outputs)        # declared outputs read from final context
```

For monitoring or debugging, build an `ObservableContext`, attach
observers, and pass it to `run()`:

```python
from vcti.ywf import ObservableContext, Runner
from vcti.ywf.execution.observers.history_observer import HistoryObserver

context = ObservableContext()
context.add_observer(HistoryObserver())

Runner(workflow).run({"input_file": "data/model.h5"}, context)
```

A minimal workflow directory looks like:

```
my-workflow/
├── about.yaml              # title, description, tags, images
├── variables/
│   └── main.yaml           # inputs + outputs declarations
└── steps/
    └── main.yaml           # execution steps
```

`variables/main.yaml` declares the workflow's external contract —
inputs (supplied by the caller) and outputs (produced by the run):

```yaml
inputs:
  - name: input_file
    type: input_file_path
    description: Path to the simulation results
    default: data/sample.h5

outputs:
  - name: mode_count
    type: int
    default: 0
    description: Number of modes extracted.
```

`steps/main.yaml` lists actions, evaluators, predicates, and iterables —
each is a YAML mapping keyed by its method name. The data-scope actions
illustrate the typical pattern:

```yaml
- action: define_data_scope
  name: simulation
  type: paths-group

- action: add_path_source
  path: !var input_file
  format: hdf5-file
  label: results

- action: load_data_scope

- if:
    predicate: is_data_scope_valid
  then:
    - action: log_message
      message: "Data scope ready."
  else:
    - action: log_message
      log_level: error
      message: "Data scope failed to load."
```

## Documentation

**User guides** — start here:

- [docs/tutorial.md](docs/tutorial.md) — walk through the greeting workflow end-to-end with the Python API, then expand to control-flow blocks and data-scope actions.
- [docs/yaml-reference.md](docs/yaml-reference.md) — exhaustive field reference for `about.yaml`, `variables/main.yaml`, `steps/main.yaml`, and every step type (action / evaluator / predicate / iterable / if-else / for-each / try-except).
- [docs/yaml-tags.md](docs/yaml-tags.md) — `!var`, `!include`, and `!module` semantics with scoping rules.
- [docs/plugins.md](docs/plugins.md) — author guide for custom workflow methods.
- [docs/api.md](docs/api.md) — autogenerated API reference.

**Developer reference** — engine concepts subpackage by subpackage:

- [docs/developer/README.md](docs/developer/README.md) — index for the developer documentation.
- [docs/developer/core/workflow-structure.md](docs/developer/core/workflow-structure.md) — how a workflow is organised on disk.
- [docs/developer/core/variables.md](docs/developer/core/variables.md) — `inputs:` / `outputs:` sections, host-facing helpers.
- [docs/developer/core/runtime.md](docs/developer/core/runtime.md) — `Runtime`, `Runner`, `RunResult`, caller-supplied context.
- [docs/developer/core/execution-context.md](docs/developer/core/execution-context.md) — the sandboxed `Context`, trust posture, custom Jinja2 filters.
- [docs/developer/core/methods.md](docs/developer/core/methods.md) — method registry, dispatchers, plugin authoring.
- [docs/developer/core/flow.md](docs/developer/core/flow.md) — `Block` variants and dispatch.
- [docs/developer/core/dsl.md](docs/developer/core/dsl.md) — `Value` / `Property`, `ContextBase`, executables.
- [docs/developer/core/stdlib.md](docs/developer/core/stdlib.md) — built-in actions, evaluators, predicates, iterables.

**Internals** — for engine maintainers and tool builders:

- [docs/developer/internals/overview.md](docs/developer/internals/overview.md) — source-location tracking, observation, and control as three interlocking pieces.
- [docs/developer/internals/dsl.md](docs/developer/internals/dsl.md) — `MetadataMixin`, `Source` / `StepMetadata`, the `_meta` field.
- [docs/developer/internals/yaml-loader.md](docs/developer/internals/yaml-loader.md) — the three loader classes, `subdir_dir` / `current_file` invariants, `!var` / `!include` / `!module` implementation.
- [docs/developer/internals/execution-observing.md](docs/developer/internals/execution-observing.md) — `ObservableContext`, the event hierarchy, built-in observers, the `ExecutionController`.
- [docs/developer/internals/debug-command.md](docs/developer/internals/debug-command.md) — how to build a step-by-step debugger on top of the controller + observer pair.

---

## Dependencies

Third-party:

- [pydantic](https://docs.pydantic.dev/) (>=2.0) — model validation
- [PyYAML](https://pyyaml.org/) (>=6.0) and [pyyaml-include](https://pypi.org/project/pyyaml-include/) (>=2.0) — YAML parsing with `!include`
- [Jinja2](https://palletsprojects.com/p/jinja/) (>=3.0) — `{{ var }}` interpolation in `Property[str]` fields (sandboxed via `SandboxedEnvironment` in the default `Context`)
- [simpleeval](https://github.com/danthedeckie/simpleeval) (>=1.0) — sandboxed expression evaluator for `- evaluator: evaluate`
- [NumPy](https://numpy.org/) (>=1.24) — array handling via `vcti-array-tree`
- [pandas](https://pandas.pydata.org/) (>=2.0) — CSV reading in the `read_file` evaluator and DataFrame iteration in `list_items`

VCollab platform packages:

- [vcti-array-tree](https://pypi.org/project/vcti-array-tree/) — hierarchical array data
- [vcti-config](https://pypi.org/project/vcti-config/) — environment-variable decoding (`get_env` evaluator)
- [vcti-data-scope](https://pypi.org/project/vcti-data-scope/) — `DataScope` / `PathsGroup` types used by the data-scope actions
- [vcti-enum](https://pypi.org/project/vcti-enum/) — value-generated enums + Pydantic helpers
- [vcti-error](https://pypi.org/project/vcti-error/) — error codes
- [vcti-fileloader](https://pypi.org/project/vcti-fileloader/) — pluggable loader registry
- [vcti-logging](https://pypi.org/project/vcti-logging/) — logger
- [vcti-lookup](https://pypi.org/project/vcti-lookup/) — `Rule`-based attribute filtering
- [vcti-path](https://pypi.org/project/vcti-path/) — safe path & filename validation
- [vcti-path-format](https://pypi.org/project/vcti-path-format/), [vcti-path-format-descriptors](https://pypi.org/project/vcti-path-format-descriptors/) — file-format identification
- [vcti-path-tree](https://pypi.org/project/vcti-path-tree/) — directory tree scanning

Optional extras:

- `vcti-ywf[hdf5]` — pulls in [vcti-fileloader-hdf5](https://pypi.org/project/vcti-fileloader-hdf5/), which `Runner` auto-registers if installed.
