Metadata-Version: 2.4
Name: orcca
Version: 0.1.1
Summary: Manifest-driven clinical workflow orchestration framework
Author: Athenkosi Nkonyeni
License-Expression: MIT
Keywords: clinical-trials,workflow,orchestration,manifest,data-pipeline
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.11
Description-Content-Type: text/markdown
Requires-Dist: PyYAML>=6.0
Requires-Dist: jsonschema>=4.0
Requires-Dist: linkml>=1.8
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"

# Orcca

Orcca is a manifest-driven orchestration framework for clinical programming workflows.
It is designed for teams that need reproducible execution, explicit dependency modeling,
and clear run reporting.

## Current State

- Package version: `0.1.0` (alpha)
- Python: `>=3.11`
- Core dependencies: `PyYAML`, `jsonschema`, `linkml`
- Entry point: `orcca` CLI (`orcca.cli:main`)

This project is currently optimized for practical workflow execution and validation with a
strong traceability model, not for broad general-purpose scheduling.

## What Orcca Does

Orcca provides:

- declarative workflow modeling in YAML
- manifest include/config resolution
- LinkML + business-rule validation
- executable task orchestration with dependency-aware scheduling
- runtime preflight for package-manager and container environments
- run reporting (`run_report.html`) and task logs
- validation task execution for independent double programming (IDP)
- `blinding_mode`-based blinded/unblinded workflow control

## Core Architecture

Main modules:

- `orcca/resolver.py`: include/config merge and placeholder interpolation
- `orcca/validator.py`: LinkML validation + business rules
- `orcca/executor.py`: task execution engine + runtime preflight
- `orcca/workflow_runner.py`: end-to-end orchestration pipeline and run reports
- `orcca/runtime_launchers.py`: built-in + custom launcher resolution
- `orcca/dagger.py`: generated-artifact DAG utilities
- `orcca/cli.py`: user-facing commands

Packaged schema/rules:

- `orcca/resources/schema/flow-linkml.yaml`
- `orcca/resources/rules/business-rules.yaml`

## Manifest Resolution Semantics

Orcca resolves manifests in this order:

1. Resolve `includes` recursively in depth-first, declaration order.
2. Merge include payloads into a single root payload.
3. Merge the root manifest payload on top.
4. Resolve `configs` in declaration order.
5. Interpolate `${config['output_dir']}`-style placeholders after merging is complete.
6. Placeholder substitutions must resolve to scalar values, not lists or dictionaries.

Merge behavior:

- dict values replace completely
- list-valued keys inside dicts: merged with the same list rules
- lists of scalars: append + dedupe while preserving first-seen order
- lists of `{id: ...}` entities: overwrite by `id`
- scalars/type mismatch: latest wins

Config files use the same merge rules as manifest root slots, so later files can override
earlier ones without changing the merge model.

Declarative slots such as `id`, `name`, `description`, `notes`, `envvar_name`, and
`param_name` must be written as explicit literals and may not use
`${config['output_dir']}`-style interpolation.
Placeholder substitutions must resolve to scalar values, not lists or dictionaries.

This matters for override design: unblinded manifests should include base first, then
override files that only change allowed slots.

## Workflow Model

Primary slots:

- `study`
- `standards`
- `artifacts`
- `artifact_groups`
- `runtimes`
- `tasks`
- `validations`
- `deliveries`
- `execution`
- `blinding_mode`

Important modeling rules:

- `tasks.outputs` must reference declared artifacts.
- `tasks.inputs` can reference declared artifacts or transient IDs used internally.
- `required_tasks` provides explicit execution sequencing independent of artifact lineage.

## Blinded/Unblinded Mode

Run mode:

```yaml
blinding_mode: blinded # or unblinded
```

Artifact variants:

- `variant` is optional and only needed for paired blinded/unblinded artifacts.

```yaml
artifacts:
  - id: rand_map
    variant: blinded
    name: Randomization Map
    type: lookup_dataset
    origin: external
    path: outputs/rand_map_blinded.csv
```

Unblinded pattern:

```yaml
includes:
  - workflow_blinded.yaml
  - manifests/artifacts_unblinded.yaml

blinding_mode: unblinded
```

Declarative slots must remain literal and may not use quoted config interpolation placeholders.

## CLI

Top-level commands:

- `orcca validate <manifest.yaml>`
- `orcca plan <manifest.yaml>`
- `orcca run <manifest.yaml> [--project-root <path>]`
- `orcca env-cache list|prune [--all]`
- `orcca init [path]`

Useful run flags:

- `--task <id>` (repeatable)
- `--artifact <id>` (repeatable)
- `--run-validations`
- `--validation-id <id>` (repeatable; requires `--run-validations`)
- `--no-execute`
- `--no-env-cache`
- `--refresh-env`

Run root behavior:

- `--project-root` controls execution root and run artifact root
- runs are stored under `<project-root>/.orcca/runs`

## Execution and Reports

`run_workflow(...)` pipeline:

1. resolve manifest
2. validate manifest
3. execute tasks (optional)
4. package deliveries (when configured)
5. optionally execute selected validation tasks
6. write run artifacts

Per-run artifacts:

- `resolved_manifest.yaml`
- `run_report.html`
- task logs under `logs/`
- task context payloads under `task-context/`

## Runtime Model

Built-in launchers:

- `python`
- `rscript`
- `julia`
- `container`

Custom launchers:

- auto-loaded from `launchers/*.py`
- optional `launchers/register.py`
- launcher file rule: filename stem must match exported function name

Package-manager environments:

- cached under `.orcca/cache/environments/`
- managers: `pip`, `venv`, `renv`, `julia_pkg` (+ explicit restore command for others)

Container environments:

- source types: `registry`, `archive`, `build`
- archive load and local build workflows supported

## Validation and IDP

Validation uses:

- LinkML schema validation
- business-rule validation
- cross-reference checks and dependency checks

IDP (`check_type: independent_double_programming`) supports:

- artifact/group targets
- explicit `validation_task`
- validation targets and validation task references
- tool metadata via `tool`

Recommended IDP pattern:

- keep `targets` for business ownership
- use `validation_task` for executable provenance
- keep validation outputs easy to inspect in reports and logs

## Examples in Repo

Reference workflows:

- `testing/e2e_python_native/workflow.yaml`
- `testing/e2e_r_native/workflow.yaml`
- `testing/e2e_julia_native/workflow.yaml`
- `testing/e2e_sdtm/workflow.yaml`

Blinding handoff example:

- `testing/blinding_handoff/README.md`

## Development

Install:

```bash
pip install -e .[dev]
```

Run tests:

```bash
pytest -q
```
