Metadata-Version: 2.4
Name: pulp-workflow
Version: 0.1.0
Summary: Pulp plugin for managing task schedules via CRUD API
License: MIT License
        
        Copyright (c) 2026 David Davis
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: Homepage, https://pulpproject.org
Project-URL: Repository, https://github.com/daviddavis/pulp_workflow
Project-URL: Bug Tracker, https://github.com/daviddavis/pulp_workflow/issues
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: POSIX :: Linux
Classifier: Framework :: Django
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pulpcore<3.115,>=3.113.0
Dynamic: license-file

# pulp-workflow

> **Warning:** This is a community plugin and is not officially supported. Scheduling tasks incorrectly can cause serious issues in your Pulp instance. Always test in a development environment first before applying changes to production.

A Pulp plugin that introduces the `Workflow` model. Workflows build on top of
tasks in Pulp allowing users to:
* Schedule tasks to run at any given time
* Run sequences of tasks in a specific order
* Set up callback services to run on workflow lifecycle events (e.g. running,
completed, failed, canceled, finished)

A `Workflow` owns one or more `WorkflowTask` rows. Each task records the
`task_name`, `task_args`, `task_kwargs`, and any `reserved_resources` to use
when dispatching it. Workflows are immutable after creation: to change a
workflow, cancel it (if it has not yet started) and create a new one.

## Endpoints

| Method | URL | Description |
|--------|-----|-------------|
| GET | `/pulp/api/v3/workflow/workflows/` | List workflows |
| POST | `/pulp/api/v3/workflow/workflows/` | Create a workflow (with tasks) |
| GET | `/pulp/api/v3/workflow/workflows/<pk>/` | Retrieve a workflow |
| PATCH | `/pulp/api/v3/workflow/workflows/<pk>/` | Cancel a workflow (body: `{"state": "canceled"}`). Works whether the workflow is `waiting` or `running`; returns 409 only if the workflow is already in a terminal state. Only `"canceled"` is accepted as the target state. |

## How execution works

When a workflow is created, TaskSchedule dispatches a single `execute_workflow`
task. Rather than looping inside one long-running task (which would pin a
worker for the entire duration of the pipeline), `execute_workflow` runs one
step at a time and re-dispatches itself for the next step. Each invocation
either transitions the workflow to `running` (on the first step) or inspects
the previous step's child task and fails the workflow if it did not complete.
If there are no more `WorkflowTask` rows at the next index, the workflow is
marked `completed`.

Sequencing relies on pulpcore's tasking locks rather than polling. Every
invocation dispatches the child task with a **shared** lock on the workflow's
resource string (`pulp_workflow:workflow:<pk>`) and then dispatches the next
`execute_workflow` continuation with an **exclusive** lock on the same
resource. Because the exclusive lock cannot be granted while the shared lock
is held, the continuation is guaranteed to wait until the child task ends —
without blocking a worker on a polling loop. This keeps concurrent workflows
from deadlocking when their count meets or exceeds the worker count.

The diagram below shows two consecutive tasks. `S` denotes a shared lock and
`X` denotes an exclusive lock on the workflow resource.

```mermaid
sequenceDiagram
    autonumber
    participant TaskSchedule
    participant EW0 as execute_workflow(0)
    participant T0 as Child Task 0
    participant EW1 as execute_workflow(1)
    participant T1 as Child Task 1
    participant EW2 as execute_workflow(2)

    TaskSchedule->>EW0: dispatch (X on workflow)
    activate EW0
    EW0->>EW0: state = RUNNING
    EW0->>T0: dispatch (S on workflow)
    EW0->>EW1: dispatch (X on workflow) — queued
    deactivate EW0

    activate T0
    Note over EW1: blocked: X waits for S to release
    T0-->>T0: run task body
    T0->>T0: state = COMPLETED
    deactivate T0

    activate EW1
    EW1->>EW1: verify Task 0 COMPLETED
    EW1->>T1: dispatch (S on workflow)
    EW1->>EW2: dispatch (X on workflow) — queued
    deactivate EW1

    activate T1
    Note over EW2: blocked until T1 finishes
    T1-->>T1: run task body
    T1->>T1: state = COMPLETED
    deactivate T1

    activate EW2
    EW2->>EW2: verify Task 1 COMPLETED
    EW2->>EW2: no task at index 2 → state = COMPLETED
    deactivate EW2
```

If any child task ends in a non-`completed` state, or if dispatching a child
raises, the next `execute_workflow` invocation records the failure on the
workflow (`error` field, including the child's traceback when available),
transitions the workflow to `failed`, and stops the chain.

## Task groups

Every workflow is backed by a pulpcore `TaskGroup`. On creation, the workflow
allocates a `TaskGroup(description="Workflow: <name>")` in the workflow's
domain and links it via the `task_group` field on the Workflow resource. The
dispatched child tasks and the `execute_workflow` continuations are members
of that group. The initial `execute_workflow` task itself is dispatched by
the `TaskSchedule` created at workflow-create time and is *not* a member of
the group. Membership means:

- `GET /pulp/api/v3/task-groups/<pk>/` lists every task the workflow has
  spawned in one place.
- `GET /pulp/api/v3/tasks/?task_group=<pk>` filters tasks to that workflow.
- Existing client tooling (`monitor_task_group`,
  `TaskGroupOperationResponse`) works against workflows the same way it does
  for replication and pulp-import.
- A child task can discover that it is part of a workflow via
  `TaskGroup.current()` without `pulp_workflow` having to plumb that context
  through itself.

The group's `all_tasks_dispatched` flag is `False` while the workflow is
running and flipped to `True` exactly once the workflow reaches a terminal
state (`completed`, `failed`, or `canceled`).

## Cancellation

A workflow can be canceled either by `PATCH`ing the workflow with
`{"state": "canceled"}` or by canceling its backing `TaskGroup` directly.
Both paths converge on the same terminal state: `Workflow.state = canceled`,
`finished_at` set, `TaskGroup.all_tasks_dispatched = True`, and any
in-flight or queued child tasks (including the `execute_workflow`
continuation) canceled. PATCH on an already-terminal workflow returns 409.

The `TaskGroup` cancel path is bridged back to the workflow row via a
`post_save` receiver on `TaskGroup`, so cancellation initiated outside the
workflow viewset still terminates the `Workflow` correctly.

```mermaid
flowchart TD
    A[PATCH /pulp/api/v3/workflow/workflows/&lt;pk&gt;/<br/>state: canceled] --> B[WorkflowViewSet]
    B --> C[Workflow.state = canceled<br/>finished_at = now]
    C --> D[cancel_task_group<br/>on commit]

    E[POST /pulp/api/v3/task-groups/&lt;pk&gt;/cancel/] --> F[pulpcore cancels<br/>group's tasks]
    F --> G[TaskGroup.all_tasks_dispatched = True<br/>TaskGroup.save]

    D --> G
    G --> H[post_save receiver<br/>on TaskGroup]
    H --> I{Workflow already<br/>terminal?}
    I -- yes --> J[no-op]
    I -- no --> K[Workflow.state = canceled<br/>finished_at = now]

    classDef terminal fill:#e8f5e9,stroke:#2e7d32;
    class C,K terminal;
```
