Imports:
  - Types:
      - compile_flow
    Usages:
      - compile-flow
      - parse-dsl
      - serialize-flow
    From: goga/pipeline/compiler
  - Types:
      - run_flow
    Usages:
      - run-flow
    From: goga/afm
  - Types:
      - ensure_in_docker
    Usages:
      - ensure-in-docker
    From: goga/docker
  - Types:
      - parse_workflow
      - WorkflowDocument
    Usages:
      - parse-workflow
    From: goga/pipeline/workflow

Usages:
  conventions: .goga/usages/conventions.md
  argparse: |
    Use the standard library argparse module for in-container CLI parsing.
    Two subcommands: list (no args) and run NAME --port PORT.
  cli_entrypoint: |
    The in-container CLI is launched through the package runpy entrypoint
    (python -m goga.pipeline). The package __main__ module MUST stay a thin
    wrapper: it imports `pipeline_cli` from the local cli module and calls
    it with process argv under the standard __main__ guard. The
    `pipeline_cli` implementation itself MUST live in the cli module —
    never in the __main__ module — so that importing the goga.pipeline
    package does not pull __main__ into sys.modules and trigger a runpy
    RuntimeWarning about __main__ being pre-imported. The single piece of
    logic permitted in __main__.py besides the runpy delegation is the
    in-container docker guard (`ensure_in_docker`, per the
    `ensure-in-docker` practice): it is invoked as the very first
    statement of the __main__ guard block, before `pipeline_cli` is called,
    so host-side invocations of python -m goga.pipeline fail loudly before
    any pipeline work. The guard does NOT move into `pipeline_cli` itself
    — `pipeline_cli` stays a pure parse-and-dispatch routine whose
    contract is unchanged by the guard. The guard invocation MUST be
    covered by tests for both branches: the success path (GOGA_DOCKER=1)
    proceeds to `pipeline_cli`, and the refusal path (marker unset or not
    "1") writes to stderr and exits with code 1 before `pipeline_cli` is
    reached.
  default_prompts: |
    The four default agent prompt files ship inside the installed goga
    package at
    goga/assets/afm/prompts/{planning,implementation,review,summary}.md.
    The directory contains exactly four files; the file stem matches the
    fixed agent key. Resolution from the installed package location is an
    implementation detail — the consumer may use any standard mechanism
    (e.g. importlib.resources or Path(__file__) composition) that yields
    the absolute path to the package's goga/assets/afm/prompts/
    directory at runtime. All four files are expected to exist in a
    properly installed image — when a default is missing AND no inline
    override is supplied for that key, materialization fails with a
    readable error before launch.

Annotations: |
  The `conventions` practice is used for:
  - Working with the codebase
  - Organizing the REPL development cycle
  - Debugging and testing
  - Organizing the test infrastructure
  - Understanding the general principles and rules of development and testing in the project

  This cell owns the entire pipeline workflow: discovery of *.yml pipeline
  files across the project and user pipeline directories, the pipeline-file
  entity model, run coordination, and the in-container CLI entrypoint
  pipeline_cli. It delegates subprocess execution to `run_flow` from
  goga/afm (per the `run-flow` practice) — afm run is invoked inside the
  container via `run_flow`, never directly from this cell.

  Use the standard library dataclasses module (NOT pydantic) for
  `PipelineEntry`, per project convention — pydantic is treated as tech debt.
  Use the standard library enum module for `PipelineSource` (str-backed).
  Use the `argparse` practice for the in-container CLI entrypoint.
  Use the `cli_entrypoint` practice to keep the __main__ module a thin
  wrapper and to ensure `pipeline_cli` is defined in the cli module — never
  in __main__ — so that the python -m goga.pipeline invocation does not emit
  a runpy RuntimeWarning about __main__ being pre-imported into sys.modules.
  Use pathlib.Path for directory and path resolution.

  The pipeline directories are <cwd>/.goga/pipelines/ (project-level,
  user-authored) and ~/.goga/pipelines/ (user-level, populated by
  goga connect). On a name conflict, the project source wins.

  The workflow directory is <cwd>/.goga/workflows/ (project-level,
  user-authored). Workflows are an OPTIONAL extension layer applied during
  pipeline compilation. Workflow resolution is environment-driven and
  happens INSIDE the container in `run_pipeline` (per the `parse-workflow`
  practice): GOGA_WORKFLOW_DISABLED=1 forces no workflow;
  GOGA_WORKFLOW_NAME=<wf-name> overrides the workflow name (file path
  resolves to <cwd>/.goga/workflows/<wf-name>.yml); when neither env var is
  set, `run_pipeline` falls back to the basename of the pipeline name
  (<cwd>/.goga/workflows/<pipeline_name>.yml) — silent miss if that file
  does not exist (no error, workflow = None). When a workflow path resolves
  and exists, `run_pipeline` calls `parse_workflow` to obtain a
  `WorkflowDocument`, which is passed to `compile_flow` as the optional
  workflow argument.

  This cell runs inside the goga Docker image when invoked through
  python -m goga.pipeline. The host-side launcher lives in
  goga/commands/pipeline (docker runtime boundary — no Python Imports).

  Compilation step: `run_pipeline` invokes `compile_flow` (per the
  `compile-flow` practice) to transform the discovered pipeline-file from
  goga DSL into an afm flow-file at runtime, inside the container. When a
  workflow is resolved, the parsed `WorkflowDocument` is forwarded to
  `compile_flow` so the compiler can reconstruct the body per the workflow
  instructions before serializing the flow-file. The output path is the
  flow.yml file inside the directory pointed to by the AFM_DIR environment
  variable. Use the `parse-dsl` and `serialize-flow` practices to understand
  the intermediate stages of the compilation pipeline when a lower-level
  view is required.

  Prompt materialization step: after `compile_flow`, `run_pipeline`
  materializes the four default agent prompt files from the installed goga
  package (goga/assets/afm/prompts/, per the `default_prompts` practice)
  into the runtime directory at <AFM_DIR>/prompts/, then applies a per-key
  override — for each non-None agents field of the documents tuple
  returned by `compile_flow` (per the `compile-flow` practice), the
  corresponding <name>.md file is
  overwritten with the inline prompt text. The override is a full
  replacement (no merge, no concatenation). All four files are written
  unconditionally — when a default is missing from the package AND no
  inline override is supplied for that key, materialization fails with a
  readable error before `run_flow` is invoked. The compiled flow-file is
  unaffected — inline prompts are a goga-side artifact, not part of the
  afm flow-file contract.

---

"PipelineEntry(name: str, source: PipelineSource)":
  location: pipeline_entry.py
  annotations: |
    Describe a single pipeline-file discovered by `list_pipelines`: its `name`
    and where it comes from (`source`).

    `name`: pipeline name without extension (e.g. "deploy"); the .yml extension
            is implied and never stored here
    `source`: origin of the pipeline — `PipelineSource` enum value

    Build the data model with the standard library dataclasses module (NOT
    pydantic, per project convention; pydantic is treated as tech debt). Use
    @dataclass(kw_only=True) and validate `name` at construction time
    (raising ValueError on invalid input).

    Requirements:
    - Use @dataclass(kw_only=True) (per `conventions`)
    - `name` must not contain path separators ("/", "\\") or the .yml extension
    - `name` must not be empty

  properties:
    "name -> str": |
      Pipeline name without extension.
    "source -> PipelineSource": |
      Origin of the pipeline: `PipelineSource`.PROJECT for project-level
      <cwd>/.goga/pipelines/, `PipelineSource`.USER for user-level ~/.goga/pipelines/.

"PipelineSource()":
  location: pipeline_entry.py
  annotations: |
    str-backed Enum declaring the origin of a pipeline-file.

    Modeled via the standard library enum module; str-mixin so values
    serialize as plain strings (per the `conventions` practice).

  properties:
    "PROJECT = \"project\"": |
      Origin = project-level <cwd>/.goga/pipelines/.
    "USER = \"user\"": |
      Origin = user-level ~/.goga/pipelines/.

"list_pipelines(project_dir: Path, user_dir: Path) -> entries: list[PipelineEntry]":
  location: list_pipelines.py
  annotations: |
    Discover pipeline files across two source directories and return them as
    `entries`: a list of `PipelineEntry`-s.

    `project_dir`: project-level pipelines directory (typically <cwd>/.goga/pipelines/)
    `user_dir`: user-level pipelines directory (typically ~/.goga/pipelines/)
    `entries`: list of `PipelineEntry`-s, one per unique pipeline name

    Algorithm:
    1. Scan flat *.yml files (non-recursive) in `project_dir`; for each valid
       stem, record a `PipelineEntry` with the name set to the stem and source
       set to `PipelineSource`.PROJECT. Skip stems that fail `PipelineEntry`
       validation (invalid chars, .yml suffix, empty) silently.
    2. Scan flat *.yml files (non-recursive) in `user_dir`; for each valid stem
       not already present from step 1, record a `PipelineEntry` with source
       set to `PipelineSource`.USER.
    3. Return the combined `entries` list.

    Apply the `conventions` practice for the filesystem scanning code
    (relative imports, logging, docstring style).

    Requirements:
    - Scan only the top level of each directory — do not descend into subdirectories
    - Drop the .yml extension when forming the entry name
    - A missing source directory is treated as empty (no error)
    - Skip stems that fail `PipelineEntry` validation silently — they are not pipelines

    Constraints:
    - Do not parse or validate the contents of pipeline files — only their names
      matter here
    - When a name exists in both sources, the project source wins (no duplicate
      entries)

"run_pipeline(name: str, project_dir: Path, user_dir: Path, port: int) -> exit_code: int":
  location: run_pipeline.py
  annotations: |
    Resolve a pipeline name to an absolute file path via `list_pipelines`,
    resolve an optional workflow-file per the workflow environment contract
    (see the cell-level annotation) and parse it via `parse_workflow` when
    applicable, compile the pipeline-file (optionally extended by the
    workflow) from goga DSL into an afm flow-file at runtime via
    `compile_flow`, materialize the four agent prompt files (defaults plus
    inline overrides) into the runtime prompts directory, then launch afm
    through `run_flow`. This is the run coordination routine — it performs
    discovery, workflow resolution, path resolution, compilation, and prompt
    materialization; the actual subprocess execution lives in `run_flow`.

    `name`: pipeline name without extension
    `project_dir`: project-level pipelines directory (same meaning as in `list_pipelines`)
    `user_dir`: user-level pipelines directory (same meaning as in `list_pipelines`)
    `port`: TCP port forwarded to afm run --port via `run_flow`
            (allocated by the host-side caller)
    `exit_code`: 0 on success, non-zero on error (missing pipeline, missing
                 binary, afm failure, structural DSL error, workflow parse
                 error, materialization error). 127 means afm is not on PATH
                 inside the container.

    Algorithm:
    1. Discover pipelines via `list_pipelines` and find the `PipelineEntry`
       whose name matches
    2. If no match — report that the pipeline is missing and return a
       non-zero exit code
    3. Build the absolute pipeline path from the matching entry's source
       directory and the pipeline name
    4. Resolve the in-container runtime directory from the AFM_DIR
       environment variable. If AFM_DIR is not set — raise an error with
       the readable message "AFM_DIR not set". Resolve the value to an
       absolute path (via pathlib.Path.resolve) to guarantee the directory
       is absolute regardless of whether the env value is relative
    5. Compose the output flow path inside that directory
    6. Resolve an optional workflow per the workflow environment contract:
       a. Read GOGA_WORKFLOW_DISABLED from the environment. When its value
          is "1" — set workflow = None (workflow disabled by the host
          launcher via --no-workflow)
       b. Otherwise determine the workflow-file path:
          - When GOGA_WORKFLOW_NAME is set in the environment — compose
            workflow_path as Path.cwd() / ".goga" / "workflows" /
            <GOGA_WORKFLOW_NAME>.yml (= /workspace/.goga/workflows/
            <wf-name>.yml inside the container). The CWD-based path is
            mandatory: project_dir itself is /workspace/.goga/pipelines,
            so project_dir.parent is /workspace/.goga and a parent-based
            composition would produce a double .goga. Workflows are
            project-only by design.
          - Otherwise (GOGA_WORKFLOW_NAME not set) — compose workflow_path
            via the basename fallback: Path.cwd() / ".goga" /
            "workflows" / <name>.yml (the same basename as the pipeline
            being run)
       c. When workflow_path exists — call `parse_workflow`
          (per the `parse-workflow` practice) on workflow_path to obtain a
          `WorkflowDocument`. Structural errors from `parse_workflow`
          propagate unchanged with their readable messages
       d. When workflow_path does not exist — set workflow = None (silent
          miss; this is the opt-in path — the absence of a workflow-file
          is not an error)
    7. Compile the pipeline-file via `compile_flow` (per the `compile-flow`
       practice) with the resolved workflow as the optional workflow
       argument — receive the documents tuple as the return value
       (per the `compile-flow` practice). When workflow is None, no
       workflow is applied. Structural DSL errors propagate unchanged;
       structural workflow errors propagate unchanged from `parse_workflow`
    8. Materialize agent prompts inline (atomic: validate-all, then wipe,
       then write):
       a. Resolve the default prompts directory in the installed goga
          package (per the `default_prompts` practice) —
          goga/assets/afm/prompts/
       b. Validate all four fixed agent keys (planning, implementation,
          review, summary): for each key, either
          the documents tuple returned by `compile_flow` carries a non-None
          inline override for that key (per the `compile-flow` practice), or
          the corresponding default prompt file
          (defaults_dir / "<key>.md") exists. A key with neither override
          nor default is a fatal error raised BEFORE the directory is wiped
          or any prompt file is written — this is the atomicity guarantee
          (no partial state on disk)
       c. After step 8b succeeds, reset <AFM_DIR>/prompts/ to a clean
          state (rmtree + mkdir) so the directory contains exactly four
          files after step 8 succeeds, regardless of any leftover files
          from previous runs or manual edits
       d. For each of the four fixed agent keys: write the inline prompt
          override to <AFM_DIR>/prompts/<key>.md when the documents tuple
          (returned by `compile_flow` per the `compile-flow` practice)
          carries one for that key; otherwise copy the corresponding
          (already-validated) default prompt from the installed package.
          By construction (step 8b) every source exists — no raise is
          possible in this step
       e. After step 8d succeeds, <AFM_DIR>/prompts/ contains exactly
          four files (planning.md, implementation.md, review.md,
          summary.md)
    9. Launch afm via `run_flow` (per the `run-flow` practice) with the
       compiled flow-file path and `port`
    10. Return the exit code returned by `run_flow`

    Apply `conventions` for error-handling style and docstring formatting.
    Apply `parse-workflow` for the contract of `parse_workflow`
    and the resulting `WorkflowDocument`.
    Apply `compile-flow` for the compilation step contract (including the
    new optional workflow parameter and the documents-tuple return value).
    Apply `default_prompts` for resolving the path to the four packaged
    default prompt files.
    Apply `run-flow` for the subprocess launch contract.

    Requirements:
    - Always pass the absolute pipeline path to `compile_flow` — never the
      bare name
    - Always pass the absolute compiled flow path to `run_flow` — never the
      bare name or the DSL path
    - Always forward `port` to `run_flow` — never omit it
    - Read AFM_DIR directly from the process environment
    - Resolve AFM_DIR to an absolute path (via pathlib.Path.resolve) before
      composing flow_path (so a relative env value still yields an absolute
      flow_path that satisfies `compile_flow`'s absolute-paths precondition)
    - Apply the `run-flow` practice's error-handling rules verbatim (as
      propagated through `run_flow`)
    - Apply the `compile-flow` practice's contract verbatim
    - Always accept the return value of `compile_flow` (the documents
      tuple); never re-invoke the parse-dsl routine to obtain inline
      prompt overrides
    - Read inline prompt overrides exclusively from
      the documents tuple returned by `compile_flow` (per the `compile-flow`
      practice — the compiled flow-file carries no agents data)
    - <AFM_DIR>/prompts/ must contain exactly four files after step 8
      succeeds; partial state (fewer files) is an error condition that must
      surface before `run_flow` is invoked. Wipe the directory clean in
      step 8c (after validation in 8b) so leftover files from a previous
      run or manual edits do not accumulate — idempotent end state: the
      same four files after every run
    - Default prompt files are read from the installed package location
      (per `default_prompts`); the source path must NOT be derived from
      AFM_DIR, CWD, or any environment variable
    - The override is a full file replacement — no merge, no concatenation
      with the default prompt text
    - A missing default prompt (file absent in the package) for a key with
      no inline override is a fatal error: raise with a readable message
      naming the key in step 8b (validation phase), BEFORE the directory
      is wiped or any prompt file is written. <AFM_DIR>/prompts/ remains in
      its previous state on error — atomicity is guaranteed by validate-all
      before wipe
    - A missing default prompt for a key WITH an inline override is NOT an
      error — the file is written from the override; the default's absence
      is ignored for that key
    - Read GOGA_WORKFLOW_DISABLED and GOGA_WORKFLOW_NAME directly from the
      process environment; the host-side launcher sets them via the env-file
    - Workflow path resolution is CWD-based:
      Path.cwd() / ".goga" / "workflows" / "<name>.yml" — the
      project_dir.parent IS NOT the project root; project_dir itself is
      /workspace/.goga/pipelines, so Path.cwd() (= /workspace in-container)
      is the project root. Workflows are project-only.
    - A non-existent workflow-file is a silent miss (workflow = None), NOT
      an error — the host-side launcher performs explicit --workflow
      existence validation; the basename fallback is opt-in
    - GOGA_WORKFLOW_DISABLED="1" takes precedence over GOGA_WORKFLOW_NAME —
      when both are set, workflow = None

    Constraints:
    - Do not invoke afm directly outside `run_flow`
    - Do not invoke the compiler outside `compile_flow`
    - Do not allocate the port — the caller allocates it
    - Do not copy pipeline files into any project-level directory
    - Do not modify or parse pipeline-file contents directly — that is
      `compile_flow`'s job
    - Do not accept relative `project_dir` or `user_dir` — both must already
      be absolute when passed in
    - Do not mask or wrap exceptions from `compile_flow` — structural DSL
      errors propagate with their readable messages
    - Do not write prompts/ inside the project directory or /workspace —
      always write to <AFM_DIR>/prompts/ (the in-container persistent state
      directory)
    - Do not write inline prompt overrides into the compiled flow-file —
      they are a goga-side artifact, not part of the afm flow-file contract
    - Do not silently skip a missing default when no inline override is
      supplied for that key
    - Do not perform any prompt-related work outside step 8 — the host
      launcher writes prompts_dir into the afm config.yaml tmpfile but
      does not know whether the pipeline-file contains an agents block
    - Do not mutate the documents tuple returned by `compile_flow` — read
      the inline prompt overrides from it as-is
    - Do not invoke the workflow parser outside `parse_workflow`
    - Do not modify or parse workflow-file contents directly — that is
      `parse_workflow`'s job
    - Do not treat a missing workflow-file as an error when neither
      GOGA_WORKFLOW_NAME nor GOGA_WORKFLOW_DISABLED="1" is set — the
      basename fallback is opt-in
    - Do not validate the workflow-file path on the host — workflow
      resolution happens inside the container only; the host-side launcher
      performs explicit --workflow existence validation before launch
      when the user passes --workflow

"pipeline_cli(argv: list[str]) -> exit_code: int":
  location: cli.py
  annotations: |
    In-container CLI implementation for python -m goga.pipeline. Parses argv
    via argparse and dispatches to `list_pipelines` (discovery) or
    `run_pipeline` (run). This routine is invoked by the host-side docker
    launcher in goga/commands/pipeline through the runpy entrypoint in
    __main__.py — never imported by Python from the host side (runtime
    docker boundary, no Imports).

    `argv`: argument list (typically the process argv minus the program name)
    `exit_code`: 0 on success, 2 on argparse error, non-zero on pipeline failure
                 (propagated from `run_pipeline` / `list_pipelines`)

    Algorithm:
    1. Build an argparse parser with two subcommands: list and run
       - list takes no positional arguments
       - run takes a required name positional argument
         and a required --port PORT integer option
    2. Parse argv; on argparse error — exit code 2 (argparse default)
    3. Resolve the project pipelines directory (project working directory's
       .goga/pipelines/) and the user pipelines directory (user home's
       .goga/pipelines/)
    4. Dispatch:
       - list → call `list_pipelines`(project_dir, user_dir), print
         "Available pipelines:" header followed by one entry per line
         (project source suffixed with " (project)"), return 0
       - run NAME --port PORT → call `run_pipeline`(NAME, project_dir,
         user_dir, PORT) and return its exit code (delegates name resolution,
         compilation, and the .yml extension to `run_pipeline`)
    5. Return the resulting exit code

    Apply the `conventions` practice for docstring style and intra-package imports.
    Apply the `argparse` practice for parser construction.
    Apply the `cli_entrypoint` practice: this routine MUST be defined in the
    cli module, and __main__ MUST only delegate to it — never define
    `pipeline_cli` inside __main__.

    Requirements:
    - run subcommand must accept --port PORT as an integer (required)
    - list subcommand takes no arguments
    - Echo the header BEFORE the list in list mode (predictable UX)

    Constraints:
    - Do not allocate a port inside this CLI — --port is required and supplied
      by the host-side launcher
    - Do not import this cell from the host side — invoke via docker only
      (docker run ... python -m goga.pipeline {list,run})

---

Author: Goga
CreatedAt: 29/06/26
Description: |
  Cell that owns the entire pipeline workflow: discovery of *.yml pipeline
  files across the project and user pipeline directories, the pipeline-file
  entity model, run coordination (including optional workflow resolution),
  and the in-container CLI entrypoint pipeline_cli.
