---
title: "Parser"
description: The nf-metro parser — Lark grammar, %%metro directive handling, and post-parse rewrites that insert ports and junctions.
sidebar:
  order: 8
---

import { Aside } from "@astrojs/starlight/components";

A walkthrough of how nf-metro turns `.mmd` text into a `MetroGraph`. If
you're adding a node shape, a `%%metro` directive, or a new statement
form - or just trying to understand the grammar - start here.

The entry point is `parse_metro_mermaid` in
[`src/nf_metro/parser/mermaid.py`](https://github.com/seqeralabs/nf-metro/blob/main/src/nf_metro/parser/mermaid.py);
the data model it builds is in
[`src/nf_metro/parser/model.py`](https://github.com/seqeralabs/nf-metro/blob/main/src/nf_metro/parser/model.py).
The parser package is split by job: `grammar.py` (grammar, statement types,
transformer), `directives.py` (directive parsing and dispatch), `resolve.py`
(the post-parse graph rewrites), and `mermaid.py` (the public entry point and
statement-application driver).
Parsing is the first of the three stages (**Parse -> Layout -> Render**);
the [layout pipeline](/nf-metro/dev/layout_pipeline/) takes over from the `MetroGraph`
this stage produces.

## What "parsing" means here

The input is a subset of Mermaid `graph LR` syntax plus `%%metro`
directives. Parsing reads that text and produces a `MetroGraph` of
sections, stations, edges, lines, and ports - with no coordinates yet
(those are the layout stage's job).

The work splits in two:

1. **Front-end** - recognise each line's shape (a node? an edge? a
   directive?) and pull out its pieces. The grammar recognises statement
   _shapes and boundaries_; directive _payloads_ and graph _semantics_ are
   handled by Python (see [Directives](#directives) and
   [Parse-then-resolve flow](#parse-then-resolve-flow)).
2. **Post-parse** - rewrite the raw graph into the form layout expects:
   resolve inter-section edges into ports and junctions, insert bypass
   and convergence stations, create the implicit section for loose
   nodes. These are plain functions operating on the model, covered in
   [Parse-then-resolve flow](#parse-then-resolve-flow) below.

## Input format

`.mmd` files use a subset of Mermaid `graph LR` syntax:

```metro title="example.mmd"
%%metro title: Pipeline Name
%%metro line: line_id | Display Name | #hexcolor | style

graph LR
    subgraph section_id [Section Name]
        %%metro entry: left | line1, line2
        %%metro exit: right | line1, line2
        node_id[Label]
        node_id -->|line_id| other_node
    end
    %% Inter-section edges live outside subgraphs
    node_a -->|line_id| node_b
```

- A Mermaid `subgraph` becomes a `Section`.
- A node (`node_id[Label]`) becomes a `Station`.
- An edge (`a -->|line1,line2| b`) becomes one `Edge` per line id; the
  pipe-delimited label lists the lines the edge carries. An endpoint may
  also be written with an inline shape (`a[A] -->|line1| b[B]`), which
  declares that node's label as well as the edge.
- Lines must be declared with `%%metro line:` before use; an edge with no
  line annotation, or one naming an undeclared line, is rejected as a
  semantic error (see [Leniency and error policy](#leniency-and-error-policy)).
- A primary graph direction other than `LR` is warned about
  (`_warn_if_non_lr_primary`); per-section flow is controlled by
  `%%metro direction:` and is independent of the header.

## What a grammar is, and why we use one

The naive way to read a line-oriented format is to write the
_instructions_ for recognising each line: "does this line start with
`graph`? else is it a `subgraph`? else does it contain an arrow? else
try these six node-shape patterns in this exact order...". That works,
but the order of the checks becomes load-bearing, and every new shape or
directive means finding the right slot in the chain.

A **grammar** flips this around: instead of writing the recognising
instructions, you write down _the rules of what a valid line is_ and let
a library do the recognising. nf-metro uses [`lark`](https://lark-parser.readthedocs.io/)
for this. The grammar lives as a string in `grammar.py` (`_GRAMMAR`),
and reads roughly:

```lark title="src/nf_metro/parser/grammar.py"
node:  NAME SHAPE?
edge:  NAME SHAPE? ARROW EDGELABEL? NAME SHAPE?

SHAPE: /\(\[...\]\)|\[\[...\]\]|\(\(...\)\)|\[...\]|\(...\)|\{...\}/
ARROW: /-->|---|==>/
NAME:  /[a-zA-Z_][a-zA-Z0-9_]*/
```

An edge endpoint may carry an inline shape: `x[X] -->|a| y[Y]` declares
node `x` with label "X", node `y` with label "Y", and the edge between
them, all in one line. The `SHAPE` terminal's inner pattern
(`_SHAPE_INNER`) excludes the arrow sequences so a source shape like
`[X]` stops at the arrow rather than greedily swallowing it.

You describe the shapes; lark works out how to match them. Adding a node
shape is one more alternative in the `SHAPE` rule, not a new regex
slotted into a hand-ordered chain, and the order of the rules stops
mattering for correctness.

It helps to think of it like describing the structure of a sentence
("a sentence is a subject, a verb, then an object") rather than writing
out, character by character, how to scan one.

## From text to `MetroGraph`

Three steps:

1. **Parse.** `_PARSER.parse(text)` turns the whole document into a
   parse tree using the grammar.
2. **Transform.** `_StatementTransformer` walks that tree and flattens
   it into an ordered list of **typed statements** - one small dataclass
   per source line, from a fixed union: `_GraphHeader`, `_Subgraph`,
   `_Directive`, `_Node`, `_Edge`, `_End`, `_Comment`, and `_Junk`. The
   transformer does the normalisation here, so each statement carries
   structured fields rather than raw tokens: `_Subgraph` already splits
   the section id from its display name, `_Directive` splits the body on
   the first colon into `key`/`value`, and `_Edge` carries its line ids
   as a list plus any inline endpoint labels.
3. **Drive.** `parse_metro_mermaid` iterates those statements _in source
   order_ and dispatches each by `isinstance`, applying it to the graph
   while tracking which `subgraph` it's currently inside
   (`current_section_id`). A `_Subgraph` opens a section, an `_End`
   closes it, and the nodes/edges/directives in between are attached to
   it.

Because the appliers receive structured fields, the driver stays a thin
dispatch: a `_Node` registers a station, an `_Edge` registers one edge
per line id (declaring any inline-shaped endpoints), a `_Directive` is
routed to a handler, and so on.

Keeping the driver a simple in-order loop is deliberate: source order
matters (e.g. dictionary insertion order of stations affects downstream
layout), so the grammar handles _recognising_ lines while the driver
handles _applying_ them in sequence.

### One simplification worth knowing

The model records only a node's **id and label** - never which shape it
was drawn as. So all six Mermaid shapes collapse to a single `SHAPE`
terminal plus a tiny "strip the delimiters" helper (`_shape_label`),
instead of six separate regexes that each had to be tried in the right
order.

## Directives

The `%%metro` directive _bodies_ are not described by the grammar - they
keep their own handler functions, because a grammar can't express
behaviour like "warn about this and ignore it", which several directives
need. What the grammar gives us is the directive line as a unit; the
transformer splits its body once on the first colon into a **key** and a
**value** (a `%%metro` line with no colon becomes a `_Comment` and is
ignored).

Dispatch on the key happens in `_apply_directive`. Most directives are
graph-wide and live in the `_GLOBAL_DIRECTIVE_HANDLERS` dict, keyed by **exact
name** and mapping to a `(value, graph) -> None` handler. Exact-key
lookup means handler order is irrelevant and a key that is a prefix of
another (`legend` vs `legend_combo`, `logo` vs `logo_scale`) cannot
shadow it. Three families are dispatched separately because they need
more than the value alone: `entry` / `exit` / `direction` need the
enclosing section, and the icon keys `file` / `files` / `dir` need the
key itself to choose the icon type. A key matching none of these is
ignored with a `UserWarning`.

The simple scalar / bool / enum knobs that also have a CLI flag (spacing,
gaps, `diamond_style`, scales, `width` / `height` / `animate`, ...) are not
hand-written handlers. They are declared once in `nf_metro.options`
(`LayoutOption` / `LAYOUT_OPTIONS`); `_make_layout_option_handler` generates a
directive handler from each entry (parsing via `coerce`, writing the named
`MetroGraph` field), and `nf_metro.cli` generates the matching `click` flag
from the same registry. Adding such an option means one registry entry, not a
handler plus a flag plus a docs row. `tests/test_options_parity.py` guards that
every registry option exists in both planes. The bespoke handlers below carry
grammar (fields, sections, coordinates) the generic registry can't express.

The directives `_apply_directive` recognises:

| Directive                                                               | Effect                                                                                          |
| ----------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------- |
| `title:` / `style:`                                                     | graph title and theme name                                                                      |
| `line: id \| name \| #color \| style`                                   | declare a `MetroLine` (style is `solid` / `dashed` / `dotted`)                                  |
| `line_order:`                                                           | `definition` or `span` line ordering                                                            |
| `entry:` / `exit:` (inside a subgraph)                                  | stored as port **hints** on the section                                                         |
| `direction:`                                                            | section flow `LR` / `RL` / `TB`                                                                 |
| `grid:`                                                                 | manual section grid placement                                                                   |
| `compact_offsets:` / `center_ports:`                                    | bundle layout toggles                                                                           |
| `diamond_style:`                                                        | fork-join layout `straight` / `symmetric`                                                       |
| `line_spread:`                                                          | how shared lines relate vertically (`bundle` / `centered` / `rails`), graph-wide or per-section |
| `fold_threshold:`                                                       | station count at which long chains wrap into serpentine rows                                    |
| `x_spacing:` / `y_spacing:` / `section_x_gap:` / `section_y_gap:`       | layout spacing and section gaps                                                                 |
| `width:` / `height:` / `animate:`                                       | render output size and animation toggle                                                         |
| `off_track:`                                                            | mark stations to lift above the section's top track                                             |
| `label_angle:`                                                          | diagonal station-label angle                                                                    |
| `legend:` / `legend_min_height:` / `legend_combo:` / `legend_logo_gap:` | legend block                                                                                    |
| `logo:` / `logo_scale:`                                                 | logo path and scaling                                                                           |
| `font_scale:`                                                           | global font scaling                                                                             |
| `group:`                                                                | annotative caption spanning stations                                                            |
| `marker:` / `marker_legend:`                                            | per-station marker shape/fill styling and its legend caption                                    |
| `file:` / `files:` / `dir:`                                             | terminus file-icon designation                                                                  |

Note that `entry:` / `exit:` do **not** create `Port` objects at parse
time. `_parse_port_hint` records them as `entry_hints` / `exit_hints`
(a `(side, [line_ids])` list) on the `Section`; the actual ports are
created later, driven by real inter-section edges.

## Leniency and error policy

The split is deliberate: the **grammar/parse layer is lenient** about
syntax it doesn't recognise (it warns rather than crashing), while
**semantic validity is a separate, stricter phase**. A typo in a node
line should not abort a render of an otherwise-fine diagram, but an edge
that names no metro line is a real modelling error.

| Input                                                                                                                            | Outcome                                                                                    |
| -------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ |
| Blank line, `%%` comment, or `%%metro` line with no colon                                                                        | ignored silently                                                                           |
| Unrecognised non-blank line (the grammar `junk` rule)                                                                            | dropped, with a `UserWarning` ("Ignored unrecognised line: ...")                           |
| Unknown `%%metro` directive key                                                                                                  | ignored, with a `UserWarning` ("Ignored unknown %%metro directive: ...")                   |
| Malformed directive _payload_ (too few `\|` fields, an unusable enum/number/bool, a section-scoped directive outside a subgraph) | warned about and ignored, uniformly across handlers (`_warn_directive`)                    |
| Foreign/unsupported syntax (Mermaid `flowchart`)                                                                                 | raises `ValueError` with guidance, via `_check_unsupported_input`, before the grammar runs |
| Edge with no line annotation, or an undeclared line id                                                                           | raised by `_validate_edge_annotations` after parsing                                       |

Broader graph-semantic checks (beyond edge annotations) live in the
separate `validate` phase, `nf_metro.parser.validate.validate_graph`.

The unrecognised-line case is handled in the grammar by a low-priority
catch-all:

```lark title="src/nf_metro/parser/grammar.py"
JUNK.-10: /[^\n]+/
```

`JUNK` matches any line, but its negative priority means it only wins
when nothing more specific does. The transformer turns the match into a
`_Junk` statement, and the driver warns when it applies one.

This `junk` fallback is why the parser is configured as
`Lark(..., parser="earley", lexer="dynamic")` rather than the faster
`lalr`. A line that _begins_ like a valid statement but then hits an
unexpected token must be able to fall back to `junk` and be dropped. An
earley parser can explore that fallback; a committing `lalr` parser
cannot backtrack a partly-matched line, so it would turn such a line
into a fatal error instead.

<Aside type="danger" title="Do not switch the parser to lalr">
  Switching to `lalr` would turn any partly-matched unrecognised line into a
  fatal parse error instead of a dropped `junk` statement. The leniency policy
  (warn, don't crash) depends on Earley's ability to backtrack. The parse runs
  once per render, so Earley's extra cost over `lalr` is not worth trading away
  for this behaviour change.
</Aside>

## Parse-then-resolve flow

After the grammar parse and statement application, `parse_metro_mermaid`
runs a post-parse sequence (only when the graph has sections):

1. `_validate_edge_annotations` - reject malformed edges.
2. `_remove_empty_sections` and `_create_implicit_section` - drop empty
   subgraphs and wrap loose (section-less) stations in an implicit,
   invisible section.
3. `infer_section_layout` (from `layout/auto_layout.py`) - infer missing
   grid positions, section directions, and port sides from the section
   DAG, preserving anything set explicitly by directives.
4. `_insert_terminus_convergence_stations`.
5. `_resolve_sections` - the core rewrite (below).
6. `_insert_bypass_stations`.

Finally the parser applies pending terminus icons, `off_track` marks,
and per-station markers that were buffered during the statement scan.

### `_resolve_sections`

`_resolve_sections` rewrites inter-section edges into port/junction
chains. It is split into three helpers:

- `_build_entry_side_mapping` - per-line entry-side lookup from the
  `entry_hints`. A section gets **one** entry side: if all hints agree,
  that side is used; otherwise they collapse to the natural entry for the
  section direction (LEFT for LR, RIGHT for RL, TOP for TB).
- `_classify_edges` - split edges into internal (both endpoints in one
  section) and inter-section, and populate each section's
  `internal_edges`.
- `_create_ports_and_junctions` - create `Port` objects and rewrite each
  inter-section edge into a chain: `source -> exit_port -> entry_port ->
target`. The design rule is **one exit port per source section** (all
  lines leave together for consistent ordering) and **one entry port per
  target section per side**; junction stations handle fan-out to multiple
  target sections.

After ports and junctions exist, `_insert_merge_junctions` adds merge
junctions and `_assign_section_numbers` numbers any unnumbered sections.

The result is a `MetroGraph` whose edges all live within a section or
run port-to-port, ready for the layout stage.

## Adding things

- **A node shape** - add one alternative to the `SHAPE` terminal in
  `_GRAMMAR`; if its delimiters are two characters per side, add the
  opener to `two_char_opens` in `_shape_label`.
- **A `%%metro` directive** - write a `(value, graph) -> None` handler
  and add an entry to the `_GLOBAL_DIRECTIVE_HANDLERS` dict keyed by the exact
  directive name. No ordering concerns. (A directive that needs the
  enclosing section or the key itself is dispatched in `_apply_directive`
  instead of the dict.) On an unusable payload, call `_warn_malformed`
  (or `_warn_directive` for a more specific message) and return, rather
  than failing silently - that is the leniency policy above.
- **A new statement form** - add a rule and its terminal to `_GRAMMAR`,
  a typed statement dataclass (added to the `_Statement` union), a method
  to `_StatementTransformer` returning that dataclass, and an `isinstance`
  branch to the driver loop in `parse_metro_mermaid`.

## How equivalence is verified

The grammar must produce exactly the same `MetroGraph` as the input
implies, so changes here are checked by **comparing parsed model objects**
(not rendered SVGs): parse every fixture in `tests/fixtures/`,
`examples/`, and `examples/topologies/` and assert the model matches.
Because identical models render identically, this keeps the gallery
byte-identical without chasing sub-pixel render drift. The grammar's
coverage and behaviour are pinned by `tests/test_parser_grammar.py`.
