# NETCONFIG — YANG + TEMPLATE MIGRATION SYSTEM — PLAN
# Format: dense, grep-friendly, code-first. Written for AI-assisted contributors and grep — not narrative reading.
# Generated: 2026-04-16
# Last revised: 2026-05-04
#
# Status: this file mixes a long-term architectural sketch (most R / GAP /
# Phase items now [SHIPPED]) with active deferred-roadmap items at the
# bottom.  Active per-wave shipping notes live in CHANGELOG.md; this file
# is the slower-changing companion.

=====================================================================
## LONG-TERM ARCHITECTURE  (read first — supersedes the Phase 0-era shape)
=====================================================================
# Written after Phase 0 → 0.5 → 1 → Phase 2 UI shipped and manual QA
# revealed structural gaps.  The previous architecture conflated three
# independent concerns ("what device", "what format", "what transport")
# into a single "adapter" class.  This section defines the target
# architecture that separates them.  All existing code is forward-
# compatible; migration path is phased (see R1–R7 at the bottom).

--- FOUR-LAYER MODEL ---

  ┌─────────────────────────┐  ┌──────────────────────────┐
  │  Vendor Definition      │  │  Canonical Intent Model  │
  │  (what device is this?) │  │  (what is the tree?)     │
  └──────────┬──────────────┘  └─────────────┬────────────┘
             │                                │
             ▼                                ▼
  ┌─────────────────────────┐  ┌──────────────────────────┐
  │  Format Codec           │─▶│  Schema Validator        │
  │  (CLI/XML/JSON ↔ tree)  │  │  (strictness policy)     │
  └──────────┬──────────────┘  └──────────────────────────┘
             │
             ▼
  ┌─────────────────────────┐
  │  Transport              │
  │  (how to get bytes      │
  │   in/out of device)     │
  └─────────────────────────┘

LAYER 1 — VENDOR DEFINITION
  A small declarative struct: {name, device_classes, cli_prompt_hints,
  default_timeout}.  YAML-loaded.  No code.  Shared with the existing
  definitions/ YAML used by the backup collector (type_key).
  Purpose:
    * UX grouping ("Show me all FortiGate codecs")
    * Default device_class declaration
    * Taxonomy anchor shared with backup/storage layer

LAYER 2 — FORMAT CODEC  (renames today's "adapter")
  A codec translates between a WIRE FORMAT and the CANONICAL TREE.
  Key change: a codec declares BOTH a format AND a vendor, so the
  same vendor can have multiple codecs:

    cisco_iosxe_netconf_openconfig   — IOS-XE NETCONF OpenConfig subset
    cisco_iosxe_cli_running_config   — show running-config text parser
    fortigate_cli_config_blocks      — FortiOS CLI config/set/edit/next/end
    fortigate_rest_json              — FortiOS REST API JSON
    opnsense_xml_config              — config.xml
    mikrotik_cli_export              — /export text

  Each codec declares:
    vendor:           str          # points at a Vendor YAML
    format:           str          # machine-readable format tag
    direction:        enum         # parse_only | render_only | bidirectional
    canonical_model:  str          # which CIM it speaks (see layer 3)
    capability_matrix: CapabilityMatrix
    certainty:        enum         # certified | best_effort | experimental
    transports:       list[str]    # which Transport ids can deliver this format
    input_sample:     str | None   # working sample for the UI "Load sample" button

  Direction is critical: many CLI parsers are PARSE-ONLY because
  rendering clean CLI is harder than parsing it.  Separating direction
  means you can ship a parse-only codec that reads `show running-config`
  without also having to emit it.  The migration UI shows parse-only
  codecs only as SOURCE options, render-only codecs only as TARGET.

LAYER 3 — CANONICAL INTENT MODEL (pluggable CIMs)
  Today there is an implicit single CIM ("OpenConfig-ish xpaths").
  This fails for:
    * Firewalls (OpenConfig has no firewall model)
    * Wireless controllers
    * Load balancers
    * Any vendor with features OC doesn't cover

  Fix: named CIMs, each a set of YANG modules (or hand-authored xpath
  inventories until libyang lands):

    openconfig-lite           — the subset we actually support
    openconfig-full           — future: full OC when libyang lands
    netcanon-firewall-ext    — firewall rules, NAT, IPS/IDS
    netcanon-wireless-ext    — WLC, AP, SSID, radio profiles
    netcanon-loadbalancer    — VIPs, pools, monitors

  A codec declares which CIM(s) it emits.  A migration between two
  codecs that share at least one CIM can proceed directly.  Cross-CIM
  migration (rare) invokes a ModelProjector that maps between CIMs;
  this is a separable concern and a future piece of work.

  Most codecs (90%+ of real work) target one CIM.  Cross-CIM
  translation is an edge case handled by a small module when it arises.

LAYER 4 — TRANSPORT
  Separate from "how to parse bytes" is "how to get those bytes off
  a device."  Today Netcanon's backup collectors handle this
  (SSH+Netmiko for CLI, Paramiko for shell).  The translator ignores
  transport entirely — you hand it a text blob.

  Future: transports are first-class.  A Transport knows how to talk
  to a device type:

    ssh_cli            — existing backup path (Netmiko / Paramiko)
    netconf_ssh_830    — new, needs ncclient
    restconf_https     — Cisco/Huawei RESTCONF
    panos_xmlapi_https — Palo Alto XML-API
    fortigate_rest     — Fortinet's REST API
    snmp_get           — ancient but real for some state queries
    manual             — no transport; bytes supplied by the user

  Transports are selected by the combination of vendor × format
  capability on the device.  A codec declares which transports can
  deliver its format, and the collector picks the best available.
  Manual-input mode (paste into textarea) is just "manual transport."

--- CERTAINTY MODEL (per-codec) ---

  certified      — round-trip invariant tested against ≥3 real device
                   captures from ≥2 OS versions.  Deploy-ready.
  best_effort    — round-trip invariant tested against synthetic
                   samples only.  Expected to work; needs human review.
  experimental   — parse-only, or incomplete render.  Output is a
                   starting point for review, not deploy-ready.

  UI shows a chip on each codec.  Migration to best_effort or
  experimental auto-raises severity to warn.  Deploy (Phase 2+) is
  disabled unless both source AND target are certified.

--- AUTO-DETECTION (eliminates the "pick the right format" step) ---

  Stored configs carry their bytes.  The first few lines are usually
  diagnostic:
    * Starts with `<?xml` or `<opnsense>` → XML variant
    * Starts with `Building configuration` or `!` → IOS CLI
    * Starts with `#config-version=` → FortiGate CLI
    * Starts with `#` + `/` commands → MikroTik export
    * Starts with `{` → JSON (mock or REST capture)

  A probing function walks registered codecs, tests each against the
  first ~500 bytes, and returns a ranked list of compatible codecs.
  The UI uses this to pre-select the source codec when the user picks
  a stored config — eliminating the "pick the right adapter" step
  entirely for stored files.  Manual paste still shows the format-hint.

--- DECLARATIVE CODEC SPEC (target on-disk layout) ---

  netcanon/migration/codecs/<codec_id>/
    codec.yaml            # all declarative fields (see above)
    parse.py              # optional — if parse_only or bidirectional
    render.py             # optional — if render_only or bidirectional
    templates/*.j2        # for CLI-emitting codecs
    fixtures/*.{xml,cfg}  # sanitised samples for tests
    test_roundtrip.py     # required; must pass for codec to register

  codec.yaml example:

    id: cisco_iosxe_cli_running_config
    vendor: cisco_iosxe
    format: cli-ios-running-config
    direction: parse_only
    canonical_model: openconfig-lite
    certainty: best_effort
    transports: [ssh_cli, manual]
    input_sample_file: fixtures/show_run_sample.txt
    capabilities:
      supported_paths:
        - /interfaces/interface/config/name
        - /interfaces/interface/config/description
        ...
      lossy_paths: [...]
      unsupported_paths: [...]

--- CONTRIBUTION PATHWAY (new vendor onboarding) ---

  1. Add a vendor YAML (netcanon/migration/vendors/<name>.yaml) — 30s.
  2. Pick a CIM (usually openconfig-lite for L2/L3; netcanon-firewall-ext
     for firewalls).  New CIM = harder; requires architectural review.
  3. Write the codec.  Parse is the hard work; render is optional for
     Phase-1 "I just want to READ configs from this vendor."
  4. Declare transports.  Usually ssh_cli for CLI codecs.
  5. Ship round-trip tests using the standard pattern.
  6. Declare certainty (start at experimental; graduate to best_effort
     after synthetic samples pass; to certified after real captures).

  ~200-500 lines of code per codec.  The scaffolding handles
  registration, validation, UI surface, testing, and documentation.

--- WHAT THIS RESOLVES (current pain points → future fix) ---

  "Paste box only takes XML"
    → CLI codecs + auto-detection → any text works.

  "Stored configs can't be migrated through the translator"
    → Stored configs carry format metadata; auto-detection offers
      compatible codecs automatically.

  "OPNsense firewall rules are unsupported against Cisco"
    → Different CIMs.  The model projector decides what's
      translatable (or refuses: "firewall rules have no target
      CIM in openconfig-lite").

  "IOS-XE CLI vs NETCONF both valid"
    → Two codecs, one vendor.  User (or auto-detection) picks.

  "Adding a new vendor is too much code"
    → One vendor YAML + one or more codecs + standard test pattern.

--- MIGRATION PATH FROM CURRENT CODE (R1–R7) ---

  ┌────────────────────────────────────────────────────────────────┐
  │ REAL-CAPTURE VALIDATION — all 5 vendors covered                │
  ├────────────────────────────────────────────────────────────────┤
  │ SUPERSEDED: This section captures the state of the             │
  │   real-capture harness DURING the R6/7 bug-fix sessions.       │
  │   All 5 shipped codecs have since been promoted to             │
  │   `certified` (the "Still to do — to graduate any codec to     │
  │   certified" subsection inside this block is stale).  The      │
  │   current cert matrix, known silent drops, and hardening       │
  │   backlog live in tests/fixtures/real/RESULTS.md — single      │
  │   source of truth.  Preserved here as a historical journal     │
  │   entry showing which bugs each fixture surfaced during        │
  │   promotion; do not treat it as current state.                 │
  │                                                                │
  │ STATUS (historical): Cisco/OPNsense/MikroTik/FortiGate         │
  │   validated against captured configs.  Aruba AOS-S partially   │
  │   unblocked via                                                │
  │   Aruba Central 5MemberStack template rendered through         │
  │   scripts/render_aruba_central_template.py.  Rendered fixture  │
  │   is strictly less valuable than a real sanitised capture —    │
  │   do-better note in the aruba_aoss/README.md.                  │
  │                                                                │
  │ Infrastructure shipped:                                        │
  │   * tests/fixtures/real/ directory with per-vendor subdirs.    │
  │   * NOTICE.md tracking provenance of every committed fixture   │
  │     (Apache / MIT / BSD license only).                         │
  │   * tests/unit/migration/test_real_captures.py — parametrized  │
  │     harness that auto-discovers fixtures, asserts parse-       │
  │     doesn't-crash + emits CanonicalIntent + non-trivial        │
  │     coverage, and prints a per-fixture extraction summary.     │
  │     Deterministic-parse assertion included.                    │
  │   * RESULTS.md — human-curated per-codec coverage matrix,      │
  │     findings, certification decisions.                         │
  │                                                                │
  │ Results summary (full matrix in tests/fixtures/real/RESULTS.md):│
  │                                                                │
  │   codec           | fix | OS  | bugs | cert status              │
  │   ----------------+-----+-----+------+-------------------       │
  │   cisco_iosxe_cli |  6  |  1* |  1   | best_effort              │
  │   opnsense        |  3  |  1  |  0   | best_effort              │
  │   mikrotik        |  4  |  3  |  6   | CERTIFIED                │
  │   fortigate       |  2  |  1  |  1   | best_effort              │
  │   aruba_aoss      |  1‡ |  —  |  0   | best_effort              │
  │   * all Cisco fixtures are Batfish/NTC test grammars            │
  │   ‡ rendered from Aruba Central template, not captured          │
  │                                                                │
  │ mikrotik_routeros is the FIRST codec to hit `certified` tier.  │
  │ 4 real fixtures across 3 OS versions (6.48.1, 6.48.6, 7.18.2), │
  │ all round-tripping cleanly.  The 4th fixture (user-contributed │
  │ CRS310 /export, sanitised) surfaced two structural bugs that   │
  │ were fixed in the same session:                                │
  │   * CanonicalInterface.default_name field added to track the   │
  │     RouterOS factory default-name distinct from the renamed    │
  │     name ("Access Point" rather than "ether2").                │
  │   * CanonicalVlan.name now stores the iface name (needed for   │
  │     render's vlan-id lookup); description field holds the      │
  │     human comment.                                              │
  │                                                                │
  │ 4 bugs surfaced so far by real captures, all fixed in-session  │
  │ with regression tests.  Sample bugs — each would have survived │
  │ arbitrarily long against synthetic fixtures:                   │
  │   * cisco: _parse_lags appended Ethernet0 7x when Batfish's    │
  │     kitchen sink stacked 7 `channel-group N mode <variant>`    │
  │     lines on one iface.                                        │
  │   * mikrotik: round-trip drift from missing name-based         │
  │     interface_type inference in _parse_ip_address().  Second   │
  │     parse hit /interface ethernet section and stamped the      │
  │     type; first parse didn't.  Fixed via                       │
  │     _infer_iface_type_from_name() called consistently.         │
  │   * fortigate: `set vlanid N` + `set interface parent` without │
  │     explicit `set type vlan` wasn't recognised as a VLAN iface │
  │     — real 7.6.6 configs use this form and lost VLAN records.  │
  │     Fixed by treating the pair as an implicit VLAN signal.     │
  │   * mikrotik: hostnames containing spaces (e.g. "Quinta        │
  │     Router") were emitted unquoted on render, which RouterOS   │
  │     parses as two tokens (name + orphan).  Surfaced by         │
  │     routeros-diff's verbose_export.rsc fixture.  Fixed by      │
  │     adding a _quote_if_needed() helper that wraps any value    │
  │     containing whitespace/quotes/backslashes.                   │
  │                                                                │
  │ Harness additions this session:                                │
  │   * parse->render->parse round-trip assertion for all          │
  │     bidirectional codecs (skips parse_only).  Catches the      │
  │     silent-drift class of regressions that canonical-shape     │
  │     comparison alone misses.                                   │
  │   * Per-vendor extension whitelist (.rsc added for RouterOS).  │
  │                                                                │
  │ Known silent drops (real configs exercise them, our canonical  │
  │ doesn't model them — all mapped to existing roadmap buckets):  │
  │   * Cisco: VRFs, Q-in-Q, QoS, ACL groups, IPv6, proxy-ARP,     │
  │     uRPF, bandwidth.                                           │
  │   * FortiGate: VDOMs (beyond first), firewall policies, VIPs,  │
  │     IPsec VPN, SD-WAN rules, AV/IPS profiles, web filtering.   │
  │   * OPNsense: firewall rules, NAT, DHCPv6 prefix delegation    │
  │     advanced options.                                          │
  │   * MikroTik: firewall rules, queue tree, mangle.              │
  │ See RESULTS.md for the full per-vendor tier mapping.           │
  │                                                                │
  │ Still to do — to graduate any codec to `certified`             │
  │ (≥3 captures from ≥2 OS versions, round-trip stable):          │
  │   * DO-BETTER for Aruba: swap rendered template for a real     │
  │     sanitised capture when one surfaces.  Rendered template    │
  │     only exercises grammar Aruba's template author             │
  │     anticipated — real deployments carry corner cases that     │
  │     surface further bugs (as every real vendor capture has     │
  │     done so far).                                              │
  │   * Double fixture counts on OPNsense / MikroTik / FortiGate / │
  │     Aruba (currently 2/1/1/1 → need 3/3/3/3) and target 2 OS   │
  │     versions per codec.                                        │
  │ cisco_iosxe_cli is parse_only so the round-trip graduation     │
  │ path is actually via a consumer codec (Aruba/OPNsense/etc.)    │
  │ rendering the Cisco-parsed intent and checking vs. source.     │
  │                                                                │
  │ 1015 tests passing (was 885 at start of the bug-fix batch,     │
  │ +130 across all 4 bug fixes + real-capture harness + all       │
  │ 5 vendor validations).                                         │
  └────────────────────────────────────────────────────────────────┘

  ┌────────────────────────────────────────────────────────────────┐
  │ KNOWN DATA-LOSS BUGS  (exposed by real-config dogfooding)      │
  ├────────────────────────────────────────────────────────────────┤
  │ Surfaced while the user ran a real Cisco 9300 switch config    │
  │ through cisco_iosxe_cli -> aruba_aoss.  Both bugs silently     │
  │ drop data — no warning, just missing sections in the render.   │
  │                                                                │
  │ BUG 1 — VLAN SVI dropped on IOS-XE CLI parse.  STATUS: FIXED.  │
  │   Symptom was: `interface Vlan11 / ip address 192.168.11.252`  │
  │   produced a CanonicalInterface but NO matching CanonicalVlan, │
  │   so Aruba's renderer (which skips Vlan* interfaces expecting  │
  │   the vlan stanza to absorb them) dropped the SVI + IP.        │
  │                                                                │
  │   Fix shipped: cisco_iosxe_cli parser's new                    │
  │   _synthesize_vlans_from_svis() post-pass derives a            │
  │   CanonicalVlan(id=N) from each `interface Vlan<N>` stanza,    │
  │   attaches the SVI's ipv4_addresses, and merges with any       │
  │   existing top-level `vlan N / name X` record (keeping the     │
  │   explicit name as authoritative).  SVI description falls back │
  │   as the VLAN name when no stanza is present.                  │
  │                                                                │
  │   Also shipped: a cross-codec matrix invariant                 │
  │   (test_every_source_ip_appears_in_rendered_output) — every    │
  │   IPv4 address in the parsed-source tree MUST appear as a      │
  │   literal substring in the target codec's rendered output.     │
  │   Substring-based rather than re-parse based so it doesn't     │
  │   depend on the target accepting foreign interface names       │
  │   (e.g. AOS-S's parser won't read `GigabitEthernet0/0/0`       │
  │   but the IP still reaches the rendered text).  This guard     │
  │   would have caught this bug on day one.  8 new unit tests +   │
  │   25 parametrized invariant runs; 885 tests passing.           │
  │                                                                │
  │ BUG 2 — LAG/port-channel member associations dropped.  FIXED. │
  │   Symptom was: Cisco `interface Port-channel1 / description    │
  │   TRUNK` made it to Aruba as a LAG name with no member ports;  │
  │   Cisco's per-port `channel-group 1 mode active` was lost.     │
  │                                                                │
  │   Fix shipped: CanonicalLAG wired across all 5 codecs.         │
  │     * cisco_iosxe_cli: new _CHANNEL_GROUP_RE + _parse_lags()   │
  │       derive LAGs from both `interface Port-channelN` and      │
  │       per-member `channel-group N mode M`.  Cisco modes        │
  │       (active/passive/on/auto/desirable) -> canonical via      │
  │       _CISCO_LAG_MODE_MAP.  Members stamped lag_member_of.     │
  │     * aruba_aoss: parse+render `trunk <ports> <name> <type>`;  │
  │       non-native LAG names (Port-channel1, lagg0, bond1)       │
  │       auto-translate to AOS-S trkN form.  lacp/dt-lacp <->     │
  │       active; trunk/fec <-> static.  Empty LAGs become a       │
  │       comment (no port list = no valid trunk line).            │
  │     * fortigate_cli: parse+render `set type aggregate` +       │
  │       `set member "p1" "p2"` with lacp-mode.  Render           │
  │       synthesises missing LAG interfaces.  Two-pass            │
  │       lag_member_of stamping handles out-of-order edits.       │
  │     * mikrotik: parse+render /interface bonding with slaves=,  │
  │       mode= (802.3ad <-> active, active-backup <-> static),    │
  │       name=.                                                   │
  │     * opnsense: parse+render <laggs>/<lagg> XML with           │
  │       <laggif>/<members>/<proto> (lacp <-> active,             │
  │       failover <-> static).                                    │
  │   25 new tests: parse, render, round-trip, mode mapping,       │
  │   naming translation, empty-LAG graceful handling, end-to-end  │
  │   Cisco->Aruba flow.                                           │
  │                                                                │
  │ BUG 3 — Cisco per-port switchport data never reaches target.   │
  │   STATUS: FIXED.                                               │
  │   Symptom was: `interface Ten1/0/5 / switchport access vlan 20 │
  │   / switchport trunk allowed vlan 11,20 / switchport mode      │
  │   trunk` populated CanonicalInterface fields -- but no VLAN-   │
  │   centric render read them back.  Aruba emitted `interface N / │
  │   enable / exit` and nothing about VLAN membership.            │
  │                                                                │
  │   Fix shipped: netcanon/migration/canonical/transforms.py     │
  │   with two idempotent in-place bridges.                        │
  │     * project_switchport_to_vlan (port -> VLAN-centric):       │
  │       access port -> append iface to vlans[N].untagged_ports;  │
  │       trunk port -> append iface to vlans[vid].tagged_ports    │
  │       for each allowed vid; native VLAN demoted from           │
  │       tagged_ports to untagged_ports (Cisco allows listing     │
  │       native in allowed but it's still sent untagged).         │
  │       Missing VLAN stanzas are synthesised so membership       │
  │       isn't lost.  Wired into cisco_iosxe_cli.parse().         │
  │     * project_vlan_to_switchport (inverse, dormant until a     │
  │       port-centric renderer exists).                           │
  │   22 new tests (16 transform-level, 2 Cisco->Aruba end-to-end, │
  │   4 inverse).                                                  │
  │                                                                │
  │ BUG 4 — `ip default-gateway` not parsed on Cisco IOS CLI.      │
  │   STATUS: FIXED.  Added _DEFAULT_GATEWAY_RE and extended       │
  │   _parse_static_routes to emit CanonicalStaticRoute(           │
  │   destination="0.0.0.0/0", gateway=X).  Aruba's renderer       │
  │   already collapses 0.0.0.0/0 routes back to the native        │
  │   `ip default-gateway` form, so the round-trip is correct.     │
  │   4 new tests: parse, coexistence with explicit ip route,      │
  │   round-trip through Aruba, case-insensitivity.                │
  │                                                                │
  │ All four bugs now closed.  Real-capture validation (URGENT     │
  │ NEXT above) can now graduate codecs to `certified` with the    │
  │ structural gaps closed.  987 tests passing (up from 885 at     │
  │ start of this bugfix batch, +102 across the four fixes).       │
  └────────────────────────────────────────────────────────────────┘

  ┌────────────────────────────────────────────────────────────────┐
  │ TIER 2 — ALL SHIPPED                                           │
  ├────────────────────────────────────────────────────────────────┤
  │ STATUS: complete.  SNMP (earlier), LAGs (Bug 2), local_users,  │
  │ DHCP pools, RADIUS servers — all wired across all 5 codecs     │
  │ (with pragmatic skip for Aruba DHCP since AOS-S is a relay     │
  │ platform, not a server).  See per-item notes below for what    │
  │ shipped and where.  1081 tests passing.                         │
  │                                                                │
  │ Canonical model defines these but no codec wires them.  Same   │
  │ shape of work as Session E's SNMP wire-through.  Each is one   │
  │ session: model is already in canonical/intent.py, just needs   │
  │ per-codec parse+render+tests and capability-matrix declaration.│
  │                                                                │
  │   * local_users  — SHIPPED.  Cisco `username X privilege N     │
  │                    secret ...`, Aruba `password manager user-  │
  │                    name ...`, FortiGate `config system admin`, │
  │                    MikroTik `/user add`, OPNsense              │
  │                    `<system><user>`.  Hashed passwords carry   │
  │                    a per-vendor algorithm tag                   │
  │                    (sha1/bcrypt/fortios/cisco-type-N:) so      │
  │                    renderers can route correctly.  26 tests.   │
  │                                                                │
  │   * DHCP server pools — SHIPPED.  Cisco `ip dhcp pool`,         │
  │                         OPNsense `<dhcpd>/<zone>`, MikroTik    │
  │                         `/ip dhcp-server network` + `/ip pool` │
  │                         with deferred-merge (file-order        │
  │                         agnostic), FortiGate `config system    │
  │                         dhcp server` with nested `ip-range`.   │
  │                         Aruba renders a comment block (AOS-S   │
  │                         is a relay platform, not a server).    │
  │                         18 tests.                               │
  │                                                                │
  │   * RADIUS servers — SHIPPED.  Cisco modern named-stanza +     │
  │                      legacy one-liner, Aruba inline-key +      │
  │                      global-key backfill, OPNsense             │
  │                      `<authserver type="radius">` (ldap/local  │
  │                      correctly filtered out), MikroTik         │
  │                      `/radius add`, FortiGate `config user     │
  │                      radius`.  22 tests.                        │
  │                                                                │
  │   * (LAGs shipped as Bug 2 — see KNOWN DATA-LOSS BUGS above.)  │
  │   * (SNMP shipped in Tier 2 SNMP session — see CHANGELOG.)     │
  └────────────────────────────────────────────────────────────────┘

  ┌────────────────────────────────────────────────────────────────┐
  │ FIDELITY POLISH  (low-priority, catch-all bucket)              │
  ├────────────────────────────────────────────────────────────────┤
  │ Small gaps surfaced by real-config dogfooding.  None are       │
  │ blocking; all are honest "we don't carry this field" cases     │
  │ that could be improved incrementally once the big data-loss    │
  │ bugs are closed.                                                │
  │                                                                │
  │   * Per-interface MTU — SHIPPED.  Wired on all 5 codecs:        │
  │     cisco_iosxe_cli parse (`mtu 1500`); opnsense parse+render  │
  │     (`<mtu>` element); mikrotik parse+render                    │
  │     (`/interface ethernet set ... mtu=N`); fortigate            │
  │     parse+render (`set mtu-override enable` + `set mtu N`).    │
  │     Aruba AOS-S intentionally skipped — no per-port MTU on     │
  │     that platform (global `jumbo` only).  14 regression tests. │
  │     Real captures now surface real MTUs (9000 on Cisco,        │
  │     1500 on MikroTik, 9096/1546 on the NTC carrier config).    │
  │   * Per-interface speed/duplex — not modelled at all.  Small   │
  │     CanonicalInterface addition + per-codec wiring.            │
  │   * Cisco negative commands (`no ip igmp snooping vlan 20`,    │
  │     `no errdisable detect cause ...`) — silently ignored.      │
  │     Decide: Tier 3 opaque-carry, or Tier 2 structured?         │
  │   * Port admin-status default — bare `interface TenGigE1/1/1`  │
  │     with no body currently defaults to enabled=True on parse,  │
  │     which is correct for most Cisco platforms but not          │
  │     universally.  Verify against real 9300 capture.            │
  │   * Cisco `vrf definition <name>` and `vrf forwarding <name>`  │
  │     on interfaces — currently dropped.  VRFs aren't in the     │
  │     canonical model yet.                                       │
  │   * Cisco `spanning-tree mode rapid-pvst` and related STP      │
  │     globals — Tier 2 model exists for per-port settings only.  │
  │   * Cisco `crypto pki certificate chain ...` multi-line blobs  │
  │     — should land in Tier 3 raw_sections opaque carry-through  │
  │     so they survive parse even if never auto-rendered.         │
  │   * Cisco `privilege exec level <N>` aliases — Tier 3          │
  │     informational; currently dropped.                          │
  │                                                                │
  │   * MikroTik /interface bridge render — SHIPPED.  Emits        │
  │     `/interface bridge` section for every CanonicalInterface    │
  │     typed `ianaift:bridge`, preserving name (with quoting for  │
  │     spaces) and description.                                    │
  │   * MikroTik VLAN interface name preservation — SHIPPED.        │
  │     Render now filters by interface_type="ianaift:l3ipvlan"    │
  │     (not just `vlan\d+` name pattern), so named VLAN           │
  │     interfaces like `gn-mgmt` survive without being rewritten  │
  │     to synthetic `vlan<N>`.  Deduplication in the synthetic-    │
  │     fallback loop tracks by vlan-id instead of name so an      │
  │     iface-vlan pair doesn't emit twice.                         │
  └────────────────────────────────────────────────────────────────┘

  R1  Rename adapter → codec everywhere.  Add vendor_id to the
      capability matrix.  Back-compat shims.  1 session.  No
      behaviour change.  STATUS: SHIPPED.

  R2  Extract vendor declarations to YAML files.  Load at startup.
      1 session.  STATUS: SHIPPED.

  R3  Add direction + canonical_model + certainty fields.
      Wire into validation + UI.  1 session.  STATUS: SHIPPED.

  R4  First CLI codec: cisco_iosxe_cli.  First parse_only codec,
      first multi-codec-per-vendor usage.  Proves the architecture.
      2-3 sessions.  STATUS: SHIPPED.

  CANONICAL BRIDGE — Session 1 of vendor-config-research plan.
      STATUS: SHIPPED.  Defined CanonicalIntent pydantic model
      (netcanon/migration/canonical/intent.py) with Tier-1 fields
      (hostname, domain, dns/ntp/syslog servers, interfaces, vlans,
      static_routes), Tier-2 (dhcp, snmp, lags, local_users, radius),
      and Tier-3 (raw_sections opaque blob).  Refactored
      cisco_iosxe_cli, opnsense, and cisco_iosxe codecs to emit/consume
      CanonicalIntent directly — the tree shape is now shared across
      all three.  First successful cross-vendor translation proven
      end-to-end: IOS-XE `show running-config` → CanonicalIntent →
      OPNsense config.xml, with hostname, interfaces, descriptions,
      enabled flags, and IPv4 addresses all transferring.  Legacy
      dict-shape acceptance kept in render() paths for back-compat.
      740 tests passing; no regressions.

  MIKROTIK CODEC — Session 2 of vendor-config-research plan.
      STATUS: SHIPPED.  Third real codec:
      netcanon/migration/codecs/mikrotik_routeros/.  Parses and
      renders RouterOS `/export verbose` text through the canonical
      dict.  Scope: system identity, DNS + NTP servers, ethernet
      port tweaks, VLAN interfaces, bridges, IPv4 addresses, static
      routes.  Bidirectional; `best_effort` certainty.  Validates
      that the canonical dict design from Session 1 is genuinely
      portable — RouterOS's section/add/set grammar is structurally
      the furthest thing from XML or indented IOS CLI, and the same
      tree shape works unchanged.  Proven cross-vendor pairs:
      IOS-XE CLI → MikroTik, MikroTik → IOS-XE NETCONF, MikroTik →
      OPNsense.  Vendor-pair combinatorial surface grew from 6 to 24.
      659 tests passing; no regressions (38 new, 1 updated).

  R5  Auto-detection from stored-config bytes.  STATUS: SHIPPED.
      Added CodecBase.probe(raw_prefix) classmethod hook; each of the
      four real codecs (cisco_iosxe, cisco_iosxe_cli, opnsense,
      mikrotik_routeros) overrides it with a format-specific signature
      test returning (confidence, reason).  New service
      netcanon/services/migration_detect.py walks the registry,
      returns ranked list.  New endpoint POST /api/v1/migration/detect.
      /migrate UI gained a debounced auto-detection banner with
      "Use this source" button; banner goes green when the user has
      already picked the detected codec.  Also patched the MikroTik
      FORMAT_CATALOGUE gap (sample + exts) and generalized
      guessExtension to read input_format from /adapters instead of
      hard-coding vendor names.  705 tests passing; no regressions.

  TIER 2 — SNMP — STATUS: SHIPPED.
      First Tier 2 feature wired end-to-end across all 5 real codecs
      (cisco_iosxe_cli, opnsense, mikrotik_routeros, aruba_aoss,
      fortigate_cli).  Parse + render + roundtrip + cross-vendor.
      Per-codec grammars:
        * Cisco IOS CLI: `snmp-server community/location/contact/host`
        * OPNsense:      `<snmpd>` plugin element with rocommunity,
                         syslocation, syscontact, traphost
        * MikroTik:      `/snmp set` (sysinfo) + `/snmp community set`
        * Aruba AOS-S:   `snmp-server community/location/contact/host`
        * FortiGate:     `config system snmp sysinfo` +
                         `config system snmp community` with nested
                         `config hosts` sub-table
      Canonical xpath paths declared in every codec's capability
      matrix.  24 new unit tests (11 per-codec parse/render/roundtrip
      + 4+4 parametrized universal-render + universal-roundtrip).
      852 tests passing.  Paves the way for local_users, LAGs,
      RADIUS servers, DHCP server pools — same shape of work.

  FORTIGATE CLI CODEC — STATUS: SHIPPED.
      Fifth real codec.  netcanon/migration/codecs/fortigate_cli/
      parses and renders FortiOS 7.x CLI (`config/edit/set/next/end`
      grammar) via a recursive block model.  Handles: hostname,
      DNS (primary/secondary), NTP (nested ntpserver subtable),
      interfaces (physical + VLAN sub-interfaces via `set type vlan`
      + `set vlanid`), static routes with dst + gateway + device.
      Quoted values with spaces, multi-token set values, integer +
      quoted edit IDs.  Auto-detection probe matches the `#config-
      version=` banner (98%) and the 5-keyword grammar presence.
      Best_effort certainty; 41 new unit tests.  Also landed the
      FULL-MESH CROSS-CODEC MATRIX test (tests/unit/migration/
      test_cross_codec_matrix.py) which auto-enumerates every
      (source, target) pair where the two codecs share a device
      class and runs each source's sample_input through the pipeline.
      26 real cross-vendor pairs now covered; 2 pre-existing latent
      bugs exposed and fixed on first run (MockCodec couldn't
      serialize CanonicalIntent; cisco_iosxe NETCONF parse() still
      returned the legacy nested dict — migrated to CanonicalIntent).
      828 tests passing.

  ARUBA AOS-S CODEC — STATUS: SHIPPED.
      Fourth real codec.  netcanon/migration/codecs/aruba_aoss/
      parses and renders ArubaOS-Switch (ProCurve) 16.x
      `show running-config` text.  Architecturally the most important
      codec so far: it's the first one where VLAN port membership
      naturally lives on the VLAN object (Aruba's `vlan 10` ->
      `untagged 1-24` / `tagged 25-26` grammar), which validates
      the canonical VLAN-centric design decision.
      Handles: hostname, VLANs with port lists + SVI IPs, physical
      interfaces with `enable`/`disable`/`routing`/IP, static routes
      with `ip route` + `ip default-gateway` -> 0.0.0.0/0 conversion,
      port-range expansion (`1-24` / `A1-A4`) and compression,
      `;` comment character (vs Cisco's `!`), CIDR + dotted-decimal
      mask forms.  Best_effort certainty; 49 new unit tests.
      Also fixed an OPNsense renderer bug exposed by cross-vendor
      use: bare-numeric interface names (`1`, `25` — legal on Aruba)
      now sanitise to valid XML element tags.  760 tests passing.

  UI METADATA MIGRATION — STATUS: SHIPPED.
      CodecBase gained description / sample_input / output_extension
      ClassVars; each codec is now the single source of truth for its
      own UI-presentation strings.  CodecInfo pydantic model +
      /api/v1/migration/adapters response both grew three fields.
      migrate.html lost the 130-line FORMAT_CATALOGUE dict; all six
      call sites (renderFormatHint, applyPlaceholder,
      loadSampleForSourceAdapter, refreshFilenameCompatWarn,
      guessExtension, downloadMigrateOutput) now read from the
      adapters array via new adapterEntry() / compatibleExtensions()
      helpers.  Adding a new codec no longer requires editing the
      template.  708 tests passing; zero regressions.

  R6  Named CIMs with netcanon-firewall-ext as the first non-OC
      CIM.  ModelProjector between CIMs.  3-4 sessions.

  R7  Transport layer extraction, NETCONF transport, migration-
      aware collector.  Unlocks live device round-trips.
      3-4 sessions.

  RECOMMENDATION: ship R1–R4 first (5-7 sessions total).  Stop.
  Let real codecs accumulate.  Revisit R5-R7 once the ecosystem
  tells you which parts are load-bearing and which are speculative.

--- HONEST WORRY ---

  Premature modelling.  We have 3 adapters.  Four-layer architecture
  for 3 adapters is overkill.  The biggest risk isn't the design
  being wrong — it's landing the whole thing before we know which
  parts are load-bearing.  Ship R1-R4, let real adapters accumulate,
  then decide whether R5-R7 earn their complexity.

=====================================================================
## PORT-NAME TRANSLATION (Tiers 1+2+3) — SHIPPED + DEFERRED WORK
=====================================================================
# Ships: cross-vendor port-name bridge so source-vendor names don't
# leak into target-vendor output (the "GigabitEthernet1/0/24 in an
# Aruba config" complaint).  Three-tier design, all committed bar
# the small roadmap items tracked below.
#
# CODE LOCATIONS
#   * netcanon/migration/canonical/port_names.py
#       - PortIdentity pydantic model (vendor-agnostic bridge)
#       - translate_port_names(intent, source, target, rename_map=None)
#       - build_port_rename_transform(source, target, rename_map=None)
#   * netcanon/migration/codecs/<vendor>/codec.py
#       - classify_port_name(name) → PortIdentity
#       - format_port_identity(ident) → str | None
#       Each codec implements ONLY its own vendor's rules.
#   * netcanon/services/migration_pipeline.py
#       - run_plan_with_rename(source, target, raw, port_rename_map=)
#         NEW public function; existing run_plan stays frozen per the
#         "pipeline stages never change shape" rule.
#   * netcanon/migration/target_profiles.py
#       - TargetProfile pydantic model + YAML loader
#   * definitions/target_profiles/*.yaml
#       - aruba_2930f_48g_poep.yaml
#       - cisco_c9300_24ux.yaml
#       Add more profiles as vendor YAML — no code change needed.
#   * netcanon/api/routes/migration.py
#       - POST /plan now accepts optional port_rename_map +
#         target_profile; returns job.port_renames + job.warnings
#       - GET /target-profiles (+ /{vendor}/{model})
#   * netcanon/templates/migrate.html
#       - Tier 3 draggable 2-pane modal with mapping table + live
#         preview + collision detection + target-profile selector

--- SHIPPED ---

TIER 1 — auto-heuristic cross-vendor port-name rewrite.
  * classify/format methods on each codec; orchestrator rewrites
    every port-name field in CanonicalIntent uniformly.
  * Cisco→Aruba: Gi1/0/24→1/24, Port-channel1→Trk1, etc.
  * Full mesh: every (source, target) pair works without
    vendor-pair conditionals.
  * Complexity cases modelled: breakout, hw_aggregate,
    name_speed_hint vs operational_speed, subslot_letter.

TIER 2 — user-supplied rename_map overrides the auto-heuristic.
  * Per-source-name map wins over auto; missing names auto.
  * New pipeline function run_plan_with_rename; existing run_plan
    stays frozen.
  * MigrationJob gains port_renames + warnings fields.

TIER 3 — interactive rename modal in /migrate UI.
  * Draggable modal with mapping table (left) + live preview (right).
  * Per-kind collapsible sections (physical / lag / svi / loopback /
    tunnel / breakout / hw_aggregate / virtual / unknown).
  * Per-port dropdown options driven by selected target profile,
    free-text fallback.
  * Collision detection disables Apply when two sources map to the
    same target.
  * Client-side live preview; Apply button fires server-side
    re-render for canonical output.

--- DEFERRED ROADMAP ---

[WIP] CROSS-MESH FIDELITY AUDIT — multi-phase, branched off `audit/cross-mesh-throughput`.

  Phase 1 [SHIPPED]: mechanical drift matrix.
    * tools/run_full_mesh.py — standalone runner that walks every
      committed real-capture fixture × every bidirectional codec,
      performs parse(render(parse(raw))) and records per-canonical-
      field drift.  Output: timestamped JSON under
      tests/fixtures/real/_cross_mesh_runs/ (gitignored) and
      tests/fixtures/real/CROSS_MESH_RESULTS.md (committed snapshot).
    * tests/unit/audit/test_run_full_mesh.py — pins drift-computation
      building blocks (identical → all preserved, list-order-cosmetic,
      unsupported-by-design classification, scalar-empty-state
      semantics, _AUDITED_FIELDS coverage of CanonicalIntent).
    * tools/README.md documents the runner + cell-status legend.
    * .gitignore excludes per-run JSON.
    * tests/fixtures/real/RESULTS.md cross-references the new matrix.

  Phase 2 [DEFERRED]: drift trend tracking — diff the latest matrix
    vs the previous committed one to highlight regressions on each
    audit pass.

  Phase 3 [DEFERRED]: vendor-doc-grounded expectations YAML
    (tests/fixtures/cross_vendor_expectations.yaml planned) that
    classifies each drift as expected-vendor-mismatch vs codec defect.
    Phase 1's matrix is the input substrate for this work.

  Phase 4 [DEFERRED]: automated codec-defect ticket creation from
    the expected-vs-defect classification.

[TODO] EOS 4.26 EVPN/VXLAN GAPS — surfaced when user replayed
       karneliuk_a_eos1_eos4260.txt through arista_eos → juniper_junos
       and noticed dropped semantic surface.  Five separable items
       ordered smallest → largest:

  (1) [SHIPPED] ARISTA `router bgp / vlan N / rd ... /
      route-target both ...` → CanonicalRoutingInstance with
      `instance_type="mac-vrf"`.  Parser extends the existing
      `_parse_router_bgp` block-walker; render emits per-VLAN
      `vlan <vid>` blocks under `router bgp <asn>` for every
      mac-vrf routing-instance.  The L3 `vrf <name>` path
      continues to use `instance_type="vrf"` (unchanged).
      Critical depth check: the new `vlan <N>` matcher only fires
      at the 3-space top-level indent within router-bgp so it
      does NOT spuriously fire on nested `vlan-aware-bundle ...
      / vlan <range>` lines (a separate EOS form that is
      parse-and-ignore today).  See test_arista_eos
      TestBgpVlanMacVrf + the karneliuk EOS 4.26 real-capture
      assertion.

  (2) [SHIPPED] CanonicalVxlan source-interface + udp-port.
      CanonicalVxlan now carries {vlan_id, vni, mcast_group,
      flood_list, source_interface, udp_port}.  Arista + Junos
      parse + render fully wired; capability matrices on all 8
      codecs declare /vxlan-vnis/source-interface and
      /vxlan-vnis/udp-port (supported on arista_eos +
      juniper_junos; unsupported elsewhere).  Pattern documented
      in docs/adding-a-canonical-field.md ("Switch-level globals
      stamped onto every record").  See test_arista_eos
      TestVxlanSourceInterfaceUdpPort + test_juniper_junos
      TestVxlanSwitchOptions + cross-mesh round-trip cases.

  (3) [SHIPPED] IPv6 ADDRESSES on interfaces (GAP-EVPN-3).
      CanonicalIPv6Address (ip, prefix_length,
      scope='global'/'link-local') + CanonicalInterface.ipv6_addresses
      now wired through every bidirectional codec's parse + render.
      Link-local discriminator handles the keyword form on Cisco /
      Arista (``ipv6 address X link-local``); Junos / MikroTik /
      OPNsense infer scope from the fe80::/10 prefix.  Vendor-
      specific placeholders dropped on parse: FortiGate ``::/0``,
      OPNsense ``dhcp6`` / ``idassoc6``, AOS-S ``dhcp full``.
      Karneliuk EOS 4.26 fixture's
      ``ipv6 address fc00:192:168:100::62/64`` on Management1
      now survives parse end-to-end.  Cross-mesh: 72 new smoke
      tests cover every (source, target) pair preserving an IPv6
      address through the round-trip.  See test_ipv6_wire_through
      (36 unit tests) + test_cross_mesh_overrides
      (test_ipv6_address_survives_round_trip / _cross_mesh_render)
      + test_real_captures::test_ipv6_addresses_survive_real_capture_parse.

  (4) FULL BGP CANONICAL MODEL — CanonicalBGPProcess
      (multi-session).  Biggest of the five.  Schema needs:
      * autonomous_system: int
      * router_id: str
      * neighbors: list[CanonicalBGPNeighbor]
        - peer_address, remote_as, address_families,
          route_map_in / route_map_out
      * address_families: list[CanonicalBGPAddressFamily]
        - name (ipv4/ipv6/evpn/vpnv4), redistribute,
          neighbor_activations
      * redistribute_connected_route_map (str)
      Ship-before-wire on all DC codecs (Arista, NX-OS-future,
      Junos), then per-codec parse+render.  Pair with the
      CanonicalRoutingInstance work (#1 above) so RD/RT/router-id
      compose cleanly.

  (5) ROUTE-MAP + PREFIX-LIST CANONICAL MODEL (multi-session).
      Same scope as #4 — schema first, then per-codec wire-up.
      Without these, EVPN/MPLS migrations are perpetually
      losing route-policy data.  Decision deferred until #4
      ships (route-maps reference prefix-lists; need to choose
      whether they stay as separate canonical lists or fold
      under CanonicalBGPProcess.policies).

  See also: ALL FIVE produce parse-side bugs visible only when a
  bidirectional Arista or Junos render runs against a real EVPN
  fixture.  The cross-mesh smoke matrix (every bidir pair) does
  NOT detect these silently-dropped surfaces because it asserts
  "doesn't crash" not "round-trip preserves source semantic".  A
  generic guard could enforce: for every real-capture fixture,
  source_field_count must equal post-parse canonical-field-count
  (where field maps exist) — flagging silent drops when codec
  parse misses a documented field.

[SHIPPED] SNMPv3 USM CROSS-MESH (P2C6):
    # Fifth per-pane override category after ports / vlans /
    # local_users / snmp_community.  New canonical type:
    #   CanonicalSNMPv3User (name, group, auth_protocol,
    #   auth_passphrase, priv_protocol, priv_passphrase, engine_id)
    # New field: CanonicalSNMP.v3_users: list[CanonicalSNMPv3User].
    #
    # Codec wire-up (parse + render):
    #   * arista_eos — native `aes` / `aes256` single-token form
    #     + Cisco-pasted `aes 128` two-token tolerated on ingest
    #   * aruba_aoss — `snmpv3 user "X" auth {md5|sha} "K" priv
    #     {des|aes} "K"` + separate `snmpv3 group "G" user "X"
    #     sec-model ver3` binding line (merged by name, order-
    #     agnostic)
    #   * cisco_iosxe_cli — parse-only; `snmp-server user <n>
    #     <g> v3 auth <p> <k> priv <c> <keybits?> <k>`
    #   * fortigate_cli — `config system snmp user / edit <n>
    #     / set security-level / set auth-proto / set auth-pwd
    #     ENC <hash> / set priv-proto / set priv-pwd ENC <hash>`
    #   * juniper_junos — `set snmp v3 usm local-engine user <n>
    #     authentication-<p> authentication-key "<k>"` + matching
    #     `privacy-<c> privacy-key` + VACM `security-to-group`
    #     binding; multi-line parse merges by name
    #   * mikrotik_routeros — `/snmp community` section overloaded
    #     (v1/v2c communities vs v3 users disambiguated by
    #     `authentication-protocol=` presence)
    #
    # OPNsense + cisco_iosxe (NETCONF stub) declare /snmp/v3-user
    # Unsupported + add "snmpv3" to unsupported_rename_categories
    # for the UI compat banner.
    #
    # Orchestrator: netcanon/migration/canonical/snmpv3_user_names.py
    # Pipeline:  run_plan_with_overrides(snmpv3_user_rename_map=...)
    # Endpoint:  POST /api/v1/migration/plan/snmpv3
    # UI pane:   migrate.html + _partials/snmpv3-user-rename-table.js,
    #            rail button data-testid=migrate-rename-rail-snmpv3
    # Job fields: snmpv3_user_renames / snmpv3_user_drops /
    #             source_snmpv3_users
    #
    # Covered by tests/unit/migration/test_snmpv3_user_names.py (13),
    # test_snmpv3_wire_through.py (25), cross_mesh additions in
    # test_cross_mesh_overrides.py (+9), integration
    # test_migration_api.py TestPlanSnmpV3Endpoint +
    # TestPlanMultiCategoryRoutingSnmpV3 (+8).
    #
    # Rename semantic: identity-only.  auth / priv keys + group
    # + engine_id follow the renamed user; collisions merge on
    # first-wins (keys NEVER combined across users — hash-by-union
    # would produce incoherent crypto).

[CROSS-MESH VIABILITY AUDIT] MANAGEMENT-PLANE CATEGORIES:
    # Per-audit after P2C6 SNMPv3 ship: which other management-plane
    # surfaces already have per-codec parse+render coverage and are
    # therefore viable candidates for the same per-pane override
    # treatment.  None require new codec work — the canonical surface
    # already populates on parse; the work is orchestrator + pipeline
    # wiring + UI pane.
    #
    # Viable (ordered smallest → largest effort):
    #
    # [P2C7] NTP SERVER RENAME — list of IP / hostname strings on
    #   CanonicalIntent.ntp_servers.  Populated today by arista_eos,
    #   aruba_aoss, cisco_iosxe_cli, fortigate_cli, juniper_junos,
    #   mikrotik_routeros (6/8 codecs).  OPNsense + cisco_iosxe
    #   NETCONF don't populate — declare snmpv3-style Unsupported.
    #   Rename semantic: pure string rewrite (10.0.0.1 →
    #   ntp.corp.example).  Orchestrator ntp_server_names.py ~50
    #   LOC; pipeline param ntp_server_rename_map; UI pane
    #   ntp-rename-table.js; 1 session.
    #
    # [P2C8] DNS SERVER RENAME — list on
    #   CanonicalIntent.dns_servers.  Populated by all 7 non-stub
    #   codecs (8/8 minus cisco_iosxe NETCONF).  Effectively
    #   identical shape to NTP — same orchestrator template.  Can
    #   land in the same session as NTP if desired (two very small
    #   orchestrators + one common UI pattern).
    #
    # [P2C9] SYSLOG SERVER RENAME — list on
    #   CanonicalIntent.syslog_servers.  CURRENT CAVEAT: only
    #   juniper_junos populates on parse today (per the codec
    #   grep audit).  Before shipping the rename mesh, need to
    #   extend parse on the other five bidirectional codecs:
    #     - arista_eos: `logging host <ip>` / `logging vrf <n>
    #       host <ip>`
    #     - aruba_aoss: `logging <ip>` / `logging <ip> <severity>`
    #     - cisco_iosxe_cli: `logging host <ip>` / `logging
    #       server <ip>`
    #     - fortigate_cli: `config log syslogd setting / set
    #       server <ip>` (plus per-backend syslog-d1..d3
    #       distinct blocks)
    #     - mikrotik_routeros: `/system logging action` +
    #       target=remote with `remote=<ip>`
    #     - opnsense: `<syslog><destinations>` XML nesting
    #   Landing this is ~2 sessions: 1 session = parse+render
    #   wire-through on 5 codecs; 1 session = orchestrator + UI.
    #
    # [P2C10] SNMP TRAP-HOST RENAME — list on
    #   CanonicalSNMP.trap_hosts.  Populated by all 6 v3-capable
    #   codecs (same coverage set as v3 users; OPNsense's trap
    #   hosts ARE in the XML config.xml so it populates too).
    #   Rename semantic: string rewrite of IPs; drops remove the
    #   trap-host entry.  Orchestrator ~50 LOC; pipeline param
    #   snmp_trap_host_rename_map.  Same 1-session recipe as v2c
    #   community.
    #
    # [P2C11] RADIUS SERVER HOST RENAME — list of
    #   CanonicalRADIUSServer on CanonicalIntent.radius_servers.
    #   Populated by 6/8 codecs.  Rename surface is the .host
    #   field; .key passes through verbatim.  Drops remove the
    #   server entry; collisions merge on first-wins .key (keys
    #   never combined across hosts).  Shape = shipped SNMPv3
    #   user mesh minus the auth/priv complexity.  1 session.
    #
    # Not viable today (need codec work first):
    #
    # [BLOCKED] DHCP POOL RENAME — CanonicalDHCPPool has many
    #   fields (network, start_ip, end_ip, gateway, dns_servers,
    #   lease_time, domain_name).  Rename surface ambiguous
    #   ("rename a pool" could mean rewrite .interface, rewrite
    #   .network, rewrite .gateway, rewrite .dns_servers
    #   individually).  Would need a per-field sub-pane rather
    #   than a flat map — out of scope for the ports/vlans/
    #   local_users/snmp/snmpv3 three-step recipe.  Revisit when
    #   a concrete operator use case surfaces.
    #
    # [BLOCKED] TIMEZONE / DOMAIN RENAME — scalar fields
    #   (CanonicalIntent.timezone, .domain) but the cross-vendor
    #   rename use case is weak: timezone is usually corrected
    #   by refresh ("America/New_York" → "America/Los_Angeles"
    #   is a re-configuration, not a rename) and domain rename
    #   is rare enough it's not worth a pane.  Operators edit
    #   these server-side in their CMDB.
    #
    # Dependency ordering:  P2C7 + P2C8 are smallest (pure string-
    # list rename, no codec work); pair them in one session.
    # P2C10 + P2C11 are medium (canonical population already in
    # place).  P2C9 needs codec work first — ship the parse/render
    # wire-through session, THEN the orchestrator session.

[SHIPPED] STRUCTURAL APPLY-GROUPS COLLAPSE (interface-range):
    # Junos interface-range grammar: parse + render + auto-detect.
    #
    # Parse: `set interfaces interface-range <rname> ...`
    #   - member <iface> (collects members)
    #   - description / mtu / disable / unit 0 family inet address
    #     (shared attrs applied to each member at materialisation)
    #   - Per-member config overrides range-level defaults
    #   - Unknown sub-paths silently parse-and-ignore
    #
    # Top-level `set interfaces X mtu N` also wired (was previously
    # unparsed despite CanonicalInterface.mtu existing on the model).
    #
    # Render: auto-detect >=3 interfaces sharing identical
    # (mtu, description, enabled) tuples with at least one non-
    # default value; emit `set interfaces interface-range
    # AUTO-RANGE-<N>` blocks + suppress per-interface emission of
    # the shared attrs.  Skips VRF-bound / switchport / trunk /
    # access-vlan / sub-interface / all-default cases (richer
    # semantics warrant per-interface emission).
    #
    # Complementary to GAP 9b:
    #   * GAP 9b preserves OPERATOR-AUTHORED group structure
    #     (`set groups G` + `set apply-groups G`).
    #   * This commit synthesises interface-range blocks for raw
    #     per-interface sharing the operator didn't collapse.
    # Operators get compact Junos-native output on render.
    #
    # Covered by tests/unit/migration/test_junos_interface_range.py
    # (15 tests across parse / render / round-trip).

[SHIPPED] JUNOS APPLY-GROUPS STATEMENT + BODY PRESERVATION (GAP 9b):
    # Two new fields on CanonicalIntent:
    #   apply_groups: list[str]
    #     — operator-declared `set apply-groups <G>` statements
    #   group_content: dict[str, list[list[str]]]
    #     — per-group bucket of tokenised set-line tails,
    #     preserving operator-authored group bodies verbatim
    #
    # Parse: populates both fields from GAP 8's two-pass buckets.
    # Only groups that appear in apply_groups (i.e. actually
    # composed) get persisted — orphan groups drop.
    #
    # Render: emits `set groups <G> <body>` lines FIRST (quoting
    # each token with _quote_if_needed so multi-word values like
    # banner messages round-trip), then `set apply-groups <G>`
    # statements.  Operators who pasted a Junos config with group
    # structure see an equivalent structure in the rendered output.
    #
    # De-dup guards added alongside to prevent duplicate-accumulation
    # on re-parse (since group-content + top-level render both emit
    # the same data):
    #   _apply_routing_options: dedup on (destination, gateway)
    #   _apply_interfaces unit-N IPv4 addr: dedup on (ip, prefix)
    #
    # What's NOT in GAP 9b (deferred to future commit):
    #   * Synthesise groups from shared sub-configs — detecting
    #     N interfaces share the same MTU/description and
    #     collapsing them to a groups stanza.  Requires per-entity
    #     source-provenance metadata on canonical records that
    #     doesn't exist yet.
    #
    # Covered by TestApplyGroupsRenderPreservation in
    # test_junos_apply_groups_rich.py (8 tests including ksator
    # EX4550 real-fixture round-trip regression).

[SHIPPED] JUNOS BLOCK-FORM PARSE (GAP 9a):
    # Converts Junos curly-brace hierarchical config to set-form
    # via a grammar-agnostic recursive-descent walker, then feeds
    # the resulting set-lines through the existing parse pipeline.
    # No duplicated parse logic — one dispatcher, two input
    # grammars.
    #
    # New helpers (module scope):
    #   _looks_like_blockform(raw): heuristic detection (first
    #     meaningful line ends with `{` + content before it;
    #     `"key":` patterns rejected as JSON).
    #   _tokenise_blockform(raw): `{` / `}` / `;` as standalone
    #     tokens; strips /* ... */ comments; preserves quoted
    #     strings including escaped inner quotes.
    #   _blockform_to_setform(raw): walks the hierarchy, emits
    #     `set <path...> <value>` per leaf, raises ParseError on
    #     unbalanced braces.
    #
    # Removed: the v1-era rejection with helpful hint (GAP 4 era).
    # Operators no longer need `show configuration | display set`
    # to feed block-form output through Netcanon.
    #
    # Covered by TestBlockFormParse in test_juniper_junos.py
    # (10 tests including nested-5-deep, quoted strings, comments,
    # apply-groups in block-form, JSON rejection, unbalanced-brace
    # error).

[SHIPPED] RICHER JUNOS APPLY-GROUPS INHERITANCE (GAP 8):
    # Refactors Junos parse to a full two-pass model:
    #
    # Pass 1 (bucket): every set-line goes into top_level_lines or
    # group_lines[gname] based on whether it starts with `groups
    # <gname>`.  `apply-groups <gname>` accumulates in
    # applied_groups (first-declared wins order).
    #
    # Pass 2a (group apply): for each applied_group IN REVERSE
    # apply-groups order, replay each bucketed tokens list through
    # _dispatch_set.  Reverse iteration gives first-declared-wins
    # for scalars (Junos's first-match composition semantics).
    #
    # Pass 2b (top-level apply): dispatch top_level_lines last so
    # direct-intent scalars overwrite group inheritance.
    #
    # List-shaped fields (static_routes, local_users, dns_servers,
    # ntp_servers, syslog_servers, iface ipv4_addresses) accumulate
    # from both sources with de-dup.
    #
    # Before GAP 8, only `system host-name` was inherited (GAP 4
    # narrow path).  After GAP 8 the full dispatch surface flows:
    # system login user, system ntp/name-server/syslog, interfaces,
    # snmp community, routing-options static, routing-instances,
    # vlans — all populate canonical tree via apply-groups.
    #
    # Also added top-level parse for `set system domain-name /
    # name-server / ntp server [prefer] / syslog host`, which
    # appear at top level in some configs.  Render emits the
    # corresponding set-lines when canonical data is present.
    #
    # Covered by test_junos_apply_groups_rich.py (16 tests) +
    # real-capture regression (ksator QFX5100 + EX4550 now
    # populate hostname, user, SNMP, static routes, NTP, DNS,
    # syslog, and vme management interface — all previously empty
    # because everything lived under `set groups POC_Lab`).

[SHIPPED] ARISTA + JUNOS VXLAN / VRF / EVPN WIRE-UP (GAP 6):
    # Parses + renders CanonicalVxlan (VLAN-to-VNI mappings) +
    # CanonicalRoutingInstance (VRF declarations with RD + RTs +
    # L3 VNI) + CanonicalInterface.vrf (per-interface VRF
    # membership) on both arista_eos and juniper_junos codecs.
    #
    # Arista grammar wired:
    #   * `vrf instance <name>` (top-level) -> CanonicalRoutingInstance
    #   * `interface Ethernet X / vrf <name>` -> iface.vrf
    #   * `interface Vxlan1 / vxlan vlan X vni Y` -> CanonicalVxlan
    #   * `interface Vxlan1 / vxlan vrf X vni Y` -> ri.l3_vni
    #   * `router bgp <asn> / vrf <name> / rd <rd>` -> ri.route_distinguisher
    #   * `router bgp <asn> / vrf <name> / route-target both|import|export`
    #     -> ri.rt_imports / ri.rt_exports (strips `evpn ` prefix
    #     in `route-target import evpn <rt>` form)
    #
    # Junos grammar wired:
    #   * `set vlans <name> vxlan vni <vni>` (requires prior vlan-id)
    #     -> CanonicalVxlan
    #   * `set routing-instances <name> instance-type vrf` + RD +
    #     vrf-target (shared or split import/export) + interface +
    #     `protocols evpn ip-prefix-routes vni <N>` (-> ri.l3_vni)
    #   * Interface assignment post-pass resolves membership after
    #     _apply_interfaces materialises the CanonicalInterface.
    #   * Sub-interfaces with ``.0`` suffix collapse to parent in
    #     canonical naming; render re-adds ``.0`` to keep valid
    #     Junos grammar in set-form output.
    #
    # CanonicalRoutingInstance.l3_vni field added in GAP 6 (not
    # a separate ship-before-wire step — the only users are the
    # codecs landing in this commit).
    #
    # Demoted from Unsupported to Supported on both codecs:
    #   /vxlan-vnis/vni
    #   /routing-instances/instance
    # /evpn-type5-routes/route remains Unsupported with updated
    # reason — Type-5 per-prefix records aren't populated by any
    # codec; VRF-property modelling via l3_vni is the supported
    # path today.
    #
    # Arista-specific: interface Vxlan1 is NOT materialised as a
    # CanonicalInterface record (it's a VXLAN config container,
    # not a real interface).  The sub-commands dispatch via a
    # sentinel; render reconstructs the stanza from the canonical
    # data.  Also fixed: router-bgp parser now correctly handles
    # `!` sub-stanza separators (previously reset in_bgp mid-
    # stanza, dropping subsequent vrf blocks).
    #
    # Covered by tests/unit/migration/test_vxlan_evpn_wire_through.py
    # (35 tests) + real-capture round-trip on 3 EVPN/L3VPN fixtures
    # (arista 4.23 + junos 25.4 Type-5 + junos 25.4 L3VPN).

[SHIPPED] ARISTA + JUNOS CERTIFIED PROMOTION:
    # 3 new fixtures added to close the ≥3-fixtures-from-≥2-versions
    # certified bar for both vendors:
    #   tests/fixtures/real/arista_eos/batfish_duplicateprivate_eos4211.txt
    #     — EOS-4.21.1.1F vEOS BGP fixture (Apache-2.0)
    #   tests/fixtures/real/junos/batfish_evpntype5_router1_junos2541.set
    #     — Junos 25.4R1.12 EVPN-VXLAN leaf (Apache-2.0)
    #   tests/fixtures/real/junos/batfish_l3vpn_pe1_junos2541.set
    #     — Junos 25.4R1.12 MPLS L3VPN PE (Apache-2.0)
    # All from batfish/lab-validation @ d40faf6.
    #
    # arista_eos: best_effort → certified ✅ (3 EOS majors: 4.21/22/23)
    # juniper_junos: best_effort → certified ✅ (4 Junos majors: 15.1/17.3/18.4/25.4)
    #
    # No new bugs surfaced — all 9 new real-capture tests (3 per
    # fixture × 3 fixtures) pass on first run.
    #
    # Post-cert follow-up items (not gating):
    #   * Arista: newer LTS fixture (4.25+/4.27+/4.30+) nice-to-have
    #   * Both: GAP 6 codec wire-up for VXLAN + EVPN (currently
    #     parse-and-ignore; the 25.4 EVPN fixture + 4.23 EVPN leaf
    #     are the intended exercise surface)
    #   * Junos: GAP 8 richer apply-groups + GAP 9 v2b work

[SHIPPED] PER-UNIT 802.1Q VLAN TAGGING ON JUNOS SUB-INTERFACES (GAP 7):
    # Parse: `set interfaces <parent> unit <N> vlan-id <tag>` now
    # populates CanonicalInterface.access_vlan on the sub-interface
    # — semantically equivalent to Cisco's `encapsulation dot1Q N`.
    # Does NOT set switchport_mode (Junos sub-interfaces are L3 on
    # a tagged VLAN, not L2 access ports).
    #
    # Render: sub-interface emit path now emits
    # `set interfaces <parent> unit <N> vlan-id <tag>` when
    # access_vlan is populated.  A sub-interface carrying only a
    # vlan-id (no IP, no description) still emits content; the
    # bare-placeholder line is suppressed.
    #
    # unit 0 vlan-id N stores on the parent (uncommon but legal
    # Junos, consistent with v1's unit-0-collapses-into-parent).
    #
    # Covered by TestPerUnitVlanTagging in test_juniper_junos.py
    # (7 tests).

[SHIPPED] CANONICAL VRF / ROUTING-INSTANCE SCHEMA (GAP 5, ship-before-wire):
    # Adds CanonicalRoutingInstance + CanonicalInterface.vrf field +
    # top-level routing_instances[] list on CanonicalIntent.
    # Cross-vendor VRF primitive:
    #   * Cisco IOS / IOS-XE: `vrf definition <name>` with `rd` +
    #     `address-family ipv4 / route-target import/export`
    #   * Cisco NX-OS: `vrf context <name>` (same sub-commands)
    #   * Arista EOS: `vrf instance <name>` + `ip routing vrf <name>`
    #     (RD/RTs under `router bgp`)
    #   * Juniper Junos: `set routing-instances <name> instance-type
    #     vrf` + `route-distinguisher` + `vrf-target` + `interface`
    #
    # Per-interface VRF membership lives on CanonicalInterface.vrf
    # (back-pointer pattern, matches lag_member_of).
    # CanonicalRoutingInstance holds metadata only: name, instance_type
    # (vrf / virtual-router / l2vpn / mac-vrf), route_distinguisher,
    # rt_imports[], rt_exports[], description.
    #
    # No codec populates these in v1 — each DC codec's
    # CapabilityMatrix lists /routing-instances/instance under
    # `unsupported` until wired up.  Foundational for GAP 6
    # (EVPN Type-5 references a VRF) and GAP 8 (richer apply-groups
    # inheritance can carry per-iface VRF assignment).

[SHIPPED] JUNOS APPLY-GROUPS HOST-NAME + SUB-INTERFACES (GAP 4):
    # Two additions to the Junos codec:
    #
    # 1. Apply-groups host-name inheritance.  Junos allows host-name
    #    + other system scalars to live under a named `groups` stanza
    #    that `apply-groups` composes into the candidate config.
    #    The ksator QFX5100 + EX4550 fixtures both use this pattern;
    #    before GAP 4 their intent.hostname parsed as empty.  Parse
    #    now collects group-scoped host-names and resolves the first
    #    applied group that declared one.  Direct `set system
    #    host-name X` still wins over group fallback.  Bracketed
    #    `set apply-groups [ g1 g2 ]` form tolerated (bracket tokens
    #    filtered out).
    #
    # 2. Per-unit sub-interfaces (unit 1+).  v1 ignored units 1+;
    #    GAP 4 materialises them as distinct CanonicalInterface
    #    entries named `<parent>.<unit>` (e.g. `ge-0/0/0.100`) —
    #    matching Cisco's dot1Q convention so canonical-tree
    #    consumers see the same shape across vendors.  Render splits
    #    compound canonical name back into native Junos grammar
    #    (`set interfaces <parent> unit <N> ...`).  `irb.N` /
    #    `vlan.N` preserved as top-level SVI names (not mis-split)
    #    via a slash-requiring sub-interface regex.
    #
    # What's NOT in GAP 4 (deferred):
    #   * Routing-instances (VRF equivalent) — needs a canonical
    #     VRF model first.  Separate commit.
    #   * Richer apply-groups inheritance — currently only system
    #     host-name flows through.  Interface config / protocols /
    #     SNMP / RADIUS / etc. inheritance is future work.
    #   * Per-unit VLAN tagging (`unit N vlan-id 100`) — sub-
    #     interfaces materialise without tag info today.
    #   * Block-form (curly-brace) parse — reserved for a dedicated
    #     commit since the transformation is well-defined.
    #
    # Covered by tests/unit/migration/test_juniper_junos.py
    # (TestApplyGroupsHostname + TestSubInterfaces, 14 tests) plus
    # test_real_captures.py round-trip on the ksator fixtures with
    # populated hostname fields.

[SHIPPED] ARISTA + JUNOS BEST_EFFORT PROMOTION (GAP 3):
    # Adds 3 real-capture fixtures closing the ≥2-version gap for
    # both arista_eos and juniper_junos:
    #   tests/fixtures/real/arista_eos/batfish_labval_dc1_leaf2a_eos4230.txt
    #     — EOS-4.23.0.1F vEOS EVPN leaf (Apache-2.0, batfish/lab-validation)
    #   tests/fixtures/real/junos/ksator_labmgmt_qfx5100_junos173.set
    #     — Junos 17.3R1.10 QFX5100 (MIT, ksator/lab_management)
    #   tests/fixtures/real/junos/ksator_labmgmt_ex4550_junos151.set
    #     — Junos 15.1R6.7 EX4550 (MIT, ksator/lab_management)
    #
    # arista_eos: certainty experimental → best_effort (2 EOS majors)
    # juniper_junos: certainty experimental → best_effort (3 Junos majors)
    #
    # 2 real bugs surfaced + fixed:
    #   * Arista render dropped channel-group lines on LAG member
    #     Ethernet interfaces → round-trip lost all 5 MLAG Port-
    #     Channels on the EVPN fixture.  Regression tests:
    #     test_render_lag_member_emits_channel_group +
    #     test_render_lag_member_mode_normalised.
    #   * Junos render dropped bare interfaces (no description / no
    #     IP / enabled=True) → round-trip shrank interface count on
    #     the EX4550 fixture where every trunk port's only attributes
    #     were Tier-3 (ethernet-switching port-mode trunk + vlan
    #     members).  Regression test:
    #     test_render_bare_interface_emits_placeholder.
    #
    # Promotion to `certified` still requires:
    #   * Arista: ≥1 more real capture from a 3rd EOS major
    #     (4.25+ / 4.27+ / 4.30+) + GAP 1 EVPN-VXLAN codec wire-up
    #   * Junos: ≥2 more real captures from newer majors (20.x /
    #     21.x / 22.x / 23.x LTS) + GAP 4 routing-instances +
    #     apply-groups inheritance

[SHIPPED] JUNIPER JUNOS RENDER-SIDE v2a (flat set-form, GAP 2):
    # Promotes juniper_junos from direction=parse_only to
    # direction=bidirectional.  Render emits flat set-form commands
    # in deterministic order (system / login / interfaces / vlans /
    # routing-options / snmp) so repeated renders produce byte-
    # identical output.  Junos-style quoting: bare for simple tokens,
    # double-quoted for strings containing whitespace or shell
    # specials, backslash-escaped embedded quotes.  Hashes with
    # ``junos:`` vendor-tag prefix are stripped on render so
    # parse(render(tree)) is a true round-trip.
    #
    # What's NOT in v2a (deferred to v2b):
    #   * apply-groups inheritance detection + collapse
    #   * routing-instances (richer VRF semantics than Cisco ip vrf)
    #   * per-unit sub-interfaces (unit 1+)
    #   * block-form (curly-brace) RENDER (v2a emits set-form only)
    #
    # Covered by tests/unit/migration/test_juniper_junos.py
    # (render test classes added alongside existing parse tests)
    # plus test_real_captures.py round-trip stability on the
    # buraglio_netlab_junos184.set fixture (previously skipped).
    #
    # Codec stays at certainty=experimental — promotion to
    # best_effort still needs ≥2 more real captures (GAP 3).

[SHIPPED] EVPN-VXLAN CANONICAL SCHEMA (ship-before-wire, GAP 1):
    # Adds CanonicalVxlan + CanonicalEvpnType5Route to
    # netcanon/migration/canonical/intent.py plus vxlan_vnis[] +
    # evpn_type5_routes[] top-level Tier-2 lists on CanonicalIntent.
    # No codec populates them in v1.  Every DC-class codec (Arista
    # EOS, Juniper Junos, Cisco IOS-XE CLI) declares /vxlan-vnis/vni
    # and /evpn-type5-routes/route under CapabilityMatrix.unsupported
    # so the UI banner surfaces "VXLAN detected but not translated"
    # instead of silently dropping it.  Covered by
    # tests/unit/migration/test_vxlan_evpn_schema.py (31 tests
    # covering model construction, validation, round-trip through
    # model_dump, and capability-matrix declarations).
    #
    # Wire-up (per-vendor parse+render demoting Unsupported to
    # supported/lossy) lands on demand — see GAP 4 roadmap entry
    # for the natural bundle with Junos routing-instances.

[SHIPPED] MODULE-VARIANT TARGET PROFILES (hardware-awareness):
    # SHIPPED.  Option B (declarative `modules:` block on TargetProfile YAML)
    # was chosen and shipped; canonical schema lives in
    # `netcanon/migration/target_profiles.py` (`TargetProfile.modules:
    # dict[str, TargetModule]`); allowlist of module-variant profiles in
    # `tests/fixtures/module_variants.py`; reference YAML at
    # `definitions/target_profiles/cisco_c9300_24ux.yaml` (NM-8X / NM-2Q
    # variants).  Architecture doc in `ARCHITECTURE.md` "Module-variant
    # target profiles" section.  Authoring guide in
    # `docs/adding-a-target-profile.md`.
    #
    # The original problem statement + design space exploration is
    # preserved below for historical context.
    # ---
    # ORIGINAL PROBLEM STATEMENT (pre-ship):
    # PROBLEM
    # Target profiles currently bake ONE specific hardware config per
    # file.  All shipped Cat 9300 profiles assume the NM-8X (8× 10G
    # SFP+) network module.  Real Cat 9300 chassis accept ≥4 NM
    # modules, each with a different uplink inventory:
    #   C9300-NM-8X    8× 10G SFP+      (what we profiled)
    #   C9300-NM-2Q    2× 40G QSFP+     (MISSING — user's 40G case)
    #   C9300-NM-4G    4× 1G
    #   C9300-NM-4M    4× multigig
    #   C9300-NM-8M    8× multigig
    #
    # Same class of issue on:
    #   * Aruba 3810M 1-slot expansion — JL083A (4×10G), JL084A
    #     (4×40G QSFP+), JL085A (1×40G QSFP+)
    #   * Aruba 5400R / zl2 chassis (many blade options)
    #   * Cisco Cat 9500 NM slot variants
    #   * FortiGate 200F/300F+ SFP cage configs
    #
    # SYMPTOM (user-reported 2026-04-22)
    # Cat 9300 → Cat 9300-24UX translation: source config has
    # `FortyGigabitEthernet1/1/1-2` stanzas.  Shipped C9300-24UX
    # profile only has `TenGigabitEthernet1/1/1-8` uplinks (NM-8X
    # assumption).  FortyGig source rows get offered TenGig targets
    # in the dropdown — no 40G target available despite chassis
    # physically supporting NM-2Q.
    #
    # DESIGN SPACE — pick one before coding:
    #
    #   Option A — FLAT FILE PER (CHASSIS, MODULE) COMBO
    #     File: cisco_c9300_24ux_nm8x.yaml
    #           cisco_c9300_24ux_nm2q.yaml
    #           cisco_c9300_24ux_nm4g.yaml
    #           ... etc.
    #     Model IDs: C9300-24UX-NM-8X, C9300-24UX-NM-2Q
    #     Pros: simplest schema, UI unchanged, two-stage dropdown
    #           just gets more model entries
    #     Cons: ~7 chassis × ~4 modules = ~28 Cat 9300 files; more
    #           for chassis-based gear
    #     Scope: ~1-2 hours for initial Cat 9300 + 3810M coverage;
    #            tests that hardcode model keys need updates.
    #
    #   Option B — BASE PROFILE + INLINE MODULE VARIANTS (preferred
    #   long-term)
    #     Profile YAML gains top-level `modules:` dict keyed by
    #     module SKU, each entry with its own uplink port list:
    #       vendor: cisco_iosxe
    #       model: C9300-24UX
    #       ports:              # access ports (chassis-fixed)
    #         - {range: "GigabitEthernet1/0/1-24", kind: physical, ...}
    #       modules:
    #         NM-8X:
    #           description: "8x 10G SFP+ uplinks"
    #           ports:
    #             - {range: "TenGigabitEthernet1/1/1-8", kind: uplink, ...}
    #         NM-2Q:
    #           description: "2x 40G QSFP+ uplinks"
    #           ports:
    #             - {range: "FortyGigabitEthernet1/1/1-2", kind: uplink, ...}
    #         NM-4G:
    #           description: "4x 1G RJ45 uplinks"
    #           ports:
    #             - {range: "GigabitEthernet1/1/1-4", kind: uplink, ...}
    #     UI: third dropdown stage (Vendor → Chassis → Module).
    #     Pros: DRY, fewer files, module is first-class operator
    #           decision surfaced in UI.
    #     Cons: bigger schema + loader change + third dropdown +
    #           more test coverage; existing model-key test fixtures
    #           need migration
    #     Scope: ~3-4 hours.
    #
    #   Option C — BASE CHASSIS + OVERLAY MODULE FILES
    #     Two files merged at load time: base chassis + overlay.
    #     Maximum DRY, but merge semantics + filename discovery
    #     rules add complexity.  Probably don't do this.
    #
    # SECONDARY CONSIDERATIONS
    #   * "I don't know what module is installed" fallback — default
    #     to "any uplink" / no module gating to avoid blocking users
    #     who haven't opened the chassis yet.
    #   * Speed mismatch UX — source TenG vs target with only 40G
    #     uplinks.  Offer anyway (operator may downshift) + warn, or
    #     filter out?  Probably offer with a `(speed downshift)`
    #     suffix.  Don't hard-block — operator judgment.
    #   * Test-suite touchpoints for Option A (hardcoded model keys):
    #       tests/unit/migration/test_target_profiles.py:227,240
    #       tests/integration/test_migration_target_profiles_api.py:
    #         31,32,36,41,119,164
    #       tests/e2e/test_migrate_rename_modal.py:276,288
    #   * Aruba parity — if renaming Cat 9300 profiles, do the same
    #     for 3810M + add the 40G QSFP+ module variant (JL084A).
    #
    # IMMEDIATE SMALLEST WIN (if picking Option A):
    #   1. Rename 7 Cat 9300 profiles to -NM-8X suffix in model/file
    #   2. Add cisco_c9300_24ux_nm2q.yaml + cisco_c9300_48uxm_nm2q.yaml
    #      (2× 40G QSFP+ uplinks each)
    #   3. Add aruba_3810m_48g_poep_qsfp.yaml (JL084A variant)
    #   4. Update the 9-ish test references to new model keys
    #   Profile authoring = trivial YAML; the commit is mostly
    #   renames + 3 new files + test update.
    #
    # RECOMMENDATION: start with Option A for immediate unblock (the
    # 40G case is a real user report), revisit Option B as the
    # module-choice UX gets refined.  The profile-key rename is the
    # riskiest part — do it behind tests or tolerate a single-commit
    # renaming of 28 references.

[ ] SAVE AS PROFILE (operator convenience):
    # Persist an operator's rename decisions as
    # `profiles/<source_vendor>_<target_vendor>_<label>.json` and
    # offer "Load profile" dropdown in the modal toolbar.  Operators
    # doing repeated Cisco→Aruba migrations reuse their mappings
    # without re-entering them each time.
    Where: netcanon/migration/target_profiles.py (or a new
           netcanon/migration/rename_profiles.py if preferred) —
           YAML or JSON, user-writable (under configs_dir so it
           survives reinstall) + API endpoints POST/GET/DELETE
           /api/v1/migration/rename-profiles/<key>.
    Scope: ~1 day; UI adds two buttons (Save As, Load) in the modal
           toolbar; backend adds a small persistence layer.

[ ] HARDWARE FIT-CHECK:
    # When source config has more ports than the selected target
    # profile, surface an aggregate "N source interfaces can't fit
    # on target X" banner in the modal header.  Already have the
    # data (profile.port_count + source intent.interfaces length);
    # just need to compute + render.
    Scope: ~15 min in the modal toolbar; pairs nicely with target
           profile selection.

[ ] 3-PANE LAYOUT OPTION:
    # Current modal is 2-pane (table + preview).  Route C from the
    # brainstorm discussion added a 3rd pane showing source config
    # with port names highlighted-and-clickable (click → scroll
    # table row).  Useful on wide monitors; optional via a toolbar
    # toggle.
    Scope: ~2 hours of frontend; backend unchanged.

[ ] MORE TARGET PROFILES:
    # Ship profiles for more common target hardware so the
    # profile-driven dropdown covers real use cases out of the box.
    # Each profile is pure YAML authoring (no code change).
    Target list: Cisco 9200/9500 series, Aruba 3810M / 6300M,
    MikroTik CCR / CRS line, FortiGate 40F/60F/100E/200F, OPNsense
    Deciso A10/A20/A30.

[ ] RENAME-OPERATION DIFF:
    # Show an inline diff of rendered output BEFORE vs AFTER the
    # user's rename map, instead of a full re-render in the preview
    # pane.  Easier to spot what actually changed.
    Scope: ~1 hour; reuses the existing compute_diff infrastructure.

=====================================================================
## UPDATES SINCE INITIAL PLANNING
=====================================================================
# Several capabilities the plan assumed would need to be built now already
# exist in the repo.  Reuse, don't rebuild.  Summary in priority order:

EXISTING, REUSE DIRECTLY (do not reinvent):
  * netcanon/services/diff.py
      - check_compatibility(left, right) -> CompatibilityReport
      - compute_diff(left, left_text, right, right_text) -> DiffReport
      - fold_context(lines, context=3) -> list[DiffGroup]  (collapsed runs)
    Pipeline stage 6 (`diff`) MUST call compute_diff for the text side.
    Tree-level semantic diff is the NEW thing to add; text diff is solved.
  * netcanon/models/diff.py
      - DiffLine, DiffGroup, CompatibilityReport, DiffReport, DiffRequest
    ValidationReport shape planned for migration should mirror
    CompatibilityReport's (compatible/severity/reasons) — same pattern.
  * netcanon/services/ package
      - Convention established: stateless, no I/O, pure functions.
      - Put migration engine stages HERE, not in netcanon/migration/engine/.
      - Revised package layout is below (§4 updated).
  * netcanon/security/migration.py
      - migrate_credential_fields() — shared helper for detect-legacy-plaintext.
      - Reuse for any future credential migration in adapter code.
  * netcanon/templates/base.html — client helpers
      - _cvRenderHighlighted(text, ext) — language-aware syntax highlighter
        (.tok-keyword, .tok-string, .tok-ip, .tok-number, .tok-tag,
         .tok-attr, .tok-comment classes).  Migration UI diff viewer
        reuses this verbatim — do NOT ship a second highlighter.
      - Toast system: window.showToast(msg, 'info'|'error'|'success').
      - startJobProgress(jobId) + netcanon:job-{started,progress,
        complete,dismissed} CustomEvents — the migration UI should adopt
        the same pattern for MigrationJob progress.
      - Config viewer modal with in-modal cross-span search (see
        _cvSearch / _cvWrapSpan).  Can be reused to preview rendered
        output before deploy.
  * netcanon/templates/diff.html + /configs/{left}/vs/{right} route
      - Already uses FROM→TO directional paradigm (not "sides").
      - Context folding via <template> sibling + click-to-expand.
      - Migration stage 6 UI should LINK into this URL with
        left=<snapshot>, right=<rendered> rather than ship a second
        diff viewer.
  * Storage layer
      - All 4 stores (file, job, schedule, device_profile) now do atomic
        write (temp+rename).  Snapshot stage 8 gets this for free.
      - FileConfigStore.resolve_path has path-traversal + symlink
        defences.  Pre-deploy snapshots reuse the existing store.
  * JobStatus enum now has `partial` terminal state (plan already
    anticipated this lesson — confirmed and in production).
  * Settings.backup_concurrency (int in [1,10], default 10, thread-pool
    backed).  MigrationJob's parallel-collect should follow the same
    idiom: new setting, capped, pool-backed, pinned to 1 in tests.
  * data-testid discipline + tests/testid_reference.md
      - Migration UI testids MUST be added to the reference on landing,
        per AGENTS.md rules.

NEW INTERACTIONS (plan needs to wire these in):
  * Phase 0 now starts against a codebase where pydantic + FastAPI +
    storage + testing patterns are fully settled.  Adapter contract
    (§5) should import BaseModel and mirror DiffReport's style.
  * Pipeline stage 8 (snapshot) precondition updated: FileConfigStore
    already atomic; the remaining concern is filename collision with
    same-second backups, which the existing collision-counter handles.
  * Diff stage (6) now has TWO layers: textual (reuse services/diff.py)
    and semantic (new: tree-level xpath diff over the YANG canonical).
    Keep them as SEPARATE functions in services/migration_diff.py so
    the text-level view never depends on YANG being parseable.
  * NEW PIPELINE STAGE 0 — device-class guard.  CapabilityMatrix now
    has a device_classes: list[DeviceClass] field (enum: switch,
    router, firewall, load_balancer, wireless_controller, access_point,
    waf).  services.migration_validate.check_class_compat enforces
    non-empty-intersection; run_plan refuses mismatched pairs before
    parse unless force=True.  An "uncommitted" adapter (no classes
    declared) gets warn severity, not block.  Update the Phase 1
    adapter onboarding checklist: every new adapter MUST declare its
    device_classes in capabilities.yaml.

ABANDONED / DEFERRED FROM ORIGINAL PLAN:
  * No changes to abandoned items.  PHASED ROADMAP (§12) still stands;
    phase 0 scope slightly narrowed (see §12 update below).

=====================================================================
## 0. PROBLEM, SCOPE, NON-GOALS
=====================================================================

PROBLEM
  Extend Netcanon (currently a multi-vendor backup tool) into a config
  migration engine: read source device config (any vendor), transform
  into a canonical intent tree, emit config for a different target
  vendor, report semantic loss, optionally deploy.

SCOPE (IN)
  - Canonical intent model = YANG (OpenConfig + Netcanon extensions).
  - Per-vendor adapters with parse(raw)->YANG and render(YANG)->raw.
  - Capability matrices drive pre-flight loss reports.
  - Reuse existing collectors (SSH) for ingest.
  - FastAPI surface + Jinja UI parity (web + desktop per AGENTS.md).

SCOPE (OUT — explicitly non-goals for v1)
  - Live network-state migration (BGP sessions, IPsec SAs, ARP tables).
  - Licensed-feature translation (Cisco DNA, FortiGuard, Panorama).
  - Policy-intent synthesis (i.e. "draw me a ZTP plan from nothing").
  - Cross-vendor routing-protocol convergence guarantees.

HARD CONSTRAINTS
  - Desktop + web feature parity (AGENTS.md rule).
  - data-testid on every new interactive UI element (AGENTS.md rule).
  - No terminal length 0 for Cisco (AGENTS.md rule — collector layer).
  - Mock get_collector in tests, never ConnectHandler (AGENTS.md rule).
  - Atomic file writes pattern already adopted; reuse for snapshots.

=====================================================================
## 1. YANG VENDOR REALITY — FROZEN REFERENCE TABLE
=====================================================================
#  Do not re-research. Treat as authoritative for planning. Revisit only
#  if a specific vendor release note changes support.

| vendor            | native_yang | canonical                          | notes                                         |
|-------------------|-------------|------------------------------------|-----------------------------------------------|
| cisco_iosxr       | yes         | Cisco-IOS-XR-* + openconfig        | first-class netconf/restconf                  |
| cisco_iosxe       | yes (16.3+) | Cisco-IOS-XE-* + partial oc        | older CLI features lag YANG                   |
| cisco_nxos        | partial     | Cisco-NX-OS-*                      | uneven CLI parity                             |
| juniper_junos     | yes         | junos-conf-*                       | comprehensive via RPCs                        |
| arista_eos        | yes         | openconfig + arista-*              | best OC fidelity                              |
| nokia_sros        | yes         | nokia-conf-*                       |                                               |
| huawei_vrp        | yes         | huawei-* + partial oc              |                                               |
| aruba_cx          | yes         | aruba-* + REST                     |                                               |
| f5_bigip          | partial     | F5-bigip-*                         | iControl REST is primary                      |
| fortigate         | NO          | CLI (config/set/edit/next/end)     | REST JSON secondary                           |
| mikrotik_routeros | NO          | /export text + REST                | no YANG exists                                |
| opnsense/pfsense  | NO          | XML config.xml                     | BSD; PHP+shell underneath                     |
| palo_alto_panos   | NO          | XML API                            |                                               |
| checkpoint        | NO          | mgmt API (JSON)                    |                                               |
| sonicwall         | NO          | CLI + REST                         |                                               |
| extreme_exos      | partial     | extreme-*                          |                                               |

NETCONFIG CURRENT SUPPORT (definitions/*.yaml):
  cisco_iosxe, fortigate, mikrotik_routeros, opnsense
  => only 1 of 4 is YANG-native. Architecture must bridge the rest.

OPENCONFIG COVERAGE: ~60-70% of typical enterprise feature surface on
  YANG-native vendors. Zero coverage on non-YANG vendors.

=====================================================================
## 2. LAYERED ARCHITECTURE
=====================================================================

  ┌────────────────────────────────────────────────────────────┐
  │ MIGRATION ENGINE                                            │
  │   collect → parse → transform → validate → render → diff   │
  │                                             → deploy → snap │
  └──────────────┬────────────────────────────┬───────────────┘
                 │                            │
                 ▼                            ▼
  ┌──────────────────────────┐  ┌────────────────────────────┐
  │ Canonical Intent Tree    │  │ Capability Matrix          │
  │ openconfig + netcanon-  │◀─┤ per-adapter YAML           │
  │ ext YANG modules         │  │ supported/lossy/unsupported│
  └─────┬──────────┬─────────┘  └────────────────────────────┘
        ▲          │
  parse │          │ render
        │          ▼
  ┌─────┴────────────────────────────────────────────────────┐
  │ VENDOR ADAPTER LAYER                                      │
  │   yang-native | text/cli | xml/json                       │
  │   all expose: Adapter.parse, Adapter.render, .capabilities│
  └──────▲───────────────────────────────────────┬───────────┘
         │ collect (reuse)                       │ deploy (new)
         │                                       ▼
    SOURCE DEVICE                            TARGET DEVICE

=====================================================================
## 3. CANONICAL INTENT MODEL (CIM)
=====================================================================

BASE: openconfig/public master @ pinned commit
  - openconfig-interfaces
  - openconfig-vlan
  - openconfig-network-instance
  - openconfig-bgp, openconfig-ospf, openconfig-isis
  - openconfig-access-control-list
  - openconfig-if-ethernet, -if-ip, -if-aggregate

EXTENSION: netcanon-ext.yang (in-repo)
  - firewall/rule (ordered-by user, stateful session model)
  - nat/source, nat/destination
  - captive-portal/*
  - dashboard-widget/* (opnsense)
  - license-tag (opaque string blob for non-migratable licensed config)

TOOLING
  python_libs:
    - libyang       # validation
    - pyangbind     # python class bindings from YANG
    - ncclient      # NETCONF transport (phase 2+)
    - lxml          # XML parsing (opnsense, pan-os)
  cli:
    - pyang         # schema linting in CI

RULE: every tree produced by an adapter OR transform MUST pass
      libyang validation before it leaves its function boundary.
      This is the top-level invariant; enforce with a decorator.

=====================================================================
## 4. PACKAGE STRUCTURE  (revised to match existing conventions)
=====================================================================
# Convention now established in the repo:
#   netcanon/models/          — pydantic request/response/domain types
#   netcanon/services/        — stateless pure-function engines (I/O-free)
#   netcanon/api/routes/      — thin HTTP adapters around services
#   netcanon/templates/       — Jinja2, share helpers via base.html
# Migration work follows the same split.  Adapters + canonical schemas
# ARE an ecosystem of their own, so those live under netcanon/migration/
# (filesystem-scoped), but the engine stages live under netcanon/services/
# alongside diff.py to keep the rule "services/ = pure Python" consistent.

netcanon/
  migration/                   # ecosystem-of-adapters-and-schemas namespace
    __init__.py
    canonical/
      loader.py                # libyang ctx singleton; loads modules once
      schema/                  # openconfig/public pinned + netcanon-ext
        openconfig-*.yang
        netcanon-ext.yang
      validate.py              # @validates_yang decorator
    adapters/
      base.py                  # AdapterBase, CapabilityMatrix, exceptions
      registry.py              # entry-point discovery
      cisco_iosxe/
        __init__.py            # register(Adapter)
        parser.py              # NETCONF get-config OR CLI state machine
        renderer.py            # NETCONF edit-config OR Jinja
        capabilities.yaml
        templates/             # only if CLI-emitting path used
      fortigate/
        __init__.py
        parser.py              # CLI state machine (TextFSM or PEG)
        renderer.py             # Jinja-only
        capabilities.yaml
        templates/
          interfaces.j2
          vlans.j2
          firewall.j2
          routing/{bgp,ospf,static}.j2
      mikrotik/
        # parser: /export line tokenizer; renderer: Jinja
      opnsense/
        # parser: lxml of config.xml; renderer: lxml builder or Jinja
    transforms/
      __init__.py              # register(transform_fn); entry-point discovery
      rename_interfaces.py
      remap_vlans.py
      rebase_ip_network.py
      scrub_credentials.py
      strip_unsupported.py
  services/                    # PURE functions — already established by diff.py
    diff.py                    # EXISTING — textual diff (reused by stage 6)
    migration_pipeline.py      # NEW: stages orchestrator
    migration_diff.py          # NEW: TREE-level (semantic) xpath diff only;
                               #       text diff comes from services/diff.py
    migration_deploy.py        # NEW: netconf/ssh/rest deploy strategies
    migration_snapshot.py      # NEW: wraps existing FileConfigStore.save()
    migration_validate.py      # NEW: capability-matrix -> ValidationReport
  models/
    diff.py                    # EXISTING — DiffLine, DiffGroup, etc.
    migration.py               # NEW: MigrationJob, MigrationJobStatus,
                               #       TransformSpec, ValidationReport
  api/
    routes/
      migration.py             # NEW: /api/v1/migration/*

tests/
  unit/
    test_canonical_loader.py
    test_adapters_{iosxe,fortigate,mikrotik,opnsense}.py
    test_transforms_*.py
    test_migration_validate.py
    test_migration_pipeline.py # stage-by-stage contracts
    test_migration_diff.py     # tree-level diff only (text diff already covered)
  integration/
    test_migration_api.py
    test_roundtrip_*.py        # parse(render(tree)) == tree
  fixtures/
    canonical/                 # seed YANG trees (JSON) per feature
    raw/                       # sample vendor configs (sanitized)

=====================================================================
## 5. ADAPTER CONTRACT
=====================================================================

# base.py
from abc import ABC, abstractmethod
from pathlib import Path
from pydantic import BaseModel

class CapabilityMatrix(BaseModel):
    adapter: str
    version_range: str
    supported: list[str]            # YANG xpath expressions
    lossy: list[LossyPath]
    unsupported: list[UnsupportedPath]

    def classify(self, xpath: str) -> Literal["supported","lossy","unsupported"]: ...
    def diff(self, tree) -> CapabilityDiff: ...

class AdapterBase(ABC):
    name: str                       # e.g. "cisco_iosxe"
    version_hint: str | None        # for version-aware parsing

    @property
    @abstractmethod
    def capabilities(self) -> CapabilityMatrix: ...

    @abstractmethod
    def parse(self, raw: str) -> "YangTree": ...     # raises ParseError
                                                     # MUST validate before return

    @abstractmethod
    def render(self, tree: "YangTree") -> str: ...   # raises RenderError
                                                     # MUST accept validated tree only

    # Optional: only YANG-native adapters override
    def deploy(self, device, rendered: str) -> DeployResult: ...
    def collect_native(self, device) -> "YangTree": ...   # bypass parse()

DISCOVERY
  # pyproject.toml
  [project.entry-points."netcanon.migration.adapters"]
  cisco_iosxe = "netcanon.migration.adapters.cisco_iosxe:Adapter"
  fortigate   = "netcanon.migration.adapters.fortigate:Adapter"
  mikrotik    = "netcanon.migration.adapters.mikrotik:Adapter"
  opnsense    = "netcanon.migration.adapters.opnsense:Adapter"

=====================================================================
## 6. CAPABILITY MATRIX SCHEMA (YAML)
=====================================================================

# adapters/<vendor>/capabilities.yaml
adapter: opnsense
version_range: ">=25.1,<27"
supported:
  - openconfig-interfaces:/interfaces/interface
  - openconfig-vlan:/vlans/vlan
  - netcanon-ext:/firewall/rule
  - openconfig-if-ip:/.../ipv4/addresses
lossy:
  - path: openconfig-network-instance:/.../bgp
    reason: "FRR package required; no route-reflector support"
    severity: warn                  # warn | error
  - path: openconfig-if-ethernet:/.../config/auto-negotiate
    reason: "set via pfctl script; not round-tripped in config.xml"
    severity: warn
unsupported:
  - path: openconfig-mpls:/mpls
    reason: "BSD kernel lacks MPLS data plane"
  - path: openconfig-network-instance:/.../protocols/isis

severity semantics:
  warn  -> ValidationReport lists; user confirms to proceed
  error -> migration refuses unless --force

=====================================================================
## 7. PIPELINE STAGES — I/O CONTRACTS
=====================================================================

1. collect(device, credentials) -> raw_text: str
   # reuses existing netcanon.collectors; no changes

2. parse(raw_text, source_adapter) -> YangTree
   # validation: libyang before return
   # errors: ParseError(path, line, snippet)

3. transform(tree, transforms: list[TransformSpec]) -> YangTree
   # each transform: fn(tree, **kwargs) -> tree
   # ordered application; each output re-validated

4. validate(tree, target_adapter) -> ValidationReport
   # ValidationReport:
   #   supported_paths: list[str]
   #   lossy_paths: list[(path, reason, severity)]
   #   unsupported_paths: list[(path, reason)]
   #   blocked: bool
   # never mutates tree

5. render(tree, target_adapter) -> raw_text: str
   # precondition: validated tree; capability-filtered if strip_unsupported ran
   # errors: RenderError(yang_path, reason)

6. diff(src_raw, tgt_raw, src_tree, tgt_tree) -> MigrationDiffReport
   # Two layers, independent:
   #   TEXTUAL: reuse netcanon.services.diff.compute_diff(src_raw,
   #            tgt_raw). Already handles CompatibilityReport,
   #            context folding via fold_context(), and the FROM/TO
   #            directional paradigm.
   #   SEMANTIC: new netcanon.services.migration_diff.tree_diff(
   #            src_tree, tgt_tree) -> list[XPathDelta].
   #            Returns {added, removed, changed} xpath lists plus
   #            risk_flags (e.g. "firewall rule order changed",
   #            "interface renamed across IP / ACL / LAG references").
   # MigrationDiffReport composes both:
   #   {textual: DiffReport, semantic: list[XPathDelta], risk_flags: list[str]}
   # UI hook: link into /configs/{snapshot}/vs/{rendered} for the
   #          textual view (already implemented, DON'T build a second).

7. deploy(target_device, rendered) -> DeployResult
   # three strategies, chosen by adapter:
   #   netconf  -> ncclient <edit-config>
   #   ssh_cli  -> netmiko send_config_set
   #   rest     -> httpx POST/PUT
   # precondition: snapshot taken (stage 8 runs BEFORE this)

8. snapshot(target_device, pre_config) -> ConfigRecord
   # Reuses FileConfigStore.save() which is now atomic (temp+rename,
   # see netcanon/storage/file_store.py).  Collision-counter handles
   # the same-second-same-host case.  Filename convention:
   #   "<device_type>_<safe_host>_<YYYYMMDD_HHMMSS>.<ext>"
   # Same pattern as backups so both coexist in configs/ and the
   # diff UI can compare pre- and post-migrate snapshots with zero
   # extra plumbing.

ORCHESTRATOR
  MigrationJob state machine (reuse BackupJob pattern):
    pending -> parsing -> transforming -> validating
            -> rendering -> diffing -> (awaiting_approval?)
            -> snapshotting -> deploying -> (completed | failed | partial)
  Enforce via property setter (same lesson as JobStatus partial work).

=====================================================================
## 8. TRANSFORMS — BUILT-INS
=====================================================================

signature: fn(tree: YangTree, **kwargs) -> YangTree
contract:  pure; idempotent when inputs identical; re-validated after

built-ins (netcanon/migration/transforms/*):
  rename_interfaces(mapping: dict[str,str])
    # "Gi0/1" -> "ether2", etc. Updates name refs across tree
    # (L3 config, ACLs, VLAN memberships, LAG members).

  remap_vlans(mapping: dict[int,int])
    # bulk renumber. Enforces uniqueness post-remap.

  rebase_ip_network(old_cidr, new_cidr)
    # shifts all addresses in old_cidr into new_cidr preserving host bits.
    # Rejects if new_cidr too small.

  scrub_credentials()
    # replaces password/pre-shared-key/community with $PLACEHOLDER$.
    # used for diff-safe rendering in PRs.

  strip_unsupported(target_adapter)
    # auto-removes every path marked unsupported in target caps.
    # emits a log + report entry per stripped path.

discovery: entry-point "netcanon.migration.transforms"

=====================================================================
## 9. TEMPLATE AUTHORING RULES (Jinja2)
=====================================================================

Templates only exist for CLI/XML targets. YANG-native targets use
NETCONF <edit-config> generated directly from the tree.

RULES (enforce in code review + template linter):
  1. PURE: no I/O, no network, no datetime.now(), no randomness.
  2. DETERMINISTIC ORDER: sort all iterables by a stable key.
  3. ONE FEATURE PER FILE: interfaces.j2, vlans.j2, bgp.j2, ...
  4. NO SILENT TRUNCATION: explicit `| truncate(N, True, '')`; not
     implicit.
  5. NO INLINE BUSINESS LOGIC: push logic into filters registered
     in renderer.py.
  6. VENDOR FILTERS: per-adapter filter library
     (e.g. fortigate_netmask, cisco_ifname).
  7. TEST: each template has a fixture pair (tree.json -> expected.txt).

EXAMPLE SKELETON (fortigate/interfaces.j2):
  config system interface
  {%- for iface in tree.interfaces.interface | sort(attribute='name') %}
      edit "{{ iface.name }}"
          set vdom "{{ iface.config.vdom | default('root') }}"
          {%- if iface | has_ipv4 %}
          set ip {{ iface | primary_ipv4 | netmask_form }}
          {%- endif %}
          {%- if iface.config.description %}
          set alias "{{ iface.config.description | truncate(25, True, '') }}"
          {%- endif %}
      next
  {%- endfor %}
  end

=====================================================================
## 10. HARD PROBLEMS — KNOWN, DO NOT REDISCOVER
=====================================================================

SEMANTIC-NOT-STRUCTURAL EQUIVALENCE
  ACLs / policies / pf rules express similar intent but with different
  primitives (address objects, zones, sessions). Canonical model picks
  ONE lane (netcanon-ext:/firewall) and each adapter commits to a
  best-effort mapping. Expect `lossy` entries in capabilities.

RULE ORDERING
  Firewall rules and route-maps are ORDERED. YANG `ordered-by user`
  only preserves order if the whole transport chain preserves it.
  JSON and some XML libs silently re-sort. TEST ordering explicitly
  in every adapter's roundtrip suite.

INTERFACE NAMING
  No canonical name exists — "GigabitEthernet0/0/1", "ge-0/0/1",
  "ether1", "em0", "port1" all refer to "port with a role".
  Model uses opaque role IDs + per-adapter name tables.
  rename_interfaces transform bridges names.

STATEFUL / DYNAMIC STATE
  BGP peer state, IPsec SAs, HA sync, license keys — not migratable.
  Capability matrix MUST flag these as unsupported. No silent drop.

LICENSED FEATURES
  FortiGate SD-WAN profiles, Palo URL categories, Cisco DNA tags.
  Mark unsupported with severity=warn (render as comment stub) or
  error (block).

ROUND-TRIP INVARIANT
  parse(render(tree)) == tree  for the supported subset.
  This is THE most valuable test. Build it first per adapter.

RUNTIME VS CONFIG
  Netcanon backs up config, not runtime. Migration similarly operates
  on config-intent only. Do not try to infer runtime from config.

=====================================================================
## 11. HTTP API SURFACE
=====================================================================

GET  /api/v1/migration/adapters
  -> list of {name, version_range, native_yang, template_count}

GET  /api/v1/migration/adapters/{name}/capabilities
  -> full CapabilityMatrix

POST /api/v1/migration/plan
  body: {source_device_id, target_adapter, transforms: []}
  runs: collect -> parse -> transform -> validate
  -> ValidationReport (no render, no deploy)

POST /api/v1/migration/render
  body: same as /plan
  runs: ...+ render
  -> {rendered_config: str, validation_report, diff: optional}

POST /api/v1/migration/deploy
  body: {migration_job_id, confirm: true}
  precondition: a prior /render produced a job in awaiting_approval
  runs: snapshot -> deploy
  -> DeployResult

GET  /api/v1/migration/jobs
GET  /api/v1/migration/jobs/{id}
  # reuse BackupJob persistence pattern (FileJobStore sibling)

UI (/migrate page):
  1. Pick source device profile.
  2. Pick target (device profile OR "download only").
  3. Add transforms (rename/remap wizards).
  4. Preview ValidationReport with supported/lossy/unsupported buckets.
     Reuse CompatibilityReport's severity pattern (ok/warn/block) +
     banner styling from diff.html.
  5. Render → snapshot pre-deploy target → redirect to
     /configs/{pre-deploy}/vs/{rendered} for review.  This REUSES the
     existing diff page (FROM=snapshot, TO=rendered output) verbatim;
     no second viewer needed.  Semantic-diff deltas appear as a
     separate banner above the textual diff (new block in diff.html
     under conditional `{% if migration_delta %}`).
  6. Confirm deploy (disabled if validation blocked, or for
     "download only").

PROGRESS PANEL:
  Reuse the existing startJobProgress(jobId) pattern from base.html
  for MigrationJob lifecycle.  Emit netcanon:migration-{started,
  stage-changed,complete,failed} CustomEvents so page-level code can
  react without re-polling.  Panel rendering can be forked from the
  existing one if stage-level detail is needed, or reused as-is if
  a simple % complete suffices for MVP.

TESTIDs (follow project convention; add to testid_reference.md):
  migrate-source-select, migrate-target-select,
  migrate-transforms-list, migrate-add-transform-btn,
  migrate-validation-report, migrate-lossy-item, migrate-unsupported-item,
  migrate-render-btn,
  # migrate-diff-viewer NOT NEEDED — the /configs/{L}/vs/{R} route owns this.
  # Add these instead if a summary overlay is introduced:
  migrate-semantic-delta-banner, migrate-semantic-delta-item,
  migrate-deploy-btn, migrate-confirm-deploy-btn

=====================================================================
## 12. PHASED ROADMAP
=====================================================================

PHASE 0 — PROVE THE SHAPE  (SHIPPED)
  goal:       adapter contract + mock reference round-trip
  deliverables shipped:
    - AdapterBase + CapabilityMatrix (netcanon/migration/adapters/base.py)
    - Adapter registry (netcanon/migration/adapters/registry.py)
    - MockAdapter reference (exercises every classify() branch)
    - netcanon/services/migration_pipeline.py skeleton
    - netcanon/services/migration_validate.py (severity aggregation)
    - netcanon/models/migration.py: full type set
    - DeviceClass enum + cross-class guard (stage 0)
    - GET /api/v1/migration/adapters + /capabilities (read-only)
  deferred from original Phase 0 scope:
    - libyang ctx loader (now Phase 0.7; stub in place)
    - Real Cisco IOS-XE adapter (split into its own phase 0.5)

PHASE 0.5 — FIRST REAL ADAPTER  (SHIPPED)
  goal:       real OpenConfig NETCONF round-trip against captured payloads
  deliverables shipped:
    - CiscoIOSXEAdapter (netcanon/migration/adapters/cisco_iosxe/)
    - AdapterBase.iter_xpaths (non-abstract, dict-default, override-ready)
    - validate_against gains optional `source` param for adapter-aware walking
    - Sample NETCONF fixture (tests/fixtures/iosxe/get_config_simple.xml)
    - Scope: openconfig-interfaces + openconfig-if-ip (IPv4 only)
    - Round-trip invariant proven
  scope deliberately NOT in this phase:
    - libyang canonical validation (Phase 0.7)
    - Live ncclient transport (Phase 1 — reuses existing collector pattern)
    - IPv6 subtree (currently declared unsupported in matrix)
    - BGP, OSPF, ACLs, VLANs (Phase 1+)
  new_deps:  none — stdlib xml.etree.ElementTree

PHASE 1 — CROSS-VENDOR WRITES + OPNSENSE  (SHIPPED)
  goal:       API-driven manual testing; proof of multi-adapter architecture
  deliverables shipped:
    - OPNsenseAdapter (netcanon/migration/adapters/opnsense/)
      * Scope: system + interfaces (zone, if, descr, enable, ipaddr, subnet)
      * Declares [firewall, router]; flattens zone-keyed XML to list shape
    - POST /api/v1/migration/plan
    - POST /api/v1/migration/render (alias of /plan for now; split in Phase 2)
    - source_filename shorthand loads from existing FileConfigStore
    - MigrationPlanRequest pydantic model
  deliverables DEFERRED to later phases:
    - Live ncclient/SSH transport for deploy (still manual input-text only)
    - Firewall rules / NAT in OPNsense scope (needs netcanon-ext YANG; Phase 2)
    - Transform registry wired to /plan (Phase 2)
    - Snapshot + deploy endpoints (Phase 2)
  new_deps:  none — stdlib xml.etree.ElementTree

PHASE 2 (part 1) — /MIGRATE WORKBENCH UI  (SHIPPED)
  goal:       browser-driven manual testing of the translator
  deliverables shipped:
    - GET /migrate page (netcanon/templates/migrate.html)
    - Source + target adapter dropdowns hydrated from
      GET /api/v1/migration/adapters on load
    - Paste-raw-text OR pick-stored-config input modes
    - Force-cross-class checkbox
    - Result surface with banner (reuses diff palette),
      rendered-output panel (reuses config viewer's syntax highlighter),
      and expandable paths drill-down (supported / lossy / unsupported)
    - Copy-output button
    - 13 E2E tests; 29 new migrate-* testids
  deliverables STILL PENDING for Phase 2:
    - Transform wizard (rename_interfaces, remap_vlans, …)
    - Deploy endpoint + pre-deploy snapshot + diff link
    - "Open diff with source" button that deep-links into /configs/{L}/vs/{R}
  new_deps:  none — all reuse

PHASE 0.7 — LIBYANG CANONICAL VALIDATION  (pending)
  goal:       optional validates-if-available seam for stricter trees
  deliverables:
    - libyang ctx loader (netcanon/migration/canonical/loader.py)
    - Vendored or submoduled openconfig/public modules
    - AdapterBase adapters opt-in to canonical validation via
      a method that returns their YANG module names
    - Pipeline: validate stage runs libyang checks AFTER adapter
      classification if the ctx loader is available; warns-rather-
      than-blocks if libyang is missing (keeps the repo installable
      without C toolchains on Windows devs)
  new_deps:  libyang (optional extra), pyangbind (build-time only)

# Note: the PHASE 1 / PHASE 2 / PHASE 3 design-target blocks that
# previously sat here have been removed.  Their goals all shipped:
# every cross-vendor MVP adapter, the /migrate UI, the deploy-path
# stages, the codec set (junos / arista_eos / etc.) all landed across
# Phases 0.5 → 2 (part 1) above.  The SHIPPED entries earlier in this
# file are the authoritative record.  PAN-OS and aruba_cx remain
# pending and tracked through CHANGELOG.md / [ ] items elsewhere.

=====================================================================
## 13. QUICK-REFERENCE CHECKLIST WHEN ADDING A NEW CODEC
=====================================================================

[ ] netcanon/migration/codecs/<vendor>/ directory with:
    __init__.py, codec.py, parse.py, render.py, port_names.py
    (any vendor-specific helpers: vlan_heuristics.py, etc.)
[ ] Codec class registered via netcanon.migration.codecs.registry
    (NOT pyproject.toml entry points)
[ ] Codec class declares CapabilityMatrix as a ClassVar (in-code,
    not a separate YAML); supported / lossy / unsupported paths
    each cited
[ ] Codec class declares direction (`bidirectional` / `parse_only` /
    `render_only`) and certainty (`best_effort` / `certified`) as
    ClassVars
[ ] Unit test: parser handles malformed input without crashing
[ ] Unit test: renderer handles empty tree gracefully
[ ] Unit test: parse(render(tree)) == tree for supported paths
    (round-trip suite under tests/unit/migration/codecs/<vendor>/)
[ ] Cross-vendor expectation YAMLs under
    tests/fixtures/cross_vendor_expectations/ for every pair
[ ] Real-capture fixtures under tests/fixtures/real/<vendor>/ with
    NOTICE.md provenance + RESULTS.md row
[ ] testid_reference.md updated if any UI added
[ ] CHANGELOG.md entry under Unreleased
[ ] docs/CAPABILITIES.md updated with the new vendor row
[ ] docs/vendors/<vendor>.md operator-facing "what works for me?" page
[ ] Desktop parity verified (embedded server includes new routes)

=====================================================================
## 14. HONEST LIMITATIONS
=====================================================================

- Non-YANG adapters are reverse-engineering config semantics; lossy.
- OpenConfig models are incomplete (~60-70%); expect extensions.
- License keys, cryptographic material, and runtime state don't migrate.
- Semantic equivalence across vendors is best-effort; validate in staging.
- Roundtrip invariant is for SUPPORTED subset only; unsupported paths
  by definition can't roundtrip.
- Deploy path can brick a device; snapshot + rollback are required
  defaults, not opt-in.

=====================================================================
## 15. ANSWER SUMMARY (for status pings)
=====================================================================

Q: Are YANG templates available for all major network vendors?
A: No. Cisco XR/XE, Juniper, Arista, Nokia, Aruba CX, Huawei, and
   partially F5 have native YANG. FortiGate, MikroTik, OPNsense,
   Palo Alto, Check Point, SonicWall do NOT. Any "full YANG" claim
   either narrows vendor scope or uses YANG as internal IR with
   per-vendor parsers (this plan's approach).

Q: What's the architecture?
A: Six-stage pipeline (collect -> parse -> transform -> validate ->
   render -> deploy) over an OpenConfig + netcanon-ext YANG canonical
   tree. Pluggable per-vendor adapters implement parse/render with
   Jinja templates for CLI/XML targets and NETCONF edit-config for
   YANG-native targets. Capability matrices drive pre-flight
   ValidationReports. Sits on top of Netcanon's existing collector
   and storage layers.

=====================================================================
## 16. SEE ALSO
=====================================================================

- ARCHITECTURE.md — four-layer design and migration pipeline overview
- tests/fixtures/real/RESULTS.md — per-codec certification state and
  real-capture coverage matrix (the "what blocks promotion" notes
  feed back into this roadmap)
- netcanon/migration/codecs/README.md — codec authorship guide for
  the per-vendor parse/render adapters this plan describes
- tests/fixtures/real/PHASE4_RECONCILIATION.md — current cross-mesh
  fidelity reconciliation matrix; top-6 codec-fix backlog joins
  Phase 1 mechanical drift against Phase 3 vendor-doc expectations
- tests/fixtures/real/user_smoke_findings.md — operator-spotted
  bugs from manual smoke testing (active backlog of cross-vendor
  translation issues surfaced by running real Cisco / Arista / etc
  configs through the migrate UI and inspecting per-target output)

# END
