ModuLearn Feature Implementation Guide

Five features, in dependency order, each with the concrete code and the files it touches. Written against the current codebase — function names, endpoints and data shapes are real.

Orientation

Everything hangs off a small number of existing functions. Learn these five seams and the whole plan falls into place.

The seams you'll reuse constantly

Build order & effort

1 · Export / Import blueprint easiest win

You already serialize the canvas both ways. Export wraps serialize() in a download; import feeds a parsed file back through rebuild(). Do the position round-trip here so it's available to Undo next.

Step 1 — record node positions in the graph JSON

In serialize(), add pos to each node. It's ignored by the Python compiler (which reads only id/type/params), so this is backward-compatible.

graph.js — serialize()const nodes = graph._nodes.map((n) => ({
  id: String(n.id),
  type: n.type,
  params: collectParams(n),
  pos: [Math.round(n.pos[0]), Math.round(n.pos[1])],   // NEW: round-trip layout
}));

Then honor it in rebuild() — fall back to the grid when absent (old files & runs still load):

graph.js — rebuild()(g.nodes || []).forEach((gn, i) => {
  const node = LiteGraph.createNode(gn.type);
  if (!node) return;
  node.pos = Array.isArray(gn.pos)
    ? [gn.pos[0], gn.pos[1]]
    : [80 + (i % 4) * 240, 80 + Math.floor(i / 4) * 170];
  // …rest unchanged (params, widgets, descriptors, fitNode, graph.add)…
});

Step 2 — export button

graph.js — new functionfunction exportBlueprint() {
  const doc = {
    format: "modulearn.blueprint", version: 1,
    graph: serialize(),
  };
  const blob = new Blob([JSON.stringify(doc, null, 2)],
    { type: "application/json" });
  const a = document.createElement("a");
  a.href = URL.createObjectURL(blob);
  a.download = "modulearn-blueprint.json";
  a.click();
  URL.revokeObjectURL(a.href);
}

Step 3 — import via a hidden file input

graph.js — new functionfunction importBlueprint(file) {
  const reader = new FileReader();
  reader.onload = () => {
    let doc;
    try { doc = JSON.parse(reader.result); }
    catch (e) { setPanel("err", "not valid JSON"); return; }
    const g = doc.graph || doc;                // accept bare graph too
    if (!g.nodes) { setPanel("err", "no nodes in file"); return; }
    // Drop node types this registry doesn't know, and say so.
    const unknown = g.nodes.filter((n) => !SPEC_BY_TYPE[n.type]).map((n) => n.type);
    g.nodes = g.nodes.filter((n) => SPEC_BY_TYPE[n.type]);
    graph.clear();
    rebuild(g);
    scheduleValidate();
    if (unknown.length)
      setPanel("err", "skipped unknown nodes: " + [...new Set(unknown)].join(", "));
  };
  reader.readAsText(file);
}

Step 4 — wire the toolbar

graph.html — inside #bar, near the Load/Clear buttons<button id="exportBtn" title="Download this blueprint as JSON">⭳ Export</button>
<button id="importBtn" title="Load a blueprint file">⭱ Import</button>
<input id="importFile" type="file" accept="application/json" hidden>

graph.js — in boot(), beside the other button handlersdocument.getElementById("exportBtn").onclick = exportBlueprint;
document.getElementById("importBtn").onclick = () =>
  document.getElementById("importFile").click();
document.getElementById("importFile").onchange = (e) => {
  if (e.target.files[0]) importBlueprint(e.target.files[0]);
  e.target.value = "";                          // allow re-importing the same file
};

Verify

• Build a graph, Export, Clear, Import the file → identical graph including positions.
• Hand-edit the file to reference a fake node type → import loads the rest and reports the skip.
• Old saved runs (no pos) still Load onto the grid.

2 · Undo / Redo table stakes

A snapshot stack over serialize() — no command objects. Because scheduleValidate() already fires after every edit, it's the one place to capture history. Do Feature 1 first so snapshots carry positions.

Step 1 — the history store

graph.js — near the top, module scopeconst history = [];        // array of serialize() snapshots (JSON strings)
let histIdx = -1;         // index of the currently-shown snapshot
let restoring = false;    // true while rebuild() replays a snapshot
let histTimer = null;

function captureHistory() {
  if (restoring) return;
  clearTimeout(histTimer);
  histTimer = setTimeout(() => {                 // coalesce rapid edits (slider drags)
    const snap = JSON.stringify(serialize());
    if (snap === history[histIdx]) return;      // no structural change
    history.splice(histIdx + 1);               // drop the redo tail
    history.push(snap);
    if (history.length > 100) history.shift(); else histIdx++;
    updateUndoButtons();
  }, 350);
}

Step 2 — restore

graph.jsfunction applySnapshot(snap) {
  restoring = true;
  graph.clear();
  rebuild(JSON.parse(snap));
  restoring = false;
  validate();            // revalidate WITHOUT re-capturing (scheduleValidate would)
  persist();
  updateUndoButtons();
}
function undo() { if (histIdx > 0) { histIdx--; applySnapshot(history[histIdx]); } }
function redo() { if (histIdx < history.length - 1) { histIdx++; applySnapshot(history[histIdx]); } }

function updateUndoButtons() {
  document.getElementById("undoBtn").disabled = histIdx <= 0;
  document.getElementById("redoBtn").disabled = histIdx >= history.length - 1;
}

Step 3 — hook capture into the edit choke-point

graph.js — scheduleValidate()function scheduleValidate() {
  stopPolling();
  clearTrainStatus();
  persist();
  captureHistory();              // NEW
  clearTimeout(validateTimer);
  validateTimer = setTimeout(validate, 250);
}

Step 4 — buttons + hotkeys

graph.html — in #bar<button id="undoBtn" class="icon" title="Undo (⌘Z)" disabled>↶</button>
<button id="redoBtn" class="icon" title="Redo (⌘⇧Z)" disabled>↷</button>

graph.js — extend the global keydown in initPaletteSearch()window.addEventListener("keydown", (e) => {
  const typing = /^(input|textarea|select)$/i.test(e.target.tagName || "");
  if ((e.metaKey || e.ctrlKey) && e.key.toLowerCase() === "z") {
    e.preventDefault(); e.shiftKey ? redo() : undo(); return;
  }
  // …existing ⌘K / "/" handling…
});

Finally, seed the initial state at the end of boot(): captureHistory(); (and call it after restore() so the restored canvas is history entry 0).

Verify

• Add 3 nodes, wire them, ⌘Z three-plus times back to empty, ⌘⇧Z forward.
• Move a node, ⌘Z → it returns to its old position (proves #1's pos round-trip).
• Undo, then make a new edit → redo tail is correctly discarded.
• Launching a run does not create history noise (poll updates aren't edits).

3 · Run comparison overlay experiment tool

No server change — GET /api/run/{id} already returns full metrics. Draw selected baseline runs ghosted behind the live curve. This also becomes the display layer for the sweep in Feature 5.

Step 1 — a baseline picker

Add a multi-select the user fills from existing runs. Reuse the data refreshRuns() already fetches.

graph.html — in #bar<select id="compare" multiple size="1"
        title="Overlay past runs on the chart"></select>

graph.js — populate alongside refreshRuns()function fillCompare(runs) {
  const sel = document.getElementById("compare");
  sel.innerHTML = runs.map((r) =>
    `<option value="${r.run_id}">${r.run_id}</option>`).join("");
}

Step 2 — fetch + cache baseline metrics

graph.jsconst baselineCache = {};   // run_id -> metrics[]  (avoid refetching every poll)

async function getBaselines() {
  const ids = [...document.getElementById("compare").selectedOptions]
    .map((o) => o.value);
  const out = [];
  for (const id of ids) {
    if (!baselineCache[id]) {
      try { baselineCache[id] = (await (await fetch("/api/run/" + id)).json()).metrics || []; }
      catch { baselineCache[id] = []; }
    }
    out.push({ runId: id, metrics: baselineCache[id] });
  }
  return out;
}

Step 3 — teach drawChart to underlay baselines

Give drawChart a second argument and, before drawing the live series, plot each baseline dashed and dimmed. Crucially, fold baseline values into the y-range calc so nothing clips.

graph.js — drawChart(metrics, baselines = [])// after computing pts, extend the y-range with baseline points:
baselines.forEach((b) => b.metrics.forEach((m) =>
  CHART_SERIES.forEach((s) => {
    const v = m[s.key];
    if (typeof v === "number" && isFinite(v)) ys.push(v);
  })));

// …compute ymin/ymax/X/Y as today, then BEFORE the live series loop:
ctx.setLineDash([3, 3]);
baselines.forEach((b, bi) => {
  const col = "#5f6b7a";                       // muted; or hash bi -> hue
  CHART_SERIES.forEach((s) => {
    ctx.beginPath(); let started = false;
    b.metrics.forEach((m) => {
      const v = m[s.key];
      if (typeof v !== "number" || !isFinite(v)) return;
      const x = X(m.epoch), y = Y(v);
      started ? ctx.lineTo(x, y) : (ctx.moveTo(x, y), started = true);
    });
    ctx.strokeStyle = col; ctx.globalAlpha = 0.4;
    ctx.lineWidth = 1; ctx.stroke();
  });
});
ctx.setLineDash([]); ctx.globalAlpha = 1;
// …then the existing live-series drawing runs on top…

Step 4 — feed baselines from the poll loop

graph.js — pollRun()drawChart(j.metrics, await getBaselines());

Also invalidate a cache entry when its run is still live, and redraw when the #compare selection changes (onchange → drawChart(lastMetrics, …)).

Verify

• Run twice at different learning rates; select the first as baseline while the second trains.
• Baseline shows dashed/dim; live curve sits on top; y-axis frames both.
• Deselecting removes the overlay on next redraw.

4 · Stop / Re-run closes the lifecycle

Step 1 — give each job a stop event

server.py — _Jobclass _Job:
    def __init__(self, app_state, compiled):
        self.compiled = compiled
        self._app = app_state
        self.stop_event = threading.Event()
        self.thread = threading.Thread(target=self._run, daemon=True)

    def _run(self):
        run_dir = self._app.run_dir(self.compiled.run_id)
        reporter = RunReporter(run_dir, stop_event=self.stop_event)
        reporter.state(phase="running", epoch=0)
        try:
            self._app.on_train(self.compiled, reporter)
            if reporter._state.get("phase") == "running":
                reporter.state(phase=("stopped" if self.stop_event.is_set() else "done"))
        except Exception as e:
            reporter.state(phase="error", error=str(e))
        finally:
            self._app.jobs.pop(self.compiled.run_id, None)

Step 2 — expose should_stop() on the reporter

server.py — RunReporter@dataclass
class RunReporter:
    run_dir: Path
    _state: dict = field(default_factory=dict)
    stop_event: object = None

    def should_stop(self) -> bool:
        return bool(self.stop_event and self.stop_event.is_set())
    # …state() / metric() unchanged…

Step 3 — stop & re-run endpoints

server.py — inside create_app()@app.post("/api/run/{run_id}/stop")
def api_stop(run_id: str):
    job = state.jobs.get(run_id)
    if not job:
        raise HTTPException(404, "no live run with that id")
    job.stop_event.set()
    RunReporter(state.run_dir(run_id))._state  # (optional) mark "stopping"
    (state.run_dir(run_id) / "state.json")  # leave phase to the job's finally block
    return {"ok": True, "run_id": run_id}

@app.post("/api/graph/{run_id}/rerun")
def api_rerun(run_id: str):
    p = state.runs / run_id / "graph.json"
    if not p.exists():
        raise HTTPException(404, "no saved graph for this run")
    g = json.loads(p.read_text())
    try:
        c = compiler.compile_graph(g, registry)      # fresh run_id is generated
    except compiler.GraphError as e:
        raise HTTPException(400, {"errors": e.errors})
    d = state.run_dir(c.run_id)
    (d / "graph.json").write_text(json.dumps(g, indent=2))
    (d / "config.json").write_text(json.dumps(c.as_dict(), indent=2))
    job = _Job(state, c); state.jobs[c.run_id] = job; job.start()
    return {"run_id": c.run_id}

Step 4 — reference trainer honors it

quickstart.py — inside on_train's loopfor e in range(epochs):
    if reporter.should_stop():
        break                     # job writes phase="stopped"
    # …existing metric/state/sleep…

Step 5 — frontend controls

Add a Stop button that shows only while running. Fold a stopping state into the setTrainState() machine you already have. Re-run can live on the saved-runs dropdown (a small ↻ button beside Load).

graph.jsasync function stopRun(runId) {
  await fetch(`/api/run/${runId}/stop`, { method: "POST" });
  // the existing poll loop will observe phase="stopped" and settle the UI
}
async function rerun(runId) {
  const r = await fetch(`/api/graph/${runId}/rerun`, { method: "POST" });
  const j = await r.json();
  if (r.ok) { stopPolling(); pollRun(j.run_id); }
}

Verify

• Launch a long run (bump epochs), hit Stop → curve halts, phase reads stopped, dot stops pulsing.
• Re-run a past run → new run id, fresh curve, original preserved.
• An on_train that never checks should_stop() still completes normally.

5 · Hyperparameter sweep capstone

The payoff of the typed graph: one blueprint fans into N runs across a cartesian product of swept values. Build this after #3 (its chart is the display layer) and #4 (reuse the launch/lifecycle plumbing). Ship it in thin slices.

Design decision — how a sweep is expressed

Cleanest with the existing type system: a sweep hyperparameter node that emits the same scalar/<field> subtype (so it plugs into the Train sink's existing input) but whose param is a list of values instead of one. No new port type; wiring rules are unchanged.

Slice A — registry: a sweep node kind

registry.py — new helper (mirrors add_hyperparameter)def add_sweep(self, name, *, label, values, help=""):
    """A hyperparameter that fans out: emits scalar/<name> like its
    single-valued sibling, but carries a list the compiler expands."""
    self._hparam_ids.append(name)
    return self.add_node(NodeSpec(
        id=f"hyperparameter.{name}", category="hyperparameter", title=label,
        help=help, outputs=[Port("value", f"scalar/{name}")],
        params=[Param(name, label, "text", ",".join(map(str, values)), help=help)],
        meta={"sweep": True},
    ))

Slice B — compiler: expand to many graphs

Add a sibling to compile_graph that returns a list of CompiledGraph. Keep the single-graph function untouched for the non-sweep path.

compiler.py — new functionimport itertools

def compile_sweep(graph, reg):
    """Return [CompiledGraph, …] — the cartesian product over swept fields.
    Falls back to a 1-element list when no sweep node is wired."""
    # 1. compile once to validate structure & resolve the base config
    base = compile_graph(graph, reg)
    # 2. find swept fields = hyper inputs whose source node meta.sweep is True
    #    (look them up via the catalog + the graph's links, like compile_graph does)
    axes = {}                       # field -> [v1, v2, …]
    # …parse each sweep node's comma list into typed values…
    if not axes:
        return [base]
    combos = [dict(zip(axes, vals))
              for vals in itertools.product(*axes.values())]
    out = []
    for combo in combos:
        hp = {**base.hyperparameters, **combo}
        suffix = "_".join(f"{k}{v}" for k, v in combo.items())
        out.append(replace(base,                       # dataclasses.replace
            run_id=f"{base.dataset}_{suffix}_{uuid4().hex[:6]}",
            hyperparameters=hp))
    return out

Slice C — server: launch a run group

server.py — api_start becomes group-aware@app.post("/api/graph/start")
def api_start(req: _GraphReq):
    try:
        graphs = compiler.compile_sweep({"nodes": req.nodes, "links": req.links}, registry)
    except compiler.GraphError as e:
        raise HTTPException(400, {"errors": e.errors})
    group = uuid.uuid4().hex[:8] if len(graphs) > 1 else None
    for c in graphs:
        d = state.run_dir(c.run_id)
        (d / "graph.json").write_text(json.dumps(
            {"nodes": req.nodes, "links": req.links}, indent=2))
        cfg = c.as_dict()
        if group: cfg["group"] = group
        (d / "config.json").write_text(json.dumps(cfg, indent=2))
        job = _Job(state, c); state.jobs[c.run_id] = job; job.start()
    return {"group": group, "run_ids": [c.run_id for c in graphs]}

Then surface group in GET /api/runs so the frontend can cluster children.

Slice D — frontend: a mini status grid

When start returns multiple run_ids, replace the single panel with a small grid of tiles — one per run, each a compact drawChart (reuse Feature 3's overlay to stack them, or render side-by-side). Poll each run id; highlight the best test_score.

Verify

• Drop a sweep lr node with 1e-4, 1e-3, 1e-2 → launching starts 3 runs.
• Run ids encode the swept value; all appear grouped in the runs list.
• The >64-combo guardrail rejects an oversized product with a clear error.
• A graph with no sweep node still launches exactly one run (unchanged path).

Wrap-up

After all five, ModuLearn goes from "launch one run" to a genuine experiment tool: shareable/versioned blueprints, safe editing, run comparison, a controllable run lifecycle, and parallel sweeps. Suggested commit boundaries — one per feature — keep each change reviewable and independently shippable.