pycontrol-gui architecture

How information moves through the GUI, core, and board

This guide starts with the simplest picture, then adds names and threads one layer at a time. The goal is a practical beginner mental model: where commands go, where data comes back, and which parts run independently.

Commands go down.

Button clicks become Qt signals, then controller calls into pycontrol-core.

Events come up.

The board sends task data to core; core fans it out to GUI logs, controls, and plots.

Threads keep work separate.

The GUI thread draws widgets while worker and pump threads handle slower board I/O.

One Picture

The system has three big pieces. The GUI is what the user sees. Core is the Python library that knows how to talk to boards and workspaces. The board is the pyControl hardware running MicroPython and the task framework.

pycontrol-gui Buttons, tabs, tables, logs, controls, plots.
commands
pycontrol-core BoardSession, workspace files, recording, task setup.
serial I/O
pyControl board MicroPython, framework, task code, hardware events.
GUI updates Log rows, variable fields, task status, live plots.
events
core event bus Decodes task data and notifies subscribers.
bytes
running task States, events, prints, variables, warnings, errors.
First mental model: GUI actions travel down to the board. Run data travels back up to the GUI.

Simplest rule: GUI widgets do not directly manage the serial board session. They send requests to a controller. The controller talks to core. Core talks to the board.

Everyday Workflow

Before the internals, here is the lab path the GUI is built around. Most sessions are this straight line; the rest of this guide explains what happens underneath each step.

Open workspace Tasks, setups, hardware definitions, and experiments load.
prepare
Configure setups Probe ports, set a default hardware definition, sync firmware.
run
Run task / experiment Connect, upload, start; watch logs, controls, and live plots.
on stop
Data bundle Each run writes a .pycontrol.zip (plus optional TSV/NPY).
The everyday path: open a workspace, prepare setups once, run, and collect a per-run data bundle.

Two run modes share this shape. Run task drives one board from the Run task tab. An experiment repeats the same connect → upload → start → stop path for every subject/setup at once, one isolated worker per rig. Setup and firmware preparation is one-time, not per run.

Three Flows

Most architecture confusion goes away if you separate three different kinds of information. They use some of the same objects, but they are not the same flow.

1. Workspace information

Tasks, setups, ports, hardware definitions, experiments, data directory, and settings. This mostly moves from core workspace APIs into the tabs through WorkspaceModel.

2. Commands

Connect, upload, start, stop, set a variable, trigger an event, sync firmware, or probe a setup. These move from GUI widgets to command signals on SessionRuntime, then to SessionController.

3. Live run events

State changes, task events, printed messages, variable changes, warnings, errors, and run end. These move from core back to Qt through QtBusAdapter.

One useful distinction

Commands down do not use QtBusAdapter. Live task data up does. Controller status messages are their own Qt signals.

WorkspaceModel Reads core workspace snapshots and emits tab updates.
updates
Tabs Run task, Experiments, Setups.
commands
SessionController Queued worker-thread object that performs board actions.
core calls
BoardSession Core object that owns the live board session.
Workspace information keeps the UI choices current. Commands operate the board. Live run events return on a separate path.

Words First

A few programming words show up throughout the code. They are ordinary ideas with precise names.

Thread

A thread is a path of execution inside the same program. Two threads can make progress independently, but they share memory, so code must be careful when crossing between them.

GUI thread

The main Qt thread. It owns the widgets. In Qt apps, widget changes should happen here, otherwise the interface can freeze or behave unpredictably.

QThread

Qt's worker-thread object. The GUI uses it so slow board operations can happen away from the GUI thread.

Qt signal

A typed message. One object emits a signal, and another object's method, called a slot, receives it.

Queued connection

A safe cross-thread signal. Qt places the method call into the receiver thread's event queue, so it runs on the receiver's thread instead of the sender's thread.

Subscriber

An object that says, "tell me when core has session data." Core calls subscriber methods such as on_data(), on_warning(), and on_session_end().

Main Objects

The most confusing names are SessionRuntime, SessionController, QtBusAdapter, and BoardSession. They are not four different versions of the same thing. They are four pieces around one board workflow.

GUI tab or panel Wants to control one board workflow.
has
SessionRuntime Owns command signals, controller, adapter, and QThread.
runs
SessionController Receives commands on the worker thread and calls core.
creates
BoardSession Core's live connection to one board.
events
QtBusAdapter The listener added to BoardSession so run events can get back to Qt.
One runtime is the container for one board-control lane. The controller sends commands down. The adapter carries live run events back up.

SessionRuntime

The kit and owner

It bundles together the pieces needed to control one board lane. It creates the worker QThread, the controller, and the bus adapter, then wires its command signals to controller slots.

SessionController

The background worker

It receives requests such as connect, upload, start, stop, set variable, and trigger event. It is the GUI-side object that actually calls pycontrol-core.

BoardSession

The core board session

It represents one connection to one physical board. It knows the serial protocol, current session state, loaded task, subscribers, and the pump thread used during a run.

QtBusAdapter

The return-path translator

It is attached to BoardSession as a subscriber. Core calls it when run data arrives, and it re-emits that data as Qt signals for logs, plots, controls, and status.

A useful sentence: SessionRuntime owns the lane, SessionController drives commands down into core, BoardSession is core's connection to the board, and QtBusAdapter carries live run events back to the GUI.

Name Plain-English job Where it usually lives
MainWindow The top-level wiring place. It creates the tabs, the workspace model, and the shared runtime, then connects their signals. GUI thread
WorkspaceModel The workspace reader and writer. It turns tasks, setups, ports, experiments, settings, and data paths into tab-ready updates. GUI thread, plus short reload workers
RunTaskTab The single-board control panel. It lets the user choose a setup and task, then displays the resulting logs, controls, status, and plots. GUI thread
SetupsTab The setup editor and maintenance panel. It saves setup records and asks the shared controller to run short board-maintenance jobs. GUI thread
ExperimentsTab The experiment editor. It prepares experiment JSON and launches the multi-subject run view. GUI thread
SessionRuntime The owner of one board-control lane. It creates and wires command signals, the controller, bus adapter, and worker thread. Created by GUI objects; controller runs on its QThread
SessionController The background worker for board commands. It owns the current BoardSession and calls core methods on request. Worker QThread
QtBusAdapter The event return bridge. It does not command the board; it converts core run callbacks into Qt signals. Called by pump thread; GUI receives queued signals
BoardSession Core's live board session. It knows the serial connection, task setup, session state, subscribers, and pump thread. pycontrol-core, used by the controller

Command Path

Commands are requests from the user or UI to do something. Instead of doing serial-board work inside a button handler, the GUI sends a command signal to the controller's worker lane.

Button or control Example: Connect, Upload, Start, Stop, Set variable.
signal
RunTaskTab Emits a command signal.
signal
SessionController Queued slot runs on the worker QThread.
core call
BoardSession Connects, uploads, starts, stops, reads and writes variables.
"Queued" means Qt waits until the controller's worker thread can safely run the method.

Example: what happens when you click Connect?

The next list follows one ordinary command all the way through the system. It uses Connect because it is the first moment the GUI creates a real core BoardSession, but the same shape applies to other commands: a tab emits a request through SessionRuntime, Qt queues it across the thread boundary, and SessionController does the core work.

Task upload also applies setup variables from the selected setup. Only loaded task variables with matching setup_ names are changed; missing setup values are treated as absent optional overrides, not as task-readiness errors.

The user clicks Connect

RunTaskTab emits connect_requested with the setup name and serial port.

MainWindow routes the command

The tab signal is connected to SessionRuntime.connect_requested.

Qt queues the controller method

SessionRuntime connected commands to controller slots with QueuedConnection.

The controller calls core

SessionController creates a transport, creates BoardSession, adds QtBusAdapter as a subscriber, then calls session.connect().

Status comes back as controller signals

Signals such as connected, state_changed, info, and error update the tab.

Important: controller status signals and live task events are different. "Connected to setup A" is controller status. A task event or variable update during a run comes through the core event bus and QtBusAdapter.

Event Path

Live events start only after a task run starts. At that point, core creates a pump thread. The pump thread keeps reading bytes from the board and turns them into events.

Board task Sends bytes over serial while the task runs.
bytes
BoardSession pump A core thread reads and decodes data frames.
callback
SubscriberRegistry Calls subscriber methods such as on_data.
callback
QtBusAdapter Re-emits the event as a Qt signal.
queued
GUI slot RunTaskTab or SubjectRunPanel updates logs, controls, status, and plots.
The pump thread is allowed to read serial data, but the final widget updates are queued back to the GUI thread.

Several subscribers can listen to the same run. The GUI adapter listens so widgets can update. A data logger can also listen so rows are written to a file. A task extension can listen so task-specific Python code can react to events.

SubscriberRegistry One event fanout point in core.
calls
QtBusAdapter GUI updates
NativeEventLogger Data files
TaskExtension Optional task automation
"Fanout" means one source event is sent to several listeners.

Threads

Think of each thread as a lane of work that appears at a particular moment. The useful questions are: when does this lane exist, who created it, and what work is it responsible for?

Thread or worker
What owns it
When it exists
Main job
GUI thread
Qt application
Always
Draw widgets, handle clicks, receive queued results
Shared controller QThread
MainWindow's shared SessionRuntime
Created when MainWindow opens; stopped when the app closes
Run task commands and Setups maintenance actions for the shared runtime
Experiment subject QThread
Each SubjectRunPanel
Created when an experiment run builds that subject panel
Commands for one subject/setup, independent of other subject panels
Board pump thread
Core BoardSession
Only after session.start()
Read serial bytes and dispatch run events
Workspace reload QThread
WorkspaceModel
Short-lived; created for each background workspace reload
Scan workspace files and serial ports, then return a snapshot
A controller QThread can exist before any board is connected. A pump thread exists only while a task is actively running.

When threads appear in a normal Run task session

Application opens

The GUI thread is already running. MainWindow creates the shared SessionRuntime, and that runtime immediately starts its controller QThread.

Workspace refresh happens in the background

WorkspaceModel occasionally creates a short reload worker thread, collects a snapshot of tasks/setups/ports/settings, applies it to the GUI, then lets that worker end.

User clicks Connect

No pump thread exists yet. The shared controller QThread receives the command and creates a core BoardSession for the selected port.

User uploads a task

Still no pump thread. The controller asks BoardSession to prepare the state machine on the board.

User starts the run

BoardSession.start() starts the framework on the board and creates the pump thread. From this point until run end, the pump thread reads live data and sends it to subscribers.

Run stops or ends

The pump thread exits. The controller QThread remains alive because the same runtime may be used again for another connect, upload, or run.

Single Run task session

Usually has one GUI thread, one shared controller QThread, and one pump thread while the board is actively running.

The workspace reload worker may appear briefly in the background.

Experiment with N subjects

Has one GUI thread, N subject controller QThreads, and up to N pump threads while those subjects are running.

This is why one subject panel can have its own board workflow without sharing the same controller lane as another subject.

Experiments

The experiment UI repeats the same single-board pattern for each subject/setup assignment. The run view coordinates the panels, but each panel owns its own runtime.

ExperimentsTab Loads and edits experiment JSON.
starts
ExperimentRunView Builds one panel for each active subject.
creates
SubjectRunPanel A Own runtime, controller, adapter, plot, controls.
SubjectRunPanel B Own runtime, controller, adapter, plot, controls.
SubjectRunPanel C Own runtime, controller, adapter, plot, controls.
Each subject panel is like a smaller Run task tab with its own worker lane.

ExperimentRunView still has important coordination jobs: start all, stop all, close, optional hardware-test flow, final variable readback, summaries, persistent variables, experiment extension callbacks, and shared plot display. But the board command path and live data path inside each panel are the same basic idea as Run task.

Before a subject panel starts the main task, it applies experiment-table values and persistent fallback values to matching loaded task variables. The start command carries those requested values as provenance variable_overrides, with source metadata recording whether the final value came from the experiment table or persistent storage. Summary/readback-only values and setup_ variables are left out of that payload.

Workspace And Setups

Not everything is a live run. The GUI also keeps workspace information current and lets users maintain setups before running tasks.

Workspace refresh

MainWindow starts a timer. About once per second it asks WorkspaceModel to reload. The model starts a short worker thread, builds a snapshot, then applies the snapshot back on the GUI thread.

The snapshot updates task menus, hardware definition menus, setup tables, ports, experiment names, settings, and data directory fields.

Setups tab

Editing a setup mostly writes workspace files through WorkspaceModel. Maintenance actions, such as firmware sync or USB-mode probing, use the shared SessionRuntime and SessionController.

Setups can also store setup variables. The Run task and Setups tabs discover v.setup_* declarations in task files, and the Setup Variables dialog writes per-setup values as optional task overrides.

For maintenance, the controller opens short-lived core sessions, performs the action, reports setup-specific status signals, and closes the session.

SetupsTab User selects setup maintenance action.
command
shared SessionController Queued runtime command runs away from the GUI thread.
uses
SetupMaintenanceService Creates short-lived BoardSession objects.
status
SetupsTab Shows messages, USB mode, or detected MCU.
Setup maintenance is board I/O, but it is not the same thing as streaming task run data.

Board identity: each successful connect records the board's hardware unique ID on its setup. If that board later appears on a different serial port, the Setups tab recognizes it by that ID, updates the stored port automatically, and notes the change in the Setups log. This keeps setups working when boards are unplugged and replugged into different USB ports without creating duplicate entries.

What To Remember

  • SessionRuntime is the owner of one board-control lane. It creates command signals, the controller, adapter, and worker thread.
  • SessionController is the command worker. It receives GUI requests and calls core methods.
  • BoardSession is the core board connection. It owns the serial session, run state, subscribers, and pump thread.
  • QtBusAdapter is only the event return path. It does not start or stop runs; it carries live run events back to Qt.
  • Run task and Setups share one runtime. Experiments create one runtime per subject panel.
  • A pump thread exists only during a running task. Connect and upload use the controller thread, but they do not create the pump thread.
  • Workspace refresh is a separate flow. It scans files and ports so the UI choices stay current.

A good debugging question is: "Which flow am I in?" Workspace update, command down, controller status back, or live run event up. Once you answer that, the relevant files and thread boundary are usually much easier to find.