Metadata-Version: 2.4
Name: bspctl
Version: 0.1.0
Summary: kas wrapper for Yocto BSP builds - repo tool sync, pre-flight checks, triage
Keywords: yocto,bsp,kas,bitbake,embedded,nxp,imx,ti-sitara
Author: Javier Tia
License-Expression: Apache-2.0
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Build Tools
Classifier: Topic :: System :: Operating System Kernels :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Dist: typer>=0.12.0
Requires-Dist: rich>=13.0.0
Requires-Dist: pyyaml>=6.0
Requires-Python: >=3.11
Project-URL: Homepage, https://github.com/jetm/bspctl
Project-URL: Repository, https://github.com/jetm/bspctl
Project-URL: Bug Tracker, https://github.com/jetm/bspctl/issues
Description-Content-Type: text/markdown

# bspctl

bspctl is a kas wrapper for Yocto BSP builds. kas is the modern Yocto build
tool that describes a stack as a YAML topology (repos, layers, machine,
distro) and drives `bitbake` on the host; `kas-container` wraps it in a
Docker container for reproducibility. bspctl defaults to `kas-container` and
adds the rest of the workflow that kas leaves to you:

1. Layers a curated tuning overlay on top of your kas YAML at build time
   (ccache, MIRRORS, PREMIRRORS, fetch robustness, BSP-specific knobs)
   without touching the YAML on disk
2. Runs pre-flight checks (`bspctl doctor`) before kicking off `kas-container
   build` so a wrong container Python, full disk, or broken cache fails fast
   instead of four hours in
3. Captures structured per-run telemetry under `build/runs/<ts>/` (event log,
   kas output, env snapshot, timing, disk usage)
4. Ships `bspctl triage` to read the structured logs, locate the failing
   recipe log, and match against a suggestion table

Works with any kas YAML in generic mode - bring your own.

For vendor BSPs that ship as Google repo tool XML manifests (NXP i.MX) or
oe-layertool configs (TI Sitara) instead of kas YAML, bspctl bridges the gap:

- `bspctl sync` populates `sources/` by running the right tool for the
  manifest, and skips when sources are already current
- `bspctl gen-kas` translates the vendor manifest into a kas YAML
  (topology only: machine + repos + layers), so the rest of the pipeline
  above applies unchanged

| Without bspctl | With bspctl |
|----------------|-------------|
| Add ccache, mirror, fetch, and EULA boilerplate to every project YAML | Curated overlay applied at build time; your YAML stays topology-only |
| Start a 4-hour build only to fail on a full disk or wrong container Python | `bspctl doctor` catches environment problems before kas starts |
| Grep through `build/tmp/work/.../temp/log.do_*` to find what failed | `bspctl triage` reads structured run logs, locates the recipe log, and matches against a suggestion table |
| No build history | Per-run directory: structured event log, kas output, env snapshot, disk usage, timing |
| repo tool XML and kas YAML describe the same thing but don't interoperate | `bspctl gen-kas` translates the vendor XML manifest into a kas YAML |
| Re-run repo tool or oe-layertool by hand on every fresh workspace | `bspctl sync` fetches sources and skips if already current |

The NXP/TI presets are the batteries-included path, not a requirement.

## Installation

```bash
uv tool install bspctl
```

```bash
pip install bspctl
```

To install from source (development):

```bash
git clone https://github.com/jetm/bspctl
cd bspctl
uv tool install --editable .
```

Requires Python 3.11+. The bitbake parser compatibility constraint (3.13+
deadlocks the parser fork) applies to the container image, not the host
tool - bspctl runs fine on any supported Python including 3.14.

## Quickstart

```bash
cd ~/bsp-workspace

# NXP i.MX - manifest drives everything
bspctl build -f imx-6.12.49-2.2.0.xml -m imx95-var-dart

# TI Sitara
bspctl build -f processor-sdk-scarthgap-chromium-11.00.09.04-config_var01.txt \
             -m am62x-var-som

# BYO YAML (generic or NXP/TI)
bspctl build path/to/my-build.yml
```

On a clean workspace, the first run populates `sources/` via repo sync and
takes hours. Subsequent runs skip sync automatically and go straight to
bitbake.

## What bspctl adds

### Source sync

kas has no concept of fetching BSP sources. NXP i.MX ships a repo tool
manifest; TI Sitara ships an oe-layertool config. bspctl runs the right
tool and caches the result - re-running `bspctl build` on a populated
workspace skips sync entirely.

```bash
bspctl sync --manifest imx-6.12.49-2.2.0.xml   # explicit sync step
bspctl build -f imx-6.12.49-2.2.0.xml          # sync + build in one shot
```

### Tuning overlay

Every build - BYO or manifest-driven - layers a static tuning file onto
your kas YAML at run time. Your YAML is not modified on disk; kas merges
the overlay in when it parses the build.

What the overlay unions in:

- `CCACHE_DIR` + `INHERIT += "ccache"`
- `BB_NUMBER_THREADS` / `PARALLEL_MAKE` from `$NPROC`
- `IMAGE_FEATURES:remove` (strips dev/dbg packages from images)
- `BB_FETCH_TIMEOUT = "600"` (raise the per-URI timeout)
- `MIRRORS` (replace scarthgap's dead `sources.openembedded.org` with
  the Yocto Project mirror)
- `PREMIRRORS:prepend` for github.com (silent fallback, not a build blocker)
- `FETCHCMD_wget` (crates.io 403 workaround for Rust recipes)
- `PYTHONMALLOC=malloc` (reduces bitbake parser fork-race rate on CPython 3.13)
- **NXP only**: `ACCEPT_FSL_EULA`, renderdoc CMake-launcher fix,
  linux-imx fork PREMIRROR, `meta-varis-overrides` layer
- **TI only**: ti-linux-kernel + ti-u-boot fork PREMIRRORs,
  `meta-varis-overrides-ti` layer

Edit `overlays/bspctl-tuning-<bsp>.yml` to change any knob. The change
applies to BYO and manifest flows alike, with no risk of drift.

### Pre-flight checks

`bspctl doctor` runs before every build (and standalone). BLOCK failures
halt before kas starts; WARN prints and continues.

- Docker daemon reachable and ulimits set
- Container image present locally
- Container Python version (3.13+ deadlocks the bitbake parser fork)
- Disk space on workspace, `SSTATE_DIR`, `DL_DIR`
- NXP: fork repos populated, bitbake override applied
- TI: oe-layertool cloned, active config matches manifest

```bash
bspctl doctor               # standalone check, no build
bspctl build --skip-doctor  # bypass (not recommended)
```

### Triage

`bspctl triage` turns a failed build from a grep exercise into a
one-command diagnosis:

1. Reads `events.jsonl` to find the first `step_fail`
2. Tails `kas.log` around the failure
3. Extracts the bitbake "Logfile of failure stored in:" path, rewrites
   the container path to the host path, and tails that recipe log
4. Matches the combined output against a suggestion table: fetch
   failures, parser deadlocks, OOM, disk full, GitHub flakes, missing
   EULA, stale bitbake cache, kas/container version skew

```bash
bspctl triage                      # most recent run
bspctl triage 20260423-091014      # specific run by timestamp
bspctl triage -k path/to/kas.yml  # most recent run for a BYO YAML
```

### Structured run logs

Each `bspctl build` invocation creates `<bsp_root>/build/runs/<YYYYMMDD-HHMMSS>/`:

| File | Contents |
|------|----------|
| `events.jsonl` | One JSON object per step start/end/error |
| `console.log` | Same stream, human-readable |
| `kas.log` | `kas-container build` stdout + stderr |
| `env.txt` | `BSPCTL_*`, `KAS_*`, `BB_*`, `SSTATE_*`, `DL_*`, `NPROC` at run start |
| `time.log` | `/usr/bin/time -v` output |
| `du.tsv` | `<unix-ts>\t<bytes>` samples of `build/tmp/` every 30 s |
| `diagnosis.txt` | Pre-flight diagnosis as plain text |

## Build forms

```bash
# Manifest-driven, one shot (NXP/TI only)
bspctl build -f imx-6.12.49-2.2.0.xml -m imx95-var-dart

# BYO YAML - skips sync and gen-kas
bspctl build nxp/my-build.yml

# Staged - review the generated YAML before building
bspctl sync --manifest imx-6.12.49-2.2.0.xml
bspctl gen-kas --manifest imx-6.12.49-2.2.0.xml -o nxp/my-build.yml
bspctl build nxp/my-build.yml

# Generic - any kas YAML, bsp_root is the YAML's parent
bspctl build path/to/kas.yml
```

## BSP presets

| Family | Manifest pattern | Source sync | Overlay |
|--------|-----------------|-------------|---------|
| **NXP i.MX** | `imx-A.B.C-X.Y.Z.xml` | repo tool | `bspctl-tuning-nxp.yml` |
| **TI Sitara** | `processor-sdk-*-config_var<N>.txt` | oe-layertool | `bspctl-tuning-ti.yml` |
| **Generic** | any kas YAML | none | `bspctl-tuning-generic.yml` |

BSP detection runs on the manifest filename (manifest-driven) or the
YAML's `machine:` value and `repos:` block (BYO). No NXP/TI markers
falls through to generic mode.

## Configuration

CLI flags, env vars, and built-in defaults in that precedence order.

| Env var | Flag | Default |
|---------|------|---------|
| `BSPCTL_MACHINE` | `--machine` | `imx8mp-var-dart` |
| `BSPCTL_DISTRO` | `--distro` | `fsl-imx-xwayland` |
| `BSPCTL_IMAGE` | `--image` | `core-image-minimal` |
| `BSPCTL_MANIFEST` | `--manifest` | `imx-6.6.52-2.2.2.xml` |
| `BSPCTL_REPO_URL` | - | `https://github.com/varigit/variscite-bsp-platform.git` |
| `BSPCTL_REPO_BRANCH` | - | inferred from manifest prefix |
| `KAS_CONTAINER_IMAGE` | - | When absent: host mode active (plain `kas`, no Docker). Set to enable container builds. |
| `SSTATE_DIR` | - | kas default (unset = skip check) |
| `DL_DIR` | - | kas default (unset = skip check) |

Workspace is resolved by walking up from CWD for a `.bspctl.toml` marker
or `nxp/`/`ti/` subdirectories. Generic BYO builds skip the walk.

## Common invocations

```bash
bspctl build --image fsl-image-gui      # heavier image (Wayland, Qt6, Chromium)
bspctl build --machine imx93-var-dart   # different SoM
bspctl build --dry-run                  # apply overlay, skip kas-container
bspctl build --skip-sync                # don't re-sync sources/
bspctl build --host                     # bypass kas-container, run plain kas build on the host
bspctl doctor                           # standalone pre-flight
bspctl triage                           # post-mortem latest run
bspctl log                              # tail latest kas.log live
bspctl shell                            # kas-container interactive shell
bspctl shell -c "bitbake -e virtual/kernel | grep ^PREFERRED"
bspctl shell --host                     # plain kas shell on the host (no Docker)
bspctl gen-kas -o nxp/my-build.yml      # write topology YAML only
bspctl clean                            # remove <bsp>/build/
```

## Troubleshooting

| Symptom | Resolution |
|---------|-----------|
| `Not inside a workspace` | cd into a directory with `nxp/` or `ti/`, add a `.bspctl.toml` marker, or pass a positional YAML. |
| `kas YAML is outside bsp_root` | NXP/TI builds need the YAML under `nxp/` or `ti/`. Copy `examples/kas-*.yml` there. Generic BYO is exempt. |
| `Could not parse <path> as a kas build` | YAML lacks both `machine:` and a `repos:` block. |
| `parser thread killed/died` mid-parse | Container Python is 3.13+. Rebuild the kas image on Ubuntu 24.04 (Python 3.12). `bspctl doctor` flags this. |
| `wget: sources.openembedded.org: NXDOMAIN` | Dead mirror in scarthgap. The overlay's `MIRRORS =` line suppresses it. |
| `No space left on device` | `bspctl doctor` blocks below 50 GiB free. Check `df` on workspace, `SSTATE_DIR`, and `DL_DIR`. |
| `do_fetch: Fetcher failure` for `linux-imx` | Populate `nxp/forks/linux-imx/` so the local PREMIRROR serves the fetch. |
| `Config file validation Error` from kas | kas version skew between host and container. Align both. |
| General fetch flake | Retry. `repo sync` and `do_fetch` are idempotent. |
