Metadata-Version: 2.5
Name: blinkview
Version: 0.20.2.dev0
Summary: High-performance log viewer and telemetry tool for multi-source, high-throughput systems.
Project-URL: Homepage, https://roland2025.github.io/blinkview/
Project-URL: Repository, https://github.com/roland2025/blinkview
Project-URL: Issues, https://github.com/roland2025/blinkview/issues
Author-email: Roland Uuesoo <rolandu@gmail.com>
License-Expression: MPL-2.0
License-File: LICENSE
Keywords: arduino,can-bus,data-analysis,diagnostics,distributed-systems,embedded,esp32,jlink,log-viewer,logging,mcu,observability,plotting,real-time,rtt,sam,segger,serial,stm32,structured-logging,telemetry,uart,visualization
Classifier: License :: OSI Approved :: Mozilla Public License 2.0 (MPL 2.0)
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Visualization
Requires-Python: <3.15,>=3.11
Requires-Dist: cobs
Requires-Dist: jsonpatch
Requires-Dist: jsonschema
Requires-Dist: msgpack
Requires-Dist: packaging
Requires-Dist: readchar
Requires-Dist: requests
Requires-Dist: rich>=13.0.0
Provides-Extra: all
Requires-Dist: cantools; extra == 'all'
Requires-Dist: numba; extra == 'all'
Requires-Dist: numpy; extra == 'all'
Requires-Dist: psutil>=6.0.0; extra == 'all'
Requires-Dist: pylink-square; extra == 'all'
Requires-Dist: pyqtdarktheme-fork<2.4,>=2.3.6; extra == 'all'
Requires-Dist: pyqtgraph; extra == 'all'
Requires-Dist: pyserial>=3.5; extra == 'all'
Requires-Dist: pyside6>=6.6.0; extra == 'all'
Requires-Dist: python-can; extra == 'all'
Requires-Dist: qtpy>=2.4.0; extra == 'all'
Requires-Dist: zstandard>=0.22.0; extra == 'all'
Provides-Extra: can
Requires-Dist: cantools; extra == 'can'
Requires-Dist: python-can; extra == 'can'
Provides-Extra: gui-core
Requires-Dist: numba; extra == 'gui-core'
Requires-Dist: numpy; extra == 'gui-core'
Requires-Dist: psutil>=6.0.0; extra == 'gui-core'
Requires-Dist: pyqtdarktheme-fork<2.4,>=2.3.6; extra == 'gui-core'
Requires-Dist: pyqtgraph; extra == 'gui-core'
Requires-Dist: qtpy>=2.4.0; extra == 'gui-core'
Requires-Dist: zstandard>=0.22.0; extra == 'gui-core'
Provides-Extra: hardware
Requires-Dist: cantools; extra == 'hardware'
Requires-Dist: pylink-square; extra == 'hardware'
Requires-Dist: pyserial>=3.5; extra == 'hardware'
Requires-Dist: python-can; extra == 'hardware'
Provides-Extra: jlink
Requires-Dist: pylink-square; extra == 'jlink'
Provides-Extra: pyqt6
Requires-Dist: numba; extra == 'pyqt6'
Requires-Dist: numpy; extra == 'pyqt6'
Requires-Dist: psutil>=6.0.0; extra == 'pyqt6'
Requires-Dist: pyqt6>=6.6.0; extra == 'pyqt6'
Requires-Dist: pyqtdarktheme-fork<2.4,>=2.3.6; extra == 'pyqt6'
Requires-Dist: pyqtgraph; extra == 'pyqt6'
Requires-Dist: qtpy>=2.4.0; extra == 'pyqt6'
Requires-Dist: zstandard>=0.22.0; extra == 'pyqt6'
Provides-Extra: pyside6
Requires-Dist: numba; extra == 'pyside6'
Requires-Dist: numpy; extra == 'pyside6'
Requires-Dist: psutil>=6.0.0; extra == 'pyside6'
Requires-Dist: pyqtdarktheme-fork<2.4,>=2.3.6; extra == 'pyside6'
Requires-Dist: pyqtgraph; extra == 'pyside6'
Requires-Dist: pyside6>=6.6.0; extra == 'pyside6'
Requires-Dist: qtpy>=2.4.0; extra == 'pyside6'
Requires-Dist: zstandard>=0.22.0; extra == 'pyside6'
Provides-Extra: serial
Requires-Dist: pyserial>=3.5; extra == 'serial'
Description-Content-Type: text/markdown

# BlinkView

[🖥️ GitHub Repository](https://github.com/roland2025/blinkview) | [🐛 Report a Bug](https://github.com/roland2025/blinkview/issues)

**BlinkView** is a high-performance log and telemetry viewer for multi-source, high-throughput systems—embedded devices, distributed services, or anything in between.

It aligns and analyzes logs from multiple sources—such as firmware (UART/RTT), CAN bus, Android, or plain TCP/UDP sockets—in a single, time-synchronized timeline. Trace events across processes and devices to understand real system behavior.

## LogViewer and filter with telemetry table

Log Viewer view with one source filtered out, with latest values per module visible in telemetry table.

![BlinkView LogViewer and filter](https://raw.githubusercontent.com/roland2025/blinkview-extra/refs/heads/main/screenshots/python_client_server_logviewer.png)

### Main Ui and Watch window

Ingestion from 2 TCP sources, with a custom watch list example. Latest values from all parsed modules visible in telemetry table

![BlinkView Unified Dashboard](https://raw.githubusercontent.com/roland2025/blinkview-extra/refs/heads/main/screenshots/python_client_server_main.png)

### Split plot view with discrete mode enabled on one channel

![BlinkView Plotting](https://raw.githubusercontent.com/roland2025/blinkview-extra/refs/heads/main/screenshots/python_client_server_plot.png)

---

## The Problem: "Manual Glue"

In complex hardware/software systems, bugs rarely stay in one layer. Investigating a failure often means manually aligning timestamps from a serial terminal, a CAN log, and `adb logcat`.

BlinkView replaces ad-hoc 'log-merger' scripts with a **unified environment** that handles ingestion, time alignment, and visualization in one place.

BlinkView is a from-scratch rewrite of an earlier single-device tool. Real bugs kept crossing device boundaries, so BlinkView was designed from day one around multiple sources on one synchronized timeline—and has since grown into a general-purpose tool for any high-throughput, multi-source logging problem, hardware or software.

---

## Example Use Case

Debugging a command across a system:

- User presses a button in an Android app
- Command is sent over BLE or UART
- Controller processes it and sends CAN messages
- Motor or battery responds

BlinkView lets you see all of this in one timeline:

- Android logcat event
- Transport messages
- Firmware logs
- CAN signals (decoded via DBC)

This makes it possible to:

- trace behavior across components
- measure delays between steps
- identify where failures occur

The same approach applies just as well without any hardware in the loop. Debugging a distributed client/server app:

- Client sends a request over a TCP socket
- Backend service logs the request, does some work, and responds
- Multiple backend workers log concurrently under load

BlinkView lets you see all of this in one timeline:

- Client-side log events
- Backend request/response logs, correlated across workers
- Structured `key=value` fields extracted from each log line for filtering

This makes it possible to:

- trace a single request across processes
- spot which worker or service introduced a delay
- correlate client-observed failures with backend-side causes

---

### 🚀 Live Integration Demo

Want to see BlinkView coordinate a live system right now? Explore this fully configured client-backend simulation package, ready to clone and run:

👉 **[BlinkView Python Client-Backend Demo Repository](https://github.com/roland2025/blinkview-python-demo)**

This demo includes a multi-threaded Qt Client and a headless Backend service—no embedded hardware involved, just plain TCP sockets. It lets you click buttons, adjust sliders, and generate synthetic log streams so you can watch BlinkView extract, plot, and align the network telemetry in real time.

---

## Installation

BlinkView is installed with [uv](https://docs.astral.sh/uv/), which keeps it in its own environment and downloads a suitable Python (3.11+) if you don't have one.

**1. Install uv** (skip this if you already have it):

```bash
# Windows (PowerShell)
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

# macOS / Linux
curl -LsSf https://astral.sh/uv/install.sh | sh
```

**2. Install BlinkView from [PyPI](https://pypi.org/project/blinkview/):**

```bash
uv tool install "blinkview[all]"
blink setup
```

`[all]` brings the GUI and every hardware backend. Narrower extras: `pyside6` (GUI only), `serial`, `can`, `jlink`, `hardware` (all three).

`blink setup` compiles the data-processing kernels once per version, which takes about a minute. The first `blink` does it by itself if you skip it. On a machine without a display, use `blink setup --no-gui`.

### Updating

BlinkView checks for new versions once a day and shows a notice when one is available. Run `blink update upgrade`, or use *Check for updates* in the app. The update channel (`stable`, `rc` or `dev`) is picked in the same dialog or with `blink config --global update.channel dev`.

### Optional: install from source

Only needed if you want to change BlinkView itself or run unreleased code:

```bash
git clone https://github.com/roland2025/blinkview.git
cd blinkview
uv tool install ".[all]"
blink setup
```

A source install updates from the git tags in its checkout instead of PyPI. See [DEVELOPER.md](https://github.com/roland2025/blinkview/blob/main/DEVELOPER.md) for the development setup.

---

## Usage

```bash
# Go to your project directory
cd your/project

# Optional: make this directory a BlinkView project
blink init

# Launch the tool
blink
```

`blink init` is optional. It creates a `./.blinkview/` folder, which keeps this project's settings with the project instead of in your home directory:

| | With `blink init` (project mode) | Without it (standalone mode) |
| --- | --- | --- |
| **Profiles** (sources, parsers, layout) | `./.blinkview/profiles/` (can be committed to Git) | `~/.blinkview/profiles/` |
| **Logs** | `./logs/` (add to `.gitignore`) | `~/.blinkview/logs/` |

BlinkView looks for `.blinkview/` in the current directory and its parents, so you can launch `blink` from any subfolder of the project.

To store logs somewhere else, set a log directory with `blink config --global log_dir /path/to/logs`.

---

## Features

* **Multi-Source Ingestion:** Supported source types include:
  * Serial / UART
  * CAN-bus (with DBC decoding)
  * SEGGER RTT
  * TCP/UDP sockets
  * ADB logcat
  * Live file tailing
* **Text Log Viewer:** 
  * Advanced filtering by device, module, and log level.
  * High-speed text search and highlighting.
  * Structured `key=value` (logfmt) filtering—combine free-text search with exact-match conditions like `status=ok user_id=42`.
  * Auto-pause on high-velocity bursts to maintain UI responsiveness.
* **Table-Based Log Viewer:** A structured, columnar alternative to the text viewer—Time, Device, Level, Module, and Message as resizable columns. Shares the `key=value` filtering as the text viewer.
* **Parsing & Extraction:**
  * Multi-rule Key-Value extractor - five composable rule types for pulling structured data out of raw text streams: 
    * `key=value` pairs, 
    * anchor-word patterns (startswith/endswith/contains).
    * lightweight JSON key lookup
    * CSV-like delimiter-separated fields
    * fixed word-position slices.
* **Playback & Scrubbing:** Treats live and recorded data as one continuous timeline, not two separate modes.
  * **LIVE / REPLAY:** Drop out of the live tail at any point to scrub history, then jump straight back to live.
  * **Session Replay:** Reopen any past recording from the **Load Session...** menu (or `blink replay` on the CLI) and scrub it exactly like a live one.
  * **Jog Wheel:** Press-and-drag precise scrubbing—from single-row stepping at a slow drag up to a fast shuttle.
  * **Named Ranges:** Mark start/end points and name them like clips in a video editor. Saved alongside the session's own captured data, so ranges are there again the next time that session is replayed.
  * **Force Live:** Pin an individual view (table, watch list, plot) to live data even while the shared clock is scrubbed into REPLAY elsewhere—useful for keeping one window on "now" while you dig through history in another.
* **Raw Hex View:** Add a `raw_hex` pipeline to a source to see its bytes as hexdump rows in the log, as they arrived and next to the parsed lines: for working out a device's framing or checking what a parser was given. Stops by itself after a set number of bytes.
* **Session Export:** `blink export` writes a recorded session's unified log as plain text, filtered by device, module, or time range—for reading outside the GUI, by hand or from scripts and AI tools.
* **AI Agent Control (MCP):** `blink mcp` lets an AI agent such as Claude work with a running BlinkView—send a shell command to the device and read its reply, read live logs with the log viewer's own filters, count lines per module and level to find where the errors are, check the latest telemetry values and a signal's history, read the profile's configuration and its schema, change the configuration when you allow it (checked first, with Undo), build watch lists and open them, reset the MCU, or start a new session. Enabled per profile by adding an `mcp_server` pipeline; local connections only. See [Using BlinkView with an AI agent](docs/mcp.md) for the setup.
  * **Python Scripting:** The same tools are available to your own scripts through `blinkview.client`—reset the device, wait for a log line, read telemetry—for automated checks against a live session. See [examples/reset_connect_cycles.py](examples/reset_connect_cycles.py), which resets a board 20 times and reports how long each boot took to connect.
* **Extended Scrollback:** Evicted segments are archived to a memory-mapped disk tier (ideally on NVMe) instead of being dropped—extending scrollback far past what fits in memory, transparently, with no change to filtering/search.
  * **Automatic Hot/Cold Sizing:** The in-memory ("hot") tier can grow to use most of whatever system RAM is actually free, and shrink automatically the moment free memory gets tight elsewhere on the machine—reacting to real system pressure instead of a static size picked once at startup, with a configurable floor so recent scrollback never becomes disk-latency-bound.
  * **Compressed at Rest:** Both the disk-tier ("cold") segments and the raw session/source log files are zstd-compressed once they're done being written, shrinking a recording's on-disk footprint with no change to how it's read back. A progress toast tracks the final compression pass on app close, so the app never exits mid-write.
* **Session Persistence:** Automatically remembers window positions and active log filter settings. Pick up exactly where you left off without re-configuring your workspace.
  * **Layout Presets:** Save any number of named window layouts from the **View** menu (e.g. "desk, 4 screens", "desk, plotting" and "laptop only") and switch between them at any time. Applying a preset only moves, floats, or docks your views back into place—it never resets what's inside them.
  * **Monitor Changes:** When monitors are turned off and back on, windows tend to get shuffled around. BlinkView notices the screen setup change and offers the preset saved for that exact setup in a toast—one click puts everything back.
  * **Multiple Instances:** Every window and dialog title starts with `project / profile`, so windows from several running BlinkView instances are easy to tell apart.
* **Several Identical Devices:** One profile for several identical boards. Each run fills in its own J-Link serial number, COM port and so on, while parsers, rules and watches are kept and changed in one place. See [Several Identical Devices](#several-identical-devices-profile-parameters).
* **Watch / Command List:** 
  * Monitor specific variables and latest state values.
  * Send structured commands back to the device.
* **Live Telemetry Plotting:** Real-time visualization of numeric data streams.
  * **Discrete Mode:** Supports rendering boolean or integer state values by automatically synthesizing intermediate steps. This creates a clean, "staircased" layout that accurately represents step-wise state changes rather than continuous slopes.
* **Unified Timeline Alignment:**
  * Best-effort time alignment across sources with different transport characteristics.
  * Leverages high-precision internal clocks where the hardware/transport allows (e.g., SEGGER RTT) and provides time-correlated views for higher-latency sources like UART or ADB. 

---

## Several Identical Devices: Profile Parameters

When several boards share one configuration and differ only in, say, the J-Link serial number or COM port, keep **one** profile and give each run its own values.

```bash
# Declare which config fields can vary (the current value stays as the default)
blink switch rtt --add-param rtt_serial /sources/src_1eb62594/serial_number
blink switch rtt --add-param com_port /sources/src_ab12cd34/port

# Save one parameter set per board (rtt.left.params.json next to the profile)
blink switch rtt --save-params left  --param rtt_serial=51024923 --param com_port=COM7
blink switch rtt --save-params right --param rtt_serial=51099999 --param com_port=COM9
blink switch rtt --show-params

# One BlinkView per board
blink -p rtt --params left
blink -p rtt --params right
```

* `--params` also accepts a path to any `.json` file of `name: value` pairs, e.g. for machine-specific values you don't want in the repo. It can be repeated; later files win. `--param NAME=VALUE` overrides a single value for one run.
* Parameter values are never written to the profile, so the profile file stays the same for every board. Session logs and `metadata.json` record the values each run actually used.
* The set name appears in the window title (`project / rtt [left]`) and is the default session name. A set can store its own session name (`--save-params left ... --session "Left board"`); `-s` on the command line still wins.
* Each set keeps its own window layout (`rtt.left.gui_state.json`), starting from the profile's normal layout until it is first saved.
* If the `.blinkview` folder is committed to a repo, two `.gitignore` rules keep the per-machine files out of it: `*.params.json` (parameter sets) and `*.gui_state.json` (window layouts).
* Several instances can run the same profile at once. An edit made in one is saved without undoing the others' edits, and the others offer to **Reload** it (a notification, or **Reload Profile from Disk** in the main menu).
* **Main menu → Profile Parameters...** shows the values in use and where they came from. Change a value and press **Apply** to use it right away, without restarting and without changing the profile. **Save to Set** / **Save as New Set...** store the values for the next start, and **Copy Launch Command** gives the matching `blink ...` command line.
* In the config editor, right-click a field's label and choose **Make profile parameter...**, so you don't have to look up the config path.

---

## Architecture & Performance

BlinkView is designed for high throughput and low overhead. In synthetic benchmarks the pipeline handles about **5 million messages per second per thread** (roughly 300 MB/s of raw input), and the message reordering thread tops out at **12 million messages per second** on the same system. A typical real-world load of 10,000 messages per second barely registers in CPU usage. Ingestion is multi-threaded: each data source runs in its own thread, so a slow or bursty source never blocks the others.

<sub>Benchmarks measured on an AMD Ryzen 9 5950X.</sub>

*   **Numba JIT Compilation:** Core parsing, filtering, and reordering logic is compiled to machine code for near-native performance.
  * Minor caveat, the first run after installation needs to run LLVM compilation
*   **Time-Reordering:** A reorder layer buffers incoming packets to handle varying transport latencies and produce a cohesive chronological stream. (default window size is 100ms, but can be increased if data source needs it )
*   **Pooled, Struct-of-Arrays Storage:** Log rows are held as pooled NumPy arrays (not per-message Python objects), with an optional memory-mapped disk tier for older segments—the same filter/search kernels run unmodified whether a segment is in RAM or on disk.

```mermaid
graph TD
    %% Source Nodes
    StreamSource[Stream Sources <br/> <i>UART / RTT / Socket</i>]
    CAN[CAN Source]
    ADB[ADB Source]

    %% Pipeline Subgraphs
    subgraph Stream_Pipe [Stream Pipeline]
        Stream_Raw[Raw File Writer]
        Stream_P[Parser]
        Stream_KV[KV Extractor]
    end

    subgraph CAN_Pipe [CAN Pipeline]
        CAN_Raw[Raw File Writer]
        CAN_P[Parser]
        CAN_KV[KV Extractor]
    end

    subgraph ADB_Pipe [ADB Pipeline]
        ADB_Raw[Raw File Writer]
        ADB_P[Parser]
        ADB_KV[KV Extractor]
    end

    %% Reorder Logic
    Reorder{Reorder Layer <br/> <i>Time-Delayed Buffer</i>}
    
    %% Central Hub
    Storage((Central Storage <br/> <i>Thread-Safe Data Store</i>))

    %% Flow: Sources to Reorder
    StreamSource --> Stream_Raw
    StreamSource --> Stream_P
    Stream_P --> Stream_KV
    Stream_P & Stream_KV --> Reorder

    CAN --> CAN_Raw
    CAN --> CAN_P
    CAN_P --> CAN_KV
    CAN_P & CAN_KV --> Reorder

    ADB --> ADB_Raw
    ADB --> ADB_P
    ADB_P --> ADB_KV
    ADB_P & ADB_KV --> Reorder

    %% Flow: Reorder to Storage
    Reorder -- Ordered Stream --> Storage

    %% Output Nodes (Consumers)
    UWriter[Unified File Writer]
    LogView[Text Log Viewer]
    WatchCmd[Watch / Command List]
    Plotter[Plotter]

    %% Data Flow: Storage to Consumers
    Storage -- Push Stream --> UWriter
    Storage -.->|Poll 10Hz| LogView
    Storage -.->|Poll 10Hz| WatchCmd
    Storage -.->|Poll Variable| Plotter

    %% B&W Styling
    classDef bw fill:#fff,stroke:#000,stroke-width:2px,color:#000
    class StreamSource,CAN,ADB,Stream_P,CAN_P,ADB_P,Stream_KV,CAN_KV,ADB_KV,Stream_Raw,CAN_Raw,ADB_Raw,Reorder,Storage,LogView,WatchCmd,Plotter,UWriter bw
    style Stream_Pipe fill:none,stroke:#000,stroke-dasharray: 5 5
    style CAN_Pipe fill:none,stroke:#000,stroke-dasharray: 5 5
    style ADB_Pipe fill:none,stroke:#000,stroke-dasharray: 5 5
```

---

## Name Origin

BlinkView is named after the first embedded program everyone writes:

```c
while (1) {
    toggle_led();
}
```

The blink is the first signal that your system is alive. BlinkView helps you see everything that follows.

---

**License:** Mozilla Public License 2.0 (MPL-2.0)
