PerfLens

Real-time Linux Performance Profiler
Connecting...
Download Flamegraph SVG
Download Collapsed Stacks
Download Session JSON
Dark

Real-time Linux Performance Profiler

Connect to a remote agent, profile a process, explore results

⚙

Live Debug

Connect to an agent running on your target device and start a new profiling session

📁

Saved Sessions

Browse, replay, or import previously saved profiling sessions

Agents using --server connect in automatically — the UI will switch when an agent arrives
1 Connect
2 Process
3 Perf
4 Binary
5 Options
6 Start

Connect to Agent

Enter the IP and port of the agent running in --listen mode on your target device.

Select Process

Choose the process to profile on the target device.

Click Refresh to load process list

Perf Capabilities

Checking that perf is available on the target and probing supported events.

Binary & Source Mapping

Provide paths on this server to the unstripped binary and source directory for line-level annotation. Optional — skip if not needed.

Toolchain & Cross-compilation
Derives addr2line and readelf from the prefix. For cross-compiled targets.
Target filesystem root for resolving shared libraries and source files (like perf --symfs).

Profiling Options

Configure sampling parameters. Defaults work well for most cases.

Review & Start

0
Samples
--
Agent
Source view unavailable — start server with --binary to enable
# Function Module Self % Total % Samples
Waiting for data...

Click a function in the table to view source code.

Waiting for data...

Waiting for data...

Loading sessions...

Documentation

v0.5.0

Overview

PerfLens is a remote Linux performance profiler with a real-time web UI. Drop the agent on any Linux device (ARM or x86), point it at a PID, and watch flame graphs, function tables, perf stat metrics, and line-level annotated source update live in your browser.

No frontend frameworks. No pip dependencies. No Docker. Pure Python stdlib on the server, plain HTML/CSS/JS for the UI.

Quick Start

1. Start the Server

On the machine where you want to view profiles:

python3 server/perflens_server.py \
    --source-dir /path/to/source \
    --binary     /path/to/myprogram

Or use a pre-built release tarball:

tar xf perflens-server-*-linux-x86_64.tar.gz
./perflens-server --source-dir /path/to/src --binary /path/to/binary

Then open http://localhost:8080 in your browser.

2. Start the Agent

On the target Linux device. Two connection patterns:

# Agent connects to server
./perflens-agent --server <server-ip>
./perflens-agent --server <server-ip>          # C agent

# Agent listens, connect via UI wizard
./perflens-agent --listen
./perflens-agent --listen                       # C agent

With --server, the UI switches automatically when the agent connects. With --listen, use the Live Debug wizard.

Prerequisites

ComponentNeeds
Target deviceLinux and perf — nothing else (static agent binary)
Local machinePython 3.8+ (or frozen tarball); addr2line + readelf from binutils
BinaryCompiled with -g (debug symbols), not stripped
SourceA checkout of the source tree readable from the server machine
Cross-compile(Optional) Matching toolchain with <prefix>addr2line and <prefix>readelf

Server Options

OptionDefaultDescription
--port PORT9999TCP port the agent connects to
--http-port PORT8080HTTP port for the web UI
--source-dir DIR.Root of the source tree for line annotation
--binary PATH—Unstripped binary (enables addr2line source mapping)
--map PATH—GNU ld linker map file (optional symbol fallback)
--path-map FROM=TO—Rewrite compile-time paths to local paths
--addr2line PATH—Custom addr2line binary
--readelf PATH—Custom readelf binary
--toolchain-prefix P—Cross-compilation prefix (e.g. arm-linux-gnueabihf-); derives addr2line + readelf
--sysroot DIR—Sysroot for resolving shared library modules and source files
--max-samples N500000Ring buffer cap before oldest samples drop
--inline / --no-inlineonEnable/disable inline function resolution
--import FILE—Import a perf.data file at startup as a session

Agent Modes

The agent has three run modes (pick one):

ModeDescription
--listenDaemon: bind port, wait for server to connect in via wizard
--server HOSTDaemon: connect out to server (reconnects with backoff)
--output FILEHeadless: collect once, write to file (- = stdout). Requires --pid

Agent Options

OptionDefaultDescription
--pid PID—PID to profile (required for --output; set via wizard in daemon modes)
--port PORT9999TCP port (listen or connect)
--frequency HZ99perf record sampling frequency
--duration SECS8Length of each collection round
--rounds N1Number of rounds (--output mode only)

Cross-Compilation & Toolchain

For profiling cross-compiled targets (e.g. ARM binaries on an x86 host), use the toolchain options to point at the correct binutils:

# Using toolchain prefix (recommended)
perflens-server --toolchain-prefix arm-linux-gnueabihf- \
    --sysroot /path/to/target/rootfs \
    --binary  /path/to/unstripped-binary

# Or specify tools individually
perflens-server --addr2line /opt/toolchain/bin/arm-linux-gnueabihf-addr2line \
    --readelf /opt/toolchain/bin/arm-linux-gnueabihf-readelf

--toolchain-prefix appends addr2line and readelf to the prefix. If you only set --addr2line, PerfLens infers the matching readelf automatically.

--sysroot resolves shared library paths (like /lib/libpthread.so) under the sysroot tree, similar to perf --symfs. Source files are also looked up under sysroot as a fallback.

These can also be set at runtime from the wizard's Toolchain & Cross-compilation section in step 4.

Supported Perf Events

EventUseMode
cyclesCPU time / hot pathsrecord + stat
instructionsIPC, retired instructionsrecord + stat
cache-missesLast-level cache missesrecord + stat
cache-referencesLLC accessesrecord + stat
branch-missesBranch prediction missesrecord + stat
branch-instructionsTotal branchesrecord + stat
page-faultsMinor/major page faultsstat only
context-switchesScheduling pressurestat only
cpu-migrationsInter-CPU movementstat only

The agent probes each event on the target kernel and only collects the ones that are actually supported.

Setup Wizard

  1. Connect — Enter the agent's IP and port (default 9999)
  2. Perf — Auto-probes supported perf events and call-graph modes on the target
  3. Process — Pick a PID from the process list or enter manually
  4. Binary — Path to unstripped binary, source directory, and optional toolchain/sysroot settings
  5. Options — Sampling frequency (Hz) and collection duration (seconds)
  6. Start — Review settings and begin profiling

Profiling View

  • Functions tab — Ranked table of hottest functions by self% and total%. Click a row to view annotated source.
  • Source tab — Line-level annotated source with heat-colored sample counts (red = hot, green = cold).
  • Flame Graph tab — Interactive SVG flame graph. Click to zoom, double-click to reset. Search bar highlights matching functions.
  • Threads tab — Per-thread CPU breakdown. Shows each thread with sample count, CPU%, and top function. Click a thread to drill into its flamegraph, function table, and per-line source view.
  • Sessions tab — List of saved sessions. Replay any session or import a perf.data file.

Thread Filtering

When profiling multi-threaded programs, use the Thread dropdown next to the event selector to filter all views by a specific thread. The dropdown shows thread names (from pthread_setname_np / prctl) and TIDs.

The dedicated Threads tab provides a full overview: every thread listed with its sample count, CPU share, and hottest function. Click any row to see that thread's complete flamegraph, function breakdown, and line-level source annotation.

Device Health Strip

When the agent sends system metrics, a collapsible strip shows live CPU, memory, temperature, load, and per-process stats with sparkline charts.

Export Options

Use the Export button in the header to download:

  • Flamegraph SVG — The current flame graph as a standalone SVG file
  • Collapsed Stacks — Brendan Gregg format, compatible with FlameGraph tools
  • Session JSON — Full session data including all samples and metadata

Common Issues

perf_event_paranoid too high

The agent warns if /proc/sys/kernel/perf_event_paranoid > 1. Fix:

sudo sysctl -w kernel.perf_event_paranoid=1

No function names

Compile with -g and do not strip. Verify: file ./myprogram should say not stripped and with debug_info.

No source line mapping

Check --binary points at the exact unstripped binary and --source-dir has the source. Use --path-map /build/src=/home/me/src if build paths differ.

Agent can't connect

Server must be reachable on --port. Test: nc -zv <server-ip> 9999

Container: perf record is empty

Some containers strip perf capabilities. System-wide perf record -a usually works as a fallback.

Slow startup (6–12s)

Call-graph probing tests fp, dwarf, then lbr sequentially. Normal on first connection.

Cross-compiled binary: wrong symbols

Use --toolchain-prefix to ensure the correct addr2line and readelf for the target architecture. System binutils can't read cross-architecture binaries.

Pipeline

[Target device]                    [Local machine]
  Process (PID)                      Server (perflens_server.py)
     |                                   |
  perf record + perf stat                |
     |                                   |
  Agent (Python or C)                    |
     |                                   |
     +-- TCP (5-byte hdr + zstd) -->  recv + decompress
                                         |
                                    parser.py + source_mapper.py
                                         |
                                    SSE --> Browser (app.js)

In one sentence: perf record → agent → TCP+zstd → server → parser → source mapper → SSE → browser.

Wire Protocol

Every message: 5-byte header + payload.

FieldSizeMeaning
LEN4 bytes (uint32 BE)Payload length
FLAG1 byte0=raw, 1=zstd, 2=cmd req, 3=cmd resp, 4=health
PAYLOADLEN bytesPerf script text, JSON, or compressed data

Zstd compression ratio: typically 20–40× on real perf script output.

HTTP API

EndpointMethodDescription
/api/statusGETServer + agent state, sample totals
/api/streamGETSSE: status, event_types, per_event, perf_stat
/api/sessionsGETList saved sessions
/api/sessions/<id>GETReplay a session
/api/sourceGETAnnotated source (?file=&event=&tid=)
/api/thread-viewGETPer-thread flamegraph + functions (?event=&tid=)
/api/thread-summaryGETAll threads overview (?event=)
/api/config/toolchainPOSTSet toolchain prefix and sysroot
/api/config/binaryPOSTSet binary path
/api/config/sourcePOSTSet source directory
/api/stopGETDisconnect agent

Key Design Decisions

  • Python stdlib only — no Flask, no npm, no Docker
  • Plain HTML + vanilla JS + CSS — no bundler, no framework
  • addr2line pipelined in batches of 500 addresses
  • Session replay is lazy: raw chunks parsed on demand
  • Single SourceMapper shared across all requests
  • Agent probes events and call-graph modes before collecting
  • Single static C agent binary — zero runtime dependencies, self-updating
  • Per-thread profiling: parser extracts pid/tid/comm from perf script output