CLI
| Command | Purpose |
|---|---|
callsight init <dir> | Adopt callsight into a project: copy the runtime and build wiring, write a starter config, print the wiring snippet |
callsight run -- <cmd> | Run a binary with tracing on and report on it, in one step |
callsight scan <dir> | Preview which sources a config selects, without building |
callsight select <dir> | Explore a function's call subtree; emit the matching config lines |
callsight flags | Print the compiler flags (what the build integrations call) |
callsight analyze [traces/] | Hotspot report, JSON, folded stacks, a Perfetto timeline, or hot call sites |
callsight diff a.json b.json | Compare two JSON reports; fail a build on a regression |
callsight counters | Resolve a config's counter selection against the built binary and write the address map the runtime reads |
callsight doctor | Check the toolchain, the config, the trace directory, free space and the PMU |
callsight ui | Local web UI (needs the ui extra) |
callsight serve | TCP server for remote streams (needs the stream extra) |
callsight provision | Download the bundled static ctags used by the UI config builder |
callsight --version | Installed version |
init
callsight init <dir> [--build make|cmake] [--stream]
--build | Force the build system. Default: auto-detect (CMakeLists.txt → cmake, else make) |
--stream | Also copy the on-device streaming client (trace_stream.c + vendored zstd) |
An existing trace.config is never overwritten.
run
callsight run [options] -- <command> [args...]
Sets the environment, runs the command, then analyzes what it recorded. The four-step
loop is identical every time and easy to get subtly wrong — most often by reporting on a
traces/ directory that still holds the previous run.
--dir | Trace directory. Default traces; files from an earlier run are cleared first |
--keep | Keep those earlier files instead, and report on both runs together |
--timeout N | Stop the program after N seconds and report on what it recorded |
--mode | events (default) or summary |
--max-mb, --max-events, --full | Capture limits — see capture limits |
--threads, --clock | Thread filter and timestamp source |
--config | Selection config, whose counter directives are resolved against --exe before the run. Default trace.config |
--exe | Binary to symbolize with. Default: the command itself |
--out FILE | Write the report to a file. The traced program shares stdout, so machine-readable output needs somewhere of its own |
--top, --format, --addr2line, --subtract-overhead | Passed through to analyze |
callsight run --timeout 10 --top 20 -- ./bin/app.instr --workload heavy
callsight run --mode summary --out report.json --format json -- ./bin/app.instr
--timeout sends SIGTERM first so a clean exit can flush each
thread's buffered tail; only a program that ignores it gets killed.
scan
callsight scan <dir> [--config trace.config]
Prints how many sources would be instrumented versus excluded, lists the excluded ones,
and — for an include-func config — the subtree size and any
substring-collision warnings.
select
callsight select <dir> --function NAME [--depth N] [--threads GLOB] [--list]
--function, -f | Seed function; repeatable |
--depth, -d | Limit subtree expansion (0 = just the seed, 1 = direct callees). Default: full subtree |
--threads, -t | Also print a TRACE_THREADS runtime hint |
--list, -l | List every function callsight can see, with the files defining it |
flags
callsight flags --config CONFIG [--scan DIR] [--format make|raw]
[--compiler auto|gcc|clang] [--compiler-cmd CC] [--print] -- srcs...
--config | Config path (required) |
--scan DIR | Collect sources recursively under DIR instead of listing them |
--format | make (default) prints a CFLAGS_INSTRUMENT = … assignment for $(eval $(shell …)); raw prints only the flags |
--compiler | Target toolchain. auto (default) detects it by running --compiler-cmd |
--compiler-cmd | Compiler command used for detection. Default: $CC, else cc |
--print | Human-readable selection summary on stderr |
A selective config under a detected Clang exits with an explanation rather than emitting GCC-only flags. A failed detection is treated as GCC, so detection can never break a build that would otherwise work.
analyze
callsight analyze [tracedir] [--exe BIN] [--top N]
[--format text|json|folded|chrome|callers]
[--addr2line CMD] [--subtract-overhead]
tracedir | Directory of trace.*.bin files. Default traces |
--exe | Instrumented binary for addr2line. Default: the single *.instr under ./bin or . |
--top | Rows per table (default 20). With --format json, 0 means every row |
--format | text tables (default), json for tooling, folded collapsed stacks for flamegraph.pl and speedscope, chrome for ui.perfetto.dev, callers for hot call sites |
--addr2line | The addr2line to use. Default $CALLSIGHT_ADDR2LINE, else the host one. A cross-compiled binary needs its own toolchain's copy — host binutils cannot read a foreign ELF |
--subtract-overhead | Deduct the runtime's own measured per-hook cost from the reported times |
Summary traces (TRACE_MODE=summary) are detected automatically and merged.
They hold per-function totals rather than call paths, so folded,
chrome and callers are refused with an explanation instead of
producing an empty result.
diff
callsight diff BASE.json NEW.json [--key self_ms] [--threshold N] [--fail-over PCT]
Compares two --format json reports function by function. Exact call counts
make this a real comparison rather than two samples that happened to land differently, so
it works as a build gate:
callsight diff base.json new.json --fail-over 10 # exit 1 on a >10% regression
--key | Metric to compare. Default self_ms |
--threshold | Ignore changes smaller than this, in --key units |
--fail-over | Exit non-zero if any function regresses by more than this percentage |
doctor
callsight doctor [project] [--config trace.config] [--dir traces]
Checks the compiler and whether it is GCC, addr2line, that
-finstrument-functions is accepted, that the config selects something, and
that the trace directory is writable with room to spare. Exits non-zero if anything is
actually broken; observations that are not problems are marked note.
serve · ui · provision
callsight serve [--host 0.0.0.0] [--port 9001] [--out traces]
[--max-mb 4096] [--seg-mb 256]
callsight ui [--host 127.0.0.1] [--port 8321]
callsight provision [--force]
serve bounds and rotates its output per connection: a device streaming for
an hour should not fill the analysis host either.
provision reports where ctags comes from and installs the bundled static
copy into $CALLSIGHT_HOME/bin (default ~/.callsight/bin),
verifying its checksum. --force installs it even when a system ctags is
present.
Build integrations
Both do the same two things: generate the flags from your config and source list, and
compile the runtime without instrumentation so hooks cannot recurse.
Neither forces -no-pie any more — link the way you ship.
GNU Make
CALLSIGHT_DIR ?= callsight
include $(CALLSIGHT_DIR)/Makefile.callsight
Expects SRCS, BUILDDIR, BINDIR,
TARGET, CC, CFLAGS_SYMBOLS and LDFLAGS.
Defines:
CFLAGS_INSTRUMENT— yourCFLAGS_SYMBOLSplus hooks plus the exclude listsTRACE_OBJ— the runtime object, built without instrumentation
| Variable | Default | Meaning |
|---|---|---|
CALLSIGHT_DIR | callsight | Where trace.c/trace.h live |
CALLSIGHT_CONFIG | trace.config | Selection config |
CALLSIGHT | callsight | Command that prints the flags — point it at python3 …/cli.py to run from a source checkout |
When make instrument is requested and no flags could be generated, the
build stops with an explanation instead of silently producing an uninstrumented binary. A
normal make still succeeds even if callsight isn't installed.
CMake
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/callsight")
include(CallSight)
callsight_instrument(<target>)
| Cache variable | Default | Meaning |
|---|---|---|
CALLSIGHT_INSTRUMENT | OFF | Apply hooks to callsight_instrument() targets |
CALLSIGHT_CONFIG | <src>/trace.config | Selection config |
CALLSIGHT_COMMAND | callsight | Flag generator; a ;-list works, e.g. -D"CALLSIGHT_COMMAND=python3;/path/cli.py" |
CALLSIGHT_NO_PIE | OFF | Link instrumented targets with -no-pie. Not needed: the trace header carries the load bias |
CALLSIGHT_COMMAND is a cache variable — pass it with
-D. A plain set() before include(CallSight) gets
shadowed by the cache definition under CMP0126.
Environment variables
| Variable | Default | Meaning |
|---|---|---|
TRACE_ENABLE | off | 1 enables collection; hooks are inert otherwise |
TRACE_DIR | ./traces | Output directory, resolved to an absolute path at startup so a later chdir() cannot scatter segments |
TRACE_MODE | events | summary aggregates in-process and writes only totals — constant memory and constant output, whatever the run length |
TRACE_MAX_MB | 512 | On-disk budget for the process. 0 = unlimited |
TRACE_FULL | stop | At the budget: stop keeps the start, wrap keeps the end |
TRACE_SEG_MB | 32 | Segment size, i.e. rotation granularity |
TRACE_MIN_FREE_MB | 64 | Stop below this much free space. 0 disables the check |
TRACE_MAX | 0 (unlimited) | Global event cap, as an upper bound |
TRACE_CLOCK | auto | auto uses the invariant cycle counter where the hardware has one, else CLOCK_MONOTONIC; mono, raw and tsc force it |
TRACE_THREADS | unset (all) | Comma-separated globs matched against thread names |
TRACE_COUNTERS | — | Path to the hardware-counter map (see counters). Default: <TRACE_DIR>/callsight.counters; none disables counting |
TRACE_SHM | unset | Streaming mode: POSIX shm ring name |
TRACE_SHM_SIZE | 16 MiB | Ring capacity in bytes |
CC | cc | Read by callsight flags for compiler detection |
CALLSIGHT_ADDR2LINE | addr2line | Symbolizer for analyze; set it to a cross-toolchain copy for foreign binaries |
CALLSIGHT_HOME | ~/.callsight | Where the bundled ctags is installed |
File formats
Trace files
trace.<pid>.<tid>.<seq>.bin (file mode) or
trace.stream.<id>.<seq>.bin (streamed). An 80-byte version 2
header followed by fixed 32-byte event records in the
agent's byte order. The
architecture page documents the record; a file with a
bad magic, unknown version or mismatched event size is skipped with a warning, and a
truncated final record is tolerated.
| Field | Meaning |
|---|---|
magic, version, event_size | The first 16 bytes are laid out exactly as version 1, so any reader can identify the file before it knows the rest |
header_size | Bytes to the first event. Readers skip to it rather than assuming a size, which is what lets a later version add fields without breaking this one |
flags | Bit 0: timestamps are raw ticks, not nanoseconds. Bit 1: this capture rotated. Bit 2: written by a big-endian agent |
load_bias | The PIE relocation offset; analyze subtracts it to get link addresses. Zero for a -no-pie link |
tick_hz, t0_ticks, t0_ns | Clock calibration. A closing anchor written at exit lets the rate be derived across the whole run |
hook_ns | The runtime's own measured per-hook cost, for --subtract-overhead |
pid, seq | Owning process and segment number |
Version 1 files still analyze. They carry a bare 16-byte header, nanosecond timestamps, no load bias and no markers.
Summary files
trace.summary.<pid>.<tid>.bin: an MLSUMRY header
followed by one 688-byte record per function — address, calls, inclusive and self time,
min, max, and a 160-bucket duration histogram (four sub-buckets per octave). Merged across
threads by analyze.
Folded stacks
main;handle_request;parse_headers 148230
main;handle_request 92117
One line per distinct call path: semicolon-separated frames, then self time in nanoseconds. Read directly by flamegraph.pl and speedscope.
JSON report
Summary counters (events, threads, functions,
span_ms, unmatched_exits, unclosed_enters), a
rows array sorted by self time, a per_thread array, plus
tool and version. See the
analysis page for a full example.
Agent portability
The agent runs where your code runs; the analysis host is usually an x86-64
workstation. Those two can disagree about word size and byte order, so the format
is self-describing rather than fixed: the agent writes its native byte
order and analyze detects and swaps. The device is the
constrained side of the system and the host is not, so the host does the work.
Detection needs no extra field. Every header opens with a byte-string magic,
which reads the same either way, followed by a small u32 version — so a
version that does not fit in 16 bits is a byte-swapped one and nothing else.
TRACE_HF_BIGENDIAN records it explicitly as well, which costs nothing
and makes a hexdump legible. callsight serve relays event bytes
untouched and writes its file headers in the device's order, so a
streamed segment is never half one thing and half the other.
| Tested combination | How |
|---|---|
| 32-bit little-endian (ARMv7) | Cross-built in CI, run under qemu-user, and the resulting trace analyzed on x86 — the real deployment shape. The probe's call graph is fixed, so the check is that the counts are exactly what the same source produces natively, not merely that the file parsed. |
| 32-bit big-endian (PowerPC) | |
| 64-bit big-endian (s390x) | |
| 64-bit little-endian (x86-64, aarch64) | The native test suites. |
Big-endian ARM has no cross toolchain in Debian, so it is not a CI leg. Both axes are covered independently — 32-bit ARM by the ARMv7 build, big-endian by PowerPC and s390x — and the runtime contains no architecture-specific endian assumption.
The cycle-counter clock is used on x86-64 and aarch64 only. 32-bit ARM has the
same generic timer, but reaching it needs CP15 instructions that raise
SIGILL where the timer is absent — and a runtime check cannot avoid
that, because the check is the instruction. Those agents use
clock_gettime, which is in the vDSO on ARMv7 anyway.
Two build details matter on 32-bit agents, and both build fragments handle them for you.
The runtime is compiled with _FILE_OFFSET_BITS=64, without
which ftruncate fails past 2 GB and statvfs can return
EOVERFLOW. And the byte budget uses a 64-bit atomic: ARMv7 and up
inline it, while ARMv5/v6, MIPS32 and PowerPC32 need -latomic, which
the Make and CMake integrations probe for rather than guess.
Every on-disk and on-wire struct carries a compile-time size assertion, so an ABI whose padding rules differ from ours fails the build instead of quietly producing files the analyzer misreads.
Known limitations
- Selective instrumentation is GCC-only. Clang has
-finstrument-functionsbut not the exclude lists (LLVM #15627); under Clang only an unfiltered config builds. - Linux only. The runtime uses
SYS_gettid,pthread_getname_npand POSIX shared memory. - Inlined functions emit no hooks — there is no call boundary to hook.
- Overhead is roughly 12 ns per event with the cycle-counter clock and
16 ns with
clock_gettime, measured bytests/bench/run_bench.py. An instrumented build with tracing off costs well under a nanosecond per hook. This is still a profiling build, not a production one. - A crashed or killed process loses each thread's buffered tail. Clean
exits flush everything;
callsight run --timeoutsendsSIGTERMfirst for that reason. - Capture is bounded by default at 512 MB (capture limits). Reaching a bound is reported in the trace, not left for you to infer.
- Summary mode has no call paths. It records per-function totals, so flame graphs, the timeline export and hot call sites need event mode.
- The compiler's exclude matching is substring-based, so unusually
overlapping directory or symbol names can over-match. callsight guards the
auto-generated function excludes against this and warns, but hand-written
exclude-funcentries are yours to keep specific. - Position-independent executables are supported. The runtime records
the load bias in every trace header, so
-no-pieis no longer required — which matters, because forcing it would have you profiling a binary built differently from the one you ship. Version 1 traces predate this and still need it. - Static call-graph resolution does not follow function pointers, macro-generated calls, or C++ dynamic dispatch.