The four stages
1 · Selection becomes compiler flags
callsight flags reads trace.config and the source list your
build system passes it, and prints the flag string the compiler should use. Both build
integrations call it on every build, so the selection is always current.
- File selection maps to
-finstrument-functions-exclude-file-list. Sources that are not selected are listed as exclusions — GCC matches the list against the file each function is defined in, which is why header paths work for silencing inline helpers. - Function selection maps to
-finstrument-functions-exclude-function-list, a substring match on symbol names. include-funccompiles down to those two primitives: the files defining the subtree are instrumented, and every other function defined in those files is added to the function exclude list.
The static call graph
include-func needs to know what a function calls. callsight uses a
deliberately lightweight heuristic parser (comments and string literals blanked,
definitions matched at line starts, call sites matched inside the body) rather than pulling
in libclang — adoption stays a single uv tool install with no toolchain
dependency.
The trade-off is documented rather than hidden: function pointers, macro-generated calls and C++ dynamic dispatch are not followed. The expansion is breadth-first, so a depth limit always measures the shortest path to each function.
2 · The hook runtime
trace.c is self-contained C with no dependencies beyond pthreads, compiled
without -finstrument-functions and with
no_instrument_function on every function, so the hooks can never trigger
themselves.
- All state is per-thread (TLS): an 8192-event buffer, the output handle, the thread id. There are no locks on the hot path in file mode.
- Inert unless enabled. Without
TRACE_ENABLE=1each hook is one predictable branch and a return. - No symbolization at runtime. Events carry raw addresses; names are resolved offline. That is what keeps a hook at tens of nanoseconds.
- No malloc, no I/O on the hot path. The buffer flushes when full, at thread exit (via a pthread key destructor), and at process exit.
- Reentrancy guard per thread, so a hook can never recurse through the code it calls.
The event record
32 bytes, fixed layout, little-endian — the same on disk and on the wire.
| Field | Type | Meaning |
|---|---|---|
ts_ns | u64 | CLOCK_MONOTONIC_RAW nanoseconds |
func_addr | u64 | Address of the entered/exited function |
caller_addr | u64 | Return address in the caller |
tid | u32 | Kernel thread id |
kind | u8 | 0 = enter, 1 = exit |
_pad | u8[3] | Padding to 32 bytes |
Each trace file starts with a 16-byte header: an MLTRACE magic, a format
version, and the event size — so a mismatched or truncated file is detected and skipped
rather than misread.
3 · The analyzer
Analysis is a single streaming pass. Events are read in blocks and matched as they arrive, so analyzer memory tracks the number of functions and threads, not the number of events — a multi-million-event trace costs a few MB.
- Match per thread. Each thread has its own stack. An exit pops back
to the nearest matching enter, closing any frames left dangling above it (which happens
after a
longjmp, or when a buffer tail is lost). - Attribute time. Inclusive time is exit minus enter; the duration is also credited to the parent frame's "children" total, and self time is inclusive minus children.
- Resolve symbols. All distinct addresses go to
addr2linein batches, once, at the end —staticfunctions included. - Emit. Text tables, JSON, or folded stacks. Folded output is accumulated during the same pass, keyed by the tuple of addresses on the live stack.
Because everything is derived from the same pass, the flame graph and the
self_ms column are guaranteed to agree.
Compiler mechanisms surveyed
What each compile-time instrumentation mechanism can do, what it costs, and how it maps to callsight. Overhead figures are order-of-magnitude, per event, assuming a lean hook.
| Mechanism | Compilers | Granularity | Runtime toggle | Overhead/event | Verdict |
|---|---|---|---|---|---|
-finstrument-functions | GCC, Clang | Function entry/exit | No (compile-time) | ~30–60 ns | Default backend |
-finstrument-functions-after-inlining | Clang | Post-inline entry/exit | No | ~30–60 ns | Clang enhancement |
-pg (mcount/gprof) | GCC, Clang | Function entry | No | ~50–100 ns | Legacy, rejected |
-fpatchable-function-entry | GCC ≥ 8, Clang ≥ 11 | NOP sleds at entry | Yes (patch sleds) | ~0 off, few ns on | Roadmap candidate |
-fsanitize-coverage | Clang, GCC ≥ 12 | Edge / PC guard | No | ~5–20 ns | Alternative backend, evaluating |
XRay (-fxray-instrument) | Clang | Entry/exit sleds | Yes (official API) | ~0 when off | Design reference |
-fprofile-arcs (gcov) | GCC, Clang | Edge counters | No | Counter inc | Out of scope (no timing) |
| GCC plugin API | GCC (version-locked) | Arbitrary (GIMPLE) | Possible | Varies | Non-goal |
-finstrument-functions — the current backend
Emits a call to __cyg_profile_func_enter(this_fn, call_site) and
__cyg_profile_func_exit(...) at every function boundary, static
functions included. Selection happens at compile time through the two exclude lists, both
substring-matched, with the file list matched against the file a function is defined
in. Excluded code emits no hook at all, so selection is free at runtime.
Caveats: inlined functions emit no hooks, because no call boundary exists; hooks fire
even when you don't want data, so runtime gating costs a flag check per event; and
-no-pie (or offset bookkeeping) is needed to map addresses back to symbols.
-finstrument-functions but has never taken the exclusion patches
(LLVM #15627),
and its driver rejects unknown arguments outright. So a selective config requires GCC;
under Clang only an unfiltered "instrument everything" config compiles.
callsight flags detects the toolchain (--compiler-cmd, passed by
both build integrations) and reports this before the build rather than letting it fail
once per translation unit. Giving Clang real file-level selection would mean applying
-finstrument-functions per translation unit instead of globally — a
build-integration change, not a flag change.
-finstrument-functions-after-inlining
Same hooks, inserted after inlining, so functions that survived inlining get hooked even when they were inlined at some call sites, and you see the real optimized call graph. A candidate extra flag on Clang toolchains; it needs an analyzer-side note that the call graph differs from the source-level one.
-pg / mcount (gprof)
The original: an entry-only hook into mcount, plus flat-profile sampling.
Call-graph arcs only — no per-call exit timing — with PLT and shared-library blind spots
and effectively unmaintained semantics under modern optimization. Documented for
completeness; callsight does not use it.
-fpatchable-function-entry=N,M
Emits N NOPs at function entry (M of them before the prologue)
plus a __patchable_function_entries section listing their addresses — the
mechanism behind the Linux kernel's ftrace and uftrace's dynamic mode. A runtime can patch
those sleds into hook calls and back, giving genuinely zero-cost-when-off, toggleable
tracing. The strongest candidate for "trace a live production process for five seconds".
Costs: code-size growth, patching machinery, and entry-only hooks — exit timing needs a
return-address trampoline, which is the hard part.
-fsanitize-coverage
compiler-rt coverage callbacks (__sanitizer_cov_trace_pc_guard per edge,
with a per-guard toggle word in the guard variant), aimed at fuzzers. Leaner than function
hooks and gives basic-block-transition granularity, but there is no caller address and no
exit event, so timing must be reconstructed, and guard tables need the compiler-rt runtime.
Clang has the full menu; GCC ≥ 12 has trace-pc and trace-cmp only.
Worth an experiment as a low-overhead "which edges ran" backend, not a replacement for call
timing.
LLVM XRay
Clang-only, and the most production-grade design in this list: sleds at function
entry/exit patched at runtime through a supported API (__xray_patch()),
per-function selection at compile time, and a logging library writing binary
flight-recorder traces with tooling (llvm-xray). The closest existing model for
callsight's streaming design — its sled layout and FDR buffering are the reference. Not
usable as a default, being Clang-only.
gcov / -fprofile-arcs
Edge counters for coverage, not timing. It answers "did this line run, how often", not "how long did it take". A possible complement if a coverage view is ever added; out of scope for tracing.
GCC plugin API
Loadable modules running custom GIMPLE/RTL passes: arbitrary injection, maximum control. But plugins are locked to the exact GCC version, the C++ API is fragile, Clang has no equivalent, and distribution is a nightmare. A deliberate non-goal.
Not compile-time (context only)
perfsampling — no build integration at all; finds hot functions, not call sequences. Complementary, and always available.- eBPF/uprobes, Intel PT — runtime tracing with no rebuild; heavy machinery and root requirements. Out of scope.
- uftrace — a userspace tracer built on
-pg/-finstrument-functionsplus its own libmcount runtime, with a dynamic mode on-fpatchable-function-entry. The closest relative project; callsight differs in zero-runtime-dependency adoption and the config-file selection workflow.
Repository layout
| Path | What lives there |
|---|---|
src/callsight/flags.py | Config parsing, pattern matching, exclude-list generation, compiler detection |
src/callsight/callgraph.py | The heuristic static call graph behind include-func |
src/callsight/analyze.py | Streaming trace reader, enter/exit matcher, report formats |
src/callsight/runtime/ | trace.c, trace.h, trace_shm.h — copied into adopted projects |
src/callsight/stream/ | trace_stream.c plus vendored single-file zstd |
src/callsight/share/, cmake/ | The Make fragment and the CMake module |
src/callsight/ui/ | The optional FastAPI web UI |
tests/matrixlab/ | Multi-threaded C11 demo workload; the end-to-end fixture |