TraceRazor

Token efficiency audit layer for production AI agents

Rust Python SDK v0.2.0 Apache 2.0

The Core Insight

A workload-dependent share of agent tokens is structurally redundant — in our own audits of 24 public traces, step redundancy alone runs 36–41%. TraceRazor finds the waste and tells you exactly how to fix it.

13
Efficiency Metrics
<5ms
Analysis Time
0
API Keys Needed
8
Auto-Fix Types

TraceRazor reads a completed agent trace (JSON), scores it across thirteen efficiency metrics, and outputs a 0-100 Token Audit Score (TAS) with fix patches. No agent modification required. Everything runs offline.

How It Works

Agent Trace
(JSON)
→
Parse & Ingest
→
13 Metrics
→
Weighted Score
→
TAS 0-100
+ Fixes

The pipeline takes a raw trace from any agent framework (LangGraph, CrewAI, OpenAI Agents, or custom), computes all metrics locally, and produces a graded report with actionable fix patches and savings estimates.

GradeTAS RangeMeaning
Excellent90-100Minimal recoverable waste
Good70-89Minor addressable inefficiency
Fair50-69Significant structural waste
Poor0-49Fundamental issues, restructure needed

The 13 Metrics

Each metric targets a specific category of token waste. All normalised to 0-1 (higher = better) before being combined into the TAS composite score.

TAS 0-100 SRR Step Redundancy LDI Loop Detection TCA Tool Accuracy RDA Reasoning Depth ISR Info Sufficiency TUR Token Utilisation CCE Context Efficiency DBO Decision Optimality VDI Verbosity Density GAR Goal Advancement CSD Semantic Continuity SHL Sycophancy/Hedging CCR Compression Ratio

Structural Metrics (9)

CodeMetricDetectsTarget
SRRStep Redundancy RateNear-duplicate steps via BoW Jaccard similarity<15%
LDILoop Detection IndexRepeated tool-call cycles<0.10
TCATool Call AccuracyFailed tool calls and retries>85%
RDAReasoning DepthOver-deep reasoning for task complexity>0.75
ISRInformation SufficiencySteps lacking novel information>80%
TURToken UtilisationOff-task token spending>0.35
CCEContext EfficiencyDuplicate context across steps>0.60
DBODecision OptimalitySub-optimal tool sequences>0.70
CSDSemantic ContinuityReasoning drift mid-trace≥0.60

Verbosity Metrics (3)

CodeMetricDetectsTarget
VDIVerbosity DensityFiller words, low-substance content>0.60
SHLSycophancy/HedgingExcessive politeness and caution<0.20
CCRCompression RatioHighly compressible text<0.30

Goal Advancement (1)

CodeMetricDetectsTarget
GARGoal Advancement RatioSteps that don't advance toward the goal≥0.40

Auto-Fix Generation

When a metric fails, TraceRazor generates a targeted fix patch with estimated token savings. Eight fix types cover all common waste patterns:

TCA Misfire CCE Bloat LDI Loop RDA Over-Depth VDI Fail + AVS SHL Fail + AVS CCR Fail + AVS Reformulation Flag tool_schema context_compression termination_guard prompt_insert verbosity_reduction hedge_reduction caveman_prompt_insert reformulation_guard Fix tool schema Summarise context Break loop Add step-count rule Remove filler Strip preamble Ultra-concise mode Skip re-stating

Optimization Validation (IAR)

After applying fixes and re-running your agent, the Adherence Score checks whether each fix type actually improved its target metric.

Audit #1
Identify waste
→
Apply Fixes
→
Audit #2
Re-run agent
→
Compare
→
Adherence
Score ≥75%
Fix TypeValidates Against
tool_schemaTCA (Tool Accuracy) improved
context_compressionCCE (Context Efficiency) improved
termination_guardLDI (Loop Detection) improved
prompt_insertRDA (Reasoning Depth) improved
verbosity_reductionVDI (Verbosity Density) improved
hedge_reductionSHL (Sycophancy) improved
caveman_prompt_insertCCR (Compression) improved
reformulation_guardISR (Information Sufficiency) improved

Architecture

tracerazor-core

13 metrics, scoring engine, fix generation, reports. Zero network dependencies. This is where all analysis happens.

tracerazor-ingest

Parsers for raw JSON, LangSmith, and OpenTelemetry trace formats. Converts any format to the internal Trace struct.

tracerazor-semantic

Bag-of-words similarity backend. Optional LLM embeddings (OpenAI, Anthropic, or compatible) for enhanced mode.

tracerazor-store

SurrealDB persistence: traces, known-good-paths, baselines, and anomaly detection rolling windows.

tracerazor-server

Axum REST API + WebSocket live events + embedded React dashboard. Serves the full UI at port 8080.

tracerazor-proxy

Four-layer guardrail interceptor: semantic preservation, scope whitelist, budget injection, verbosity directive.

tracerazor-cli

CLI entry point: audit, optimize, bench, simulate, compare, cost, export.

Python Integrations

SDK (pip install tracerazor) + adapters for LangGraph, CrewAI, and OpenAI Agents. All on PyPI v0.2.0.

Your Agent (any framework) Ingest Parse trace Core Engine 13 Metrics Scoring Fix Generation Report TAS + Fixes + Savings Store Baselines + KB Semantic (optional)

CLI Commands

CommandPurpose
tracerazor auditScore a trace; optionally gate on --threshold
tracerazor optimizeRewrite system prompt via LLM to eliminate detected waste
tracerazor benchCompare before/after traces, verify savings
tracerazor simulateProject TAS impact of removing/merging steps
tracerazor compareSide-by-side metric delta between two traces
tracerazor costMonthly savings estimate across trace set
tracerazor exportForward trace to OTEL or webhook

Python Integrations

All published to PyPI v0.2.0. Each wraps the core analysis engine with framework-native callbacks.

tracerazor (SDK)

pip install tracerazor
Framework-agnostic. Use Tracer context manager with any Python agent.

tracerazor-langgraph

pip install tracerazor-langgraph
Native BaseCallbackHandler. Auto-captures LLM and tool events.

tracerazor-crewai

pip install tracerazor-crewai
CrewAI callback adapter. Traces tasks, agent actions, and tool calls.

tracerazor-openai-agents

pip install tracerazor-openai-agents
OpenAI RunHooks adapter. Captures agent ends, tool use, handoffs.

Multi-Agent Support

Each agent in a workflow gets its own tracer and independent report. Aggregate metrics across all agents for workflow-level visibility.

Query User input Triage TAS: 82 Resolution TAS: 74 Escalation TAS: 79 Knowledge TAS: 88 80.8 Avg TAS

Benchmark Results

5 synthetic traces, each isolating a waste pattern. Run with python benchmarks/run_benchmarks.py.

bloated clean looping reformulator verbose 87.9 89.4 69.4 86.9 77.1 0 50 100 TAS
TraceTASGradeTokensWasteSavingsFixes
bloated-agent87.9Good2,32030%6932
clean-agent89.4Good86034%2891
looping-agent69.4Fair1,71035%6033
reformulator-agent86.9Good1,34032%4331
verbose-agent77.1Good2,76053%1,4552

Summary

82.1
Average TAS
8,990
Total Tokens
3,473
Tokens Saveable (39%)
125+
Tests Passing

Test Coverage

CrateTestsCovers
tracerazor-core125All 13 metrics, scoring, fixes, IAR, simulation, reports
tracerazor-ingest3Raw JSON, LangSmith, OTEL parsers
tracerazor-semantic5BoW similarity, tokenization
tracerazor-store9CRUD, baselines, anomaly detection
tracerazor-server13REST API endpoints, WebSocket
tracerazor-proxy12All 4 proxy layers

Quick Start

Option 1: Docker

git clone https://github.com/ZulfaqarHafez/tracerazor
cd tracerazor
docker compose up --build
# Dashboard at http://localhost:8080

Option 2: Cargo

cargo build --release
./target/release/tracerazor audit trace.json

Option 3: Python

pip install tracerazor

from tracerazor_sdk import Tracer

with Tracer(agent_name="my-agent") as t:
    t.reasoning("Analyzing request...", tokens=200)
    t.tool("search", params={}, output="result", success=True, tokens=150)

report = t.analyse()
print(report.summary())
# TAS 78.5/100 [GOOD] | 4 steps, 350 tokens | Saved 90 tokens (26%)

CI/CD Gate

tracerazor audit trace.json --threshold 75
# Exit code 1 if TAS < 75

TraceRazor v0.2.0 | Apache 2.0 | Author: Zulfaqar Hafez
github.com/ZulfaqarHafez/tracerazor