tdfs4ds.benchmark.report¶
tdfs4ds.benchmark.report¶
Render a compelling, self-contained HTML report from the dict produced by
:func:tdfs4ds.benchmark.run_benchmark (or its saved JSON).
The HTML is fully offline — inline SVG charts + CSS, no external JS — so it renders identically in any browser and embeds directly in the admin panel.
:func:build_benchmark_html, :func:write_benchmark_report. Also a CLI:
python -m tdfs4ds.benchmark.report report.json [-o out.html][--open]
module-attribute
¶
_RATING_LABEL = {'excellent': 'Excellent', 'good': 'Good', 'fair': 'Fair', 'poor': 'Slow', 'na': 'N/A'}
module-attribute
¶
_GUIDE = {'parse': '<p>A dataset is a view joining one sub-query per feature (multi-join) or per storage table and process version (grouped). Before a request reads anything, Teradata parses that view text and optimises the plan. For a wide dataset this alone can take seconds.</p><ul><li><b>First</b> includes the dictionary look-ups of a cold parse; <b>median</b> is the typical cost.</li><li>The <b>request cache</b> can reuse a plan for an identical request, so a repeated lookup may not pay it — but every new key literal or new query text does.</li><li><b>View DDL size</b> and <b>plan steps</b> grow with the number of sub-queries. The grouped strategy reads each storage table once per process version instead of once per feature, which shortens both — measured on one dataset: 5.8k → 3.0k characters and 627 → 190 ms.</li></ul>', 'schema': '<p>The dataset is a pre-joined view. Each lookup returns one row spanning all these columns, so <b>row width drives transfer cost</b>: many wide <code>VARCHAR</code>/<code>DECIMAL</code> features mean more bytes per lookup and a higher latency floor, independent of how fast the index seek is.</p><ul><li><b>Features</b> = the non-key columns served. <b>Entity key</b> = the grain you look up by.</li><li>If latency looks high but the plan is single-AMP, suspect a wide row (large bulk-read bytes), not the seek.</li></ul>', 'explain': "<p>This is the <b>structural proof</b> behind every latency number — it inspects the query plan, not a stopwatch.</p><ul><li><span class='g'>✓ Single-AMP access</span> / <span class='g'>✓ Unique primary index</span> — ideal: the row is found on one AMP via the primary index. This is what makes point lookups fast.</li><li><span class='b'>✗ All-AMPs retrieve</span> / <span class='b'>✗ All-rows scan</span> — the engine touches every AMP / scans the whole table. Acceptable for bulk reads, but a red flag for single-key serving — latency will not scale and will rise with table size.</li></ul><p>If the latency tables look good <i>and</i> this is single-AMP, the numbers are trustworthy. If lookups are single-AMP-clean but still slow, the cost is network/transfer, not the plan.</p>", 'single_key': "<p>The core serving pattern: one entity key → one feature row. Read the <b>percentiles, not the mean</b> — p50 is the typical experience, p95/p99 are the tail your slowest users feel.</p><ul><li><b>Warm</b> rotates a small hot key set, so the row/index pages are cached — this is your <i>best-case</i> served latency.</li><li><b>Cold</b> spreads across the whole key space, forcing disk/cache misses — closer to <i>real production</i> where any key can be requested.</li><li>A big warm→cold gap means you are I/O-bound on cache misses; a small gap means the working set fits in memory.</li><li>Colors: <span class='g'>green <150 ms</span>, <span class='y'>amber 150–500 ms</span>, <span class='r'>red >500 ms</span>. These include network + driver overhead.</li></ul>", 'batch_strategies': "<p>Two ways to fetch many keys in one shot, measured side-by-side:</p><ul><li><b>in_list</b> — a single <code>SELECT … WHERE (a=… AND b=…) OR …</code> predicate. Simple, but the <b>SQL text grows with the key count</b>: a wide composite-key batch hits Teradata's ~1 MB request parser limit (Error 5639) and is shown here as <i>skipped (parser limit)</i>.</li><li><b>volatile_join</b> — the keys are bulk-loaded into a session <b>volatile table</b> whose <code>PRIMARY INDEX</code> matches the feature-store storage tables, then <code>INNER JOIN</code>ed. Keys travel as <b>data, not SQL text</b>, so there is no parser limit, and the join is AMP-local (co-located).</li></ul><p>Rule of thumb: in_list wins for small batches (no table setup); volatile_join wins as the key count grows and is the <b>only</b> option once the predicate would exceed the parser limit.</p>", 'batch': '<p>The API-scoring pattern: N keys fetched in <b>one</b> round-trip. Compare the per-batch latency to N separate single-key lookups — batching amortizes the network round-trip, so a batch of 100 should be far cheaper than 100 individual calls.</p><ul><li>Watch how p50 grows with batch size. Sub-linear growth = batching is paying off.</li><li>For a composite key the predicate is an <code>OR</code> of <code>AND</code> groups; very large batches can add parse/optimize time — if the biggest batch jumps disproportionately, that parse cost (not retrieval) is usually why.</li></ul>', 'time_travel': "<p>Cost of querying the dataset <b>AS OF a historical instant</b> vs. the current-time baseline. The baseline is the unmodified <code>CURRENT VALIDTIME</code> view; each <code>AS OF</code> row re-binds the view's validtime to a past timestamp.</p><ul><li>Compare each <code>AS OF</code> p50 against the <b>baseline</b> row — the difference is the time-travel overhead.</li><li>A small delta means historical queries are about as cheap as current ones; a large delta means scanning historical row versions is materially more expensive.</li></ul>", 'large_batch': '<p>The MPP sweet spot: large scans pulled fully to the client. Here the headline is <b>throughput (rows/sec)</b>, not per-query latency.</p><ul><li>Higher rows/sec is better. A plateau across batch sizes means you have hit a bottleneck — usually client-side fetch/deserialize or network bandwidth, not the database.</li><li>If throughput is flat while latency climbs with size, you are transfer-bound: the engine produced rows fast but the client drained them slowly.</li></ul>', 'concurrent': '<p>How the system behaves under parallel clients — the <b>QPS ceiling</b> and how latency degrades as load rises.</p><ul><li>Ideally QPS rises roughly linearly with threads while p50 stays flat. When QPS stops climbing but latency keeps rising, you have found <b>saturation</b> — adding clients only queues work.</li><li>On a shared / trial system this ceiling can be low (limited AMPs/sessions); a drop in QPS at higher concurrency usually reflects the test box, not the dataset design.</li><li>Each thread opens its own session, so high concurrency also tests session-setup capacity.</li></ul>', 'server_side': "<p>Splits client-perceived latency into <b>server execution time</b> (from DBQL: <code>FirstRespTime − StartTime</code>) vs. everything else (network round-trip + driver).</p><ul><li><b>Server p50</b> is the database's own time for the <i>same single-key lookups</i> as the warm/cold tests — not a mix of all your queries.</li><li><b>Overhead = client p50 − server p50</b> is the network + driver tax. Large overhead with small server time means you are far from the database (run the benchmark closer / same region).</li><li><b>AMP CPU</b> = compute the AMPs spent; <b>Parser CPU</b> = optimize/parse cost (high parser CPU on tiny lookups hints at plan-cache misses).</li><li>A negative or flagged-unreliable overhead means DBQL could not be scoped to the lookups — ignore the number, not the percentiles.</li></ul>"}
module-attribute
¶
CSS = '\n:root{\n --bg:#0b1020; --panel:#141b2e; --panel2:#1b2540; --ink:#e8edf7; --muted:#93a0bd;\n --line:#26314f; --accent:#ff8a3d; --accent2:#4f8cff;\n --ok:#34d399; --warn:#f87171; --good:#38bdf8; --fair:#fbbf24; --poor:#fb7185;\n --excellent:#34d399;\n}\n*{box-sizing:border-box}\nbody{margin:0;background:linear-gradient(180deg,#0b1020,#0d1426 40%,#0b1020);\n color:var(--ink);font:15px/1.55 -apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Helvetica,Arial,sans-serif}\na{color:var(--accent2)}\n.wrap{max-width:1080px;margin:0 auto;padding:32px 22px 80px}\nheader.hero{background:radial-gradient(1200px 400px at 10% -20%,rgba(255,138,61,.22),transparent 60%),\n linear-gradient(135deg,#16213e,#0e1730);border:1px solid var(--line);\n border-radius:18px;padding:30px 32px;margin-bottom:26px;position:relative;overflow:hidden}\nheader.hero:before{content:"";position:absolute;right:-60px;top:-60px;width:240px;height:240px;\n background:radial-gradient(circle,rgba(79,140,255,.25),transparent 65%)}\n.kicker{letter-spacing:.16em;text-transform:uppercase;font-size:12px;color:var(--accent);font-weight:700}\nh1{margin:.25em 0 .15em;font-size:30px;letter-spacing:-.02em}\n.dataset{font-size:18px;color:var(--ink);font-weight:600}\n.meta-line{color:var(--muted);font-size:13.5px;margin-top:10px;display:flex;flex-wrap:wrap;gap:6px 18px}\n.meta-line b{color:var(--ink);font-weight:600}\n.note{margin-top:14px;padding:10px 14px;background:rgba(79,140,255,.10);\n border:1px solid rgba(79,140,255,.30);border-radius:10px;color:#cdd9f5;font-size:13px}\n\n.kpi-grid{display:grid;grid-template-columns:repeat(auto-fit,minmax(165px,1fr));gap:14px;margin:4px 0 6px}\n.kpi{background:var(--panel);border:1px solid var(--line);border-radius:14px;padding:16px 16px 14px;position:relative}\n.kpi-label{font-size:12px;color:var(--muted);text-transform:uppercase;letter-spacing:.06em}\n.kpi-value{font-size:26px;font-weight:700;margin-top:6px;letter-spacing:-.02em}\n.kpi-sub{font-size:12px;color:var(--muted);margin-top:4px}\n.kpi:before{content:"";position:absolute;left:0;top:14px;bottom:14px;width:3px;border-radius:3px;background:var(--accent2)}\n.kpi-excellent:before,.kpi-good:before{background:var(--ok)}\n.kpi-fair:before{background:var(--fair)}\n.kpi-poor:before{background:var(--poor)}\n\nsection.card{background:var(--panel);border:1px solid var(--line);border-radius:16px;\n padding:22px 24px;margin:20px 0}\nh2{margin:0 0 4px;font-size:20px;letter-spacing:-.01em}\n/* Explicit colour: inside the admin app the report sits in a card whose h2 rule is\n navy, which on this dark panel left every section title unreadable. */\nsection.card h2,header.hero h1{color:var(--ink)}\nh4{margin:18px 0 8px;font-size:13px;color:var(--muted);text-transform:uppercase;letter-spacing:.06em}\n.muted{color:var(--muted);margin:.2em 0 14px;font-size:13.5px}\n\ntable.data{width:100%;border-collapse:collapse;margin-top:6px;font-size:13.5px}\ntable.data th{ text-align:right;color:var(--muted);font-weight:600;font-size:12px;\n text-transform:uppercase;letter-spacing:.04em;padding:6px 10px;border-bottom:1px solid var(--line)}\ntable.data th:first-child{text-align:left}\ntable.data td{text-align:right;padding:8px 10px;border-bottom:1px solid rgba(38,49,79,.5)}\ntable.data td.name{text-align:left;font-weight:600;color:var(--ink)}\ntable.data tr:hover td{background:rgba(79,140,255,.06)}\n\n.lat{font-weight:700;padding:2px 8px;border-radius:7px;font-size:12.5px}\n.lat-excellent{background:rgba(52,211,153,.16);color:#6ee7b7}\n.lat-good{background:rgba(56,189,248,.16);color:#7dd3fc}\n.lat-fair{background:rgba(251,191,36,.16);color:#fcd34d}\n.lat-poor{background:rgba(251,113,133,.16);color:#fda4af}\n.lat-na{background:#26314f;color:var(--muted)}\n\n.chips{display:flex;flex-wrap:wrap;gap:10px;margin-top:6px}\n.chip{display:inline-flex;align-items:center;gap:7px;padding:7px 13px;border-radius:999px;\n font-size:13px;font-weight:600;border:1px solid var(--line);background:var(--panel2)}\n.chip b{font-size:14px}\n.chip-ok{border-color:rgba(52,211,153,.45);color:#6ee7b7;background:rgba(52,211,153,.10)}\n.chip-warn{border-color:rgba(248,113,113,.45);color:#fca5a5;background:rgba(248,113,113,.10)}\n.chip-neutral{border-color:rgba(79,140,255,.4);color:#bcd4ff;background:rgba(79,140,255,.10);font-weight:600}\n.bar-type{fill:var(--accent2)}\ntd.mono,.mono{font-family:ui-monospace,Menlo,Consolas,monospace;font-size:12.5px;color:#bcd4ff}\n.muted .t{font-family:ui-monospace,Menlo,Consolas,monospace;color:#bcd4ff;font-size:12.5px}\n\n.verdict{margin:14px 0 4px;padding:11px 14px;border-radius:10px;background:var(--panel2);\n border:1px solid var(--line);font-size:14px}\n.verdict-ok{border-color:rgba(52,211,153,.4);background:rgba(52,211,153,.08)}\n.verdict-warn{border-color:rgba(248,113,113,.4);background:rgba(248,113,113,.08)}\n\ndetails.plan{margin-top:14px}\ndetails.plan summary{cursor:pointer;color:var(--accent2);font-size:13px;font-weight:600}\ndetails.plan pre{white-space:pre-wrap;word-break:break-word;background:#0a1224;border:1px solid var(--line);\n border-radius:10px;padding:14px;margin-top:10px;font-size:12px;color:#bcd}\n\n.chart-wrap{margin-top:14px}\nsvg.barchart{width:100%;height:auto;display:block}\n.bar-label{fill:var(--muted);font-size:13px}\n.bar-track{fill:rgba(38,49,79,.55)}\n.bar{fill:var(--accent2)}\n.bar-excellent,.bar-good{fill:var(--ok)}\n.bar-fair{fill:var(--fair)}\n.bar-poor{fill:var(--poor)}\n.bar-value{fill:var(--ink);font-size:12.5px;font-weight:600}\n.bar-value-in{fill:#0b1020}\n.bar-na{fill:var(--muted);font-style:italic}\n\n.dual{display:grid;grid-template-columns:1fr 1fr;gap:22px}\n@media(max-width:760px){.dual{grid-template-columns:1fr}}\n\n.errors{margin-top:16px;border-top:1px solid var(--line);padding-top:12px}\n.errors h4{color:var(--warn)}\n.errors ul{list-style:none;padding:0;margin:0}\n.errors li{display:flex;justify-content:space-between;gap:16px;padding:6px 0;\n border-bottom:1px solid rgba(38,49,79,.5);font-size:13px}\n.errors .name{color:var(--ink);font-weight:600}\n.err-msg{color:var(--muted);text-align:right;font-family:ui-monospace,Menlo,monospace;font-size:12px}\n\n.notice{padding:11px 14px;border-radius:10px;background:var(--panel2);border:1px solid var(--line);font-size:13.5px;color:var(--muted)}\n.notice-warn{border-color:rgba(248,113,113,.4);background:rgba(248,113,113,.08);color:#fca5a5}\n\ndetails.explain{margin-top:16px;border-top:1px dashed var(--line);padding-top:12px}\ndetails.explain>summary{cursor:pointer;color:var(--accent);font-size:13px;font-weight:600;\n list-style:none;display:inline-flex;align-items:center;gap:7px;user-select:none}\ndetails.explain>summary::-webkit-details-marker{display:none}\ndetails.explain>summary:before{content:"?";display:inline-flex;align-items:center;justify-content:center;\n width:18px;height:18px;border-radius:50%;background:rgba(255,138,61,.16);color:var(--accent);\n font-size:12px;font-weight:700}\ndetails.explain[open]>summary{margin-bottom:8px}\n.explain-body{font-size:13.5px;color:#c4d0ea;background:var(--panel2);border:1px solid var(--line);\n border-radius:10px;padding:14px 16px}\n.explain-body p{margin:.2em 0 .7em}\n.explain-body p:last-child{margin-bottom:0}\n.explain-body ul{margin:.3em 0 .5em;padding-left:20px}\n.explain-body li{margin:5px 0}\n.explain-body code{background:#0a1224;border:1px solid var(--line);border-radius:5px;\n padding:1px 6px;font-family:ui-monospace,Menlo,Consolas,monospace;font-size:12px;color:#bcd4ff}\n.explain-body .g{color:#6ee7b7;font-weight:600}\n.explain-body .b{color:#fca5a5;font-weight:600}\n.explain-body .y{color:#fcd34d;font-weight:600}\n.explain-body .r{color:#fda4af;font-weight:600}\n\nfooter{color:var(--muted);font-size:12.5px;text-align:center;margin-top:34px;line-height:1.7}\n.legend{display:flex;gap:14px;flex-wrap:wrap;justify-content:center;margin-top:8px}\n.legend span{display:inline-flex;align-items:center;gap:6px}\n.dot{width:10px;height:10px;border-radius:3px;display:inline-block}\n'
Map a p50 latency to a quality bucket used for the colored chip.
Render a horizontal bar chart as inline SVG.
items: list of (label, value, css_class) tuples. value may be None.
Headline numbers pulled from the most representative tests.
Dataset shape: feature count, type distribution, per-feature types.
Parse + optimise time of the view (EXPLAIN), and the stop decision.
The structural proof: single-AMP / PI access diagnostics.
Generic p50/p95/p99 table. rows: list of (label, dict).
One-line note explaining the entity-aware in_list batch cap, or ''.
How the batch phases query the dataset, and on which storage layout, or ''.
The HTML report of a benchmark (run_benchmark's result): a self-contained page with
inline SVG charts and CSS, no external JavaScript. Every section tolerates a phase that
is missing or failed. Use write_benchmark_report to save it to a file.
Render report to HTML and write it to disk; return the file path.
When out_path is None, writes under {tdfs4ds.BENCHMARK_REPORT_DIR}/ with
a timestamped, dataset-derived filename (directory created on demand).