TruVelocity™ for AI TEVV: web UI demo runbook
How to try the tv ui web UI yourself, and how to demo it to a client's leadership team: for every screen, what to click, what to say, what the screen shows, and the business outcome it delivers.
Release 2026.11.19.1 and later · written 19 Nov 2026 · all demo data is fictitious (Harbor Mutual).
1What the UI is
TruVelocity™ tests AI agents from the outside and turns the results into a release decision backed by evidence. The web UI (tv ui) puts that work in six steps a team walks through for each agent, plus a Home page for the whole portfolio. It runs on your own machine, listens only on 127.0.0.1, and is protected by a token in the URL. The first page in each new browser session, and after Sign out, is a sign-in page; Explore as demo user (or sign in with any email) opens Home.
| Step | The question it answers | What leadership gets |
|---|---|---|
| Home | Which agents are ready to release, which aren't, and why? | One portfolio view: verdicts, open critical defects, risk by pillar, trend. |
| 1 Context | What is this agent for, what can it do, and what must never go wrong? | A shared, written statement of intended use and risk appetite that drives the testing. |
| 2 Catalog | What do we test, and what do we deliberately leave out? | Coverage made visible; gaps recorded with a reason. |
| 3 Configure | How does TruVelocity™ reach the agent, and does it answer now? | No surprises on test day; the versions under test are recorded. |
| 4 Conduct | Run the plan and watch it go. | Anyone can run the same governed tests, without the command line. |
| 5 Conclude | Does this run meet the release bar? What do we decide? | A Go / No-go verdict, a recorded decision with who, when and why, and an evidence pack for audit. |
| 6 Continuous | Is it still good after release? | Trend, drift between runs, alert rules and a scheduled run in CI. |
Every number in the UI comes from real engine runs kept in the project's runs folder: the same runs tv gate and tv report read on the command line. Nothing is typed in by hand. To check what an agent did to its data, TruVelocity™ reads the records the agent was meant to change (chosen database tables, files), read-only, before and after each task, and compares them row by row.
2Two ways to run the demo
| A. Offline demo project recommended | B. Live pilots (advanced) | |
|---|---|---|
| Best for | Client demos, trying it yourself, a laptop with no network | Showing real agents built on a language model, run live |
| Needs | Python 3.11 or later, pip install "truvelocity-tevv[ui]". No Docker, no LLM, no keys. | This repository and its .venv, Docker Desktop, the demo agents' repository (SystemsUnderTest), LLM settings in .env |
| Agents | Three agents (ids claims, chat, desk) with at least three release runs each | Harbor Mutual: Chat assistant, Claims agent, Member desk (in Docker) |
| Story it tells | Claims: Go. Chat: No-go with critical defects. Desk: change over time. | Whatever the latest live runs found (screenshots in this page) |
| A run from Conduct | About a minute | 1–2 minutes for a claims or desk suite; much longer for the full chat catalog |
| Set-up time | About 5 minutes, once | About 20 minutes the first time; 2 minutes to redeploy |
ASet up the offline demo project about 5 minutes, once
Run these in a Command Prompt on Windows (or a terminal on macOS or Linux). Each box holds one command; Copy copies just the command, not the grey comment above it.
Windows (Command Prompt)
cd /d %USERPROFILE%py -3.12 -m venv tv-envtv-env\Scripts\activatepip install "truvelocity-tevv[ui]"tv demo-project truvelocity-demoTruVelocity demo project in truvelocity-demo: three agents, their catalogs and four weeks of runs claims-20261024-091204 claims build 3.0.0 chat-20261025-100531 chat build 2.1.0 desk-20261026-144012 desk build 1.0.0 claims-20261031-092045 claims build 3.0.1 chat-20261101-113009 chat build 2.2.0 desk-20261103-150552 desk build 1.1.0 claims-20261107-090817 claims build 3.1.0 chat-20261110-104540 chat build 2.3.0 desk-20261111-161026 desk build 1.2.0 desk-20261116-132503 desk build 1.2.1 claims-20261117-091538 claims build 3.1.1 chat-20261118-174011 chat build 2.4.0 made 12 runs in 3 s; latest verdicts: Claims back-office agent Go 100% of cases passed every try; defects: 0 critical, 0 major, 0 minor Customer chat assistant No-go 67% of cases passed every try; defects: 1 critical, 1 major, 1 minor Member service desk Go 100% of cases passed every try; defects: 0 critical, 0 major, 0 minor next: cd truvelocity-demo tv ui --allow-runs --open (The run names carry dates counted back from today, so yours differ.)
cd truvelocity-demotv ui --allow-runs --openTruVelocity UI: http://127.0.0.1:8770/?t=<your token> (the token in the URL is your key; Ctrl+C stops it)
tv-env\Scripts\Activate.ps1; if PowerShell refuses to run it, use a Command Prompt instead. Already have Python 3.11? Use py -3.11 in step 2.macOS or Linux (terminal)
cd ~python3 -m venv tv-env. tv-env/bin/activatepip install "truvelocity-tevv[ui]"tv demo-project truvelocity-democd truvelocity-demotv ui --allow-runs --openNext time
Only these steps are needed:
cd /d %USERPROFILE%tv-env\Scripts\activatecd truvelocity-demotv ui --allow-runs --openWhat is in the demo project
| Agent id | Name shown in the UI | Latest release run | Use it to show |
|---|---|---|---|
claims | Claims back-office agent | Go · 100% of cases passed every try, no defects | A release that meets the bar; the evidence pack |
chat | Customer chat assistant | No-go · 1 critical, 1 major, 1 minor; 67% of cases passed every try | The gate stopping a risky release; what a critical defect looks like |
desk | Member service desk | Go · changes over its runs | Continuous: trend, drift and alerts |
Each agent has at least three release runs, so Trend and Drift have something to compare. A run you start from Conduct finishes in about a minute, with no Docker and no LLM.
BLive pilots (advanced) real agents in Docker, with an LLM
The Harbor Mutual demo agents (a customer chat assistant, an event-driven claims agent, and a multi-agent member desk) run as Docker containers built from the SystemsUnderTest repository and call a real language model. Use this when the audience wants to see live agents; use the offline project for everything else.
Prerequisites
- This repository cloned, with its
.venv(the ops script'ssetupcreates it). - Docker Desktop running.
- A clone of the SystemsUnderTest repository next to this one (or
TV_SUT_DIRpointing to it). - LLM settings in the root
.env(copy.env.example;DEFAULT_LLM_PROVIDERand that provider's keys). The ops script never prints or changes them.
Step-by-step installation: TruVelocity™ Local Install Guide.
Start the agents and the UI (Windows)
cd C:\myprojects\TruVelocityForAITEVVscripts\truvelocity-tevv-ops.bat deploy allscripts\truvelocity-tevv-ops.bat statusEvery agent and observer line says up.
scripts\dev_deploy.batOn macOS or Linux use sh scripts/truvelocity-tevv-ops.sh deploy all, … status and sh scripts/dev_deploy.sh. The deploy script creates .venv if needed, installs TruVelocity™ with the UI extra, and runs tv ui --listen 127.0.0.1:8770 --allow-runs --open.
Optional: fresh runs before the demo
The UI shows whatever runs are in tv_runs. To have a release run from today, run a pilot from the command line beforehand (one per agent; each takes a few minutes and calls the LLM):
scripts\truvelocity-tevv-ops.bat pilot-claimsscripts\truvelocity-tevv-ops.bat pilot-deskpilot-chat just before a demo: the full chat catalog asks every question 13 ways (416 conversations in the latest run) and takes a long time.✓Pre-flight checklist 10 minutes before the audience arrives
- Set
TV_UI_USER(oruser:intv-ui.yaml) to the presenter's full name and restarttv ui: the Home greeting and every decision you record carry this name (without it, the computer's login name). What is typed on the sign-in page is not stored. - Click Explore as demo user: Home shows three agents. Reload the page once.
- The Agent picker (top right) lists all three agents.
- Open Conduct for one agent: it shows a Start a run form, not "Runs are off in this UI". If it says runs are off, restart with
--allow-runs. - Live pilots only:
scripts\truvelocity-tevv-ops.bat statussays every agent is up. - Click Report on one Home row: the execution report opens in a new tab. Close it.
- Browser zoom 100–110%, other tabs closed, notifications off. If you share one window, share the browser, not the console (the console shows the token).
- Decide your story in advance: which agent is the Go, which is the No-go, which shows change over time (see the guided path).
- Last: click Sign out (the arrow icon top right, next to the bell), so the audience starts on the welcome page.
3The guided path and timing 15–20 minutes
The path follows one question leadership cares about: can we release this agent, and can we prove it? Context and Configure get a short mention; spend the time on Catalog, Conduct, Conclude and Continuous.
| Time | Screen | What you show | The point you land |
|---|---|---|---|
| 1 min | Welcome page | The opening slide: TEVV, the eight agent types, local and private. Then Explore as demo user. | Agents act, not just talk; one assurance flow covers every kind of agent. |
| 3 min | Home | Three tiles, Agent portfolio (Verdicts, then Risk by pillar), Needs attention, Recent runs. | One view of every agent's release readiness, from real runs. |
| 1 min | Context (optional) | The agent summary and "What this means for testing". | Testing starts from what the business says the agent is for. |
| 3 min | Catalog | Questions × wordings, suites with times, library packs, Your test plan, gaps. | Coverage is visible, and every gap is a recorded decision. |
| 1 min | Configure (optional) | At a glance. | We know how the agent is reached and which model was tested. |
| 3 min | Conduct | Start a run of one suite; watch it progress; Recent runs. | Repeatable, governed runs: anyone can press the button. |
| 5 min | Conclude | Claims: release bar, verdict, decision, evidence pack. Then chat: the No-go and its critical defects. | A Go / No-go backed by evidence, with an audit trail. |
| 3 min | Continuous | Desk: Trend, Drift, Alerts, Run on a schedule. | Quality is watched after release, not checked once. |
| 1 min | Close | Back to Home. | Evidence before go-live, and every release after. |
Run started in Conduct: click Start from: Smoke and Start run as you reach Conduct. On the offline project it finishes in seconds, so you can open its execution report straight away; on the live pilots let it run while you talk through Conclude and come back to it at the end.
4The 5-minute version
| Time | Click | Say |
|---|---|---|
| 1 min | Welcome page (15 seconds), Explore as demo user, then Home: point at Release-ready, Open critical defects and the Agent portfolio. | "One workspace for every kind of AI agent, private to this laptop. Three agents under test, from real runs. Two are release-ready, one isn't, and here is why." |
| 2 min | Pick the chat agent, open 5 Conclude: Verdict, then the Defects table. | "No-go. Not a score: specific defects. This one asked the customer to approve an action they never requested. That is critical, so the release stops." |
| 1 min | Pick the claims agent, still on Conclude: Verdict, Release decision, Download evidence pack. | "This one meets the bar. The decision is recorded with who, when and why, and the evidence pack is what we hand the auditor." |
| 1 min | Pick the desk agent, open 6 Continuous: Trend and Drift. | "After release we keep watching: each run is compared with the one before, question by question, so a regression shows up the day it happens." |
5Words the UI uses use them exactly
| Term | What it means |
|---|---|
| Case and try | A case is one test question with its checks. Each case is asked several times: in several wordings (13 for the chat pilot, 3 for claims) and sometimes repeated. Each of those is a try. |
| Cases passed every try | The release gate's measure: a case that fails any of its tries counts as failed. This is the honest number for an agent that must work every time. |
| Tries passed | Each try counted on its own. Always at least as high as cases passed every try (chat's latest live run: 87% of tries, 69% of cases passed every try). |
| Flaky | A case that passed some tries and failed others. It counts as a failure at the gate. |
| Release bar | The rules in truvelocity-tevv-gate.yaml: no critical defect, at most 2 major, at least 85% of cases passing every try, evidence seams clean, parity holding. |
| Verdict | Go when the run meets the bar ("Meets the bar"), No-go when it doesn't ("Below the bar"). Home's Release-ready tile counts the Go agents among those assessed. |
| Defect severity | By what went wrong. Critical: the agent did, or offered to do, something it shouldn't (asking a customer to approve an action they didn't request, changing data that should not change, the wrong customer's data). Major: it didn't do what was asked, or said something its actions don't support. Minor: the rest. |
| Evidence seams | Automatic checks where what the agent says it did is compared with the independent records (for example "done" when nothing was saved). SE-01 to SE-04 must be clean; SE-06 is parity. Full list in the leadership demo runbook. |
| Parity | Two deployments of the same agent (another model, host or framework) behave the same, within normal run-to-run variation. |
| Release run | A run of the whole catalog in a release mode. Only these drive the verdict, risk and trend; a run of one suite is a partial run and doesn't. |
| Baseline | An accepted earlier run. When chosen in Conclude, failures it already had count as known, not new. |
| Drift | How the latest release run differs from the one before it of the same shape, question by question: regressions, fixes, and how often the outcome, tool calls and data changes were the same. |
WWelcome page 1 minute · the opening slide
Click
- Open the link
tv uiprinted (or sign out first, so the page shows). Let the audience read it for a moment. - Point at the title, then the hub on the right, clockwise from Single-turn bot to Self-learning.
- Point at the sign-in card: Sign in, Continue with SSO and Explore as demo user.
- Click Explore as demo user (Sign in with any email, or Continue with SSO, does the same). A deep link (a URL with
#…) opens its own page instead of Home after this click.
What is on screen
- Sign in to your workspace: assurance for every kind of AI agent, from a single-turn bot to a self-learning swarm; below the card, what TEVV stands for.
- Sign in to your workspace: work email, password, Sign in, Continue with SSO and Explore as demo user. In this release any of them opens the workspace; nothing typed is stored or sent (the card says so). Decisions are recorded under
TV_UI_USERoruser:intv-ui.yaml, else the computer's login name. - Access: the page runs on this computer only and opens only through the key in the link
tv uiprinted. - The hub: TruVelocity at the centre, with eight agent types around it, from simple to autonomous: single-turn bot, conversational, grounded / RAG, tool-using, loop engineering, graph engineering, multi-agent, self-learning.
- Shown first in every new browser session and after Sign out.
Business value
- Sets the frame in one picture: one approach for every agent the client has or plans.
- Answers the data-privacy question before it is asked: local, key-protected, no cloud.
- Accountability from the first click: decisions are recorded under a named person.
Watch for
- Set
TV_UI_USERto the presenter's full name beforehand, so decisions are recorded under it rather than a login name such asjdoe42. - Don't claim TruVelocity has tested a client agent of every type shown: the hub shows the types it is built to assure.
- There are no user accounts or passwords: privacy comes from the local address and the key in the link.
HHome 3 minutes · the portfolio view
Click
- After Explore as demo user, Home opens with Agent: All agents (top right).
- Point along the three tiles, left to right.
- Point at one row of the Agent portfolio; hover the trend line.
- Click Risk by pillar in the portfolio's header to switch views; click Verdicts to switch back.
- Point at Needs attention, then Recent runs.
- Optional: click Report on the Go agent's row to show the execution report in a new tab.
What is on screen
- A greeting (with the presenter's name when
TV_UI_USERoruser:is set) and a one-line summary. - Release-ready: Go agents out of those assessed (2 of 3 in the demo project). Open critical defects, and across how many agents (1, across 1 agent). Decisions waiting: release decisions and alerts to look at (2).
- Agent portfolio, Verdicts view: one row per agent: its kind (conversational, task, multi-agent), a Status (for example "Awaiting release decision"), verdict, defects by severity, last run, and the trend of cases passed every try over its release runs, with Report.
- Agent portfolio, Risk by pillar view: answer quality, agentic action, security & performance, governance & economics. Each cell shows types measured out of types in the pillar. Not assessed is never shown green.
- Needs attention (and the bell): each failed release gate with the rule that failed, and alerts such as "Cases passed every try dropped".
- Recent runs: the latest 5, with Tries passed next to Cases passed every try; All runs opens every run in the console.
Business value
- A release-readiness view across all AI agents in one place, for the people who sign off: three numbers, then the detail.
- Risk shown by business area, including what has not been tested yet.
- Drill from the portfolio to the exact defect in two clicks.
Watch for
- Tries passed is always at least as high as Cases passed every try. If asked, explain the difference (see Words); don't call either "accuracy".
- Run presets are on Conduct now, and the standards note is on Conclude's evidence pack card: don't look for them on Home.
- Recent runs shows only the latest 5; use All runs if someone asks about an older run.
Demo project: Home shows 2 of 3 release-ready (Claims back-office agent and Member service desk are Go; Customer chat assistant is No-go), 1 open critical defect across 1 agent and 2 decisions waiting; Needs attention lists "Release gate failed" and "Cases passed every try dropped" for the chat assistant.
1Context 1 minute in a demo · optional
Click
- Choose the agent in the Agent picker, then the 1 Context tab.
- Point at the topic chips (Intended use, Goals, Domain …) and the % captured.
- Scroll the summary to What this means for testing.
What is on screen
- A Context assistant that walks through 15 topics (Essential or Full) and files your answers.
- The Agent summary: purpose, users, actions with money or legal effect, systems it changes, oversight, guardrails, worst failure to prevent. Click any text to edit; Import workbook or YAML brings in an existing questionnaire.
- What this means for testing: the pillars and test families the answers call for (for example "High: money, data or legal exposure → stricter release bar").
- The "Built for this agent" chips at the top of every later step come from here.
Business value
- One written statement of intended use and risk appetite that product, risk and engineering agree on; the basis for TEVV and for later audit.
2Catalog 3 minutes · what we test
Click
- Pick the claims agent and the 2 Catalog tab.
- Point at the three tiles: questions, wordings, test types in plan.
- Point at Suites and the time columns.
- Point at Library packs (✓ included, + available).
- Scroll to Your test plan; optionally click Open … test plan document.
What is on screen
- The test plan read from the agent's
catalog.yaml, with states from its latest full release run. - Questions × wordings: each question is asked several ways, because agents built on language models answer differently to different phrasing.
- Suites (sanity, smoke, regression, positive, negative, edge, boundary, security) with run times measured on the latest run.
- Library packs: ready-made tests (prompt injection, privacy, payments and approvals, unsafe writes …). Adding one shows what it adds before anything is written.
- Your test plan: test types in plan out of the full catalog, how each is measured (Gate, Track, Info), the release bar, the four disciplines (Test, Evaluation, Verification, Validation) and recommended types left out.
Business value
- Coverage is visible: leadership sees what is tested and what is not.
- Gaps are deliberate, recorded decisions, not oversights.
- Reuse: library packs bring tested know-how to every new agent, so a new agent starts from more than a blank page.
- Time estimates make test effort plannable.
Watch for
- Don't tick types in or out, add packs or confirm the plan in a demo: those change the agent's saved plan.
- Don't walk through all 142 test types. Name one or two categories and move on.
- "Not built yet" types exist in the catalog but the engine doesn't run them today. Be plain about it.
3Configure 1 minute in a demo · optional
Click
- 3 Configure. Read out At a glance.
- Optional: open Technical details for a technical audience.
- Click Check connection now only if you confirmed in pre-flight that the agent is up.
What is on screen
- Does the agent answer? The last connection check.
- How a run starts: the run preset from
tv-console.yaml(for example "Pre-release gate, every wording"). - Model under test, as recorded on the latest release run.
- Technical details: settings, run preset and the versions the latest run used.
Business value
- Black-box testing of in-house and vendor agents alike; reproducible results tied to a version.
4Conduct 3 minutes · run the tests
Click
- 4 Conduct for the agent you are showing.
- In Start a run, click Start from: Smoke. It fills the form: the smoke suite, one wording.
- Read out the time estimate under the form. Leave the rest as it is.
- Click Start run. A card appears under Started here.
- Move on to Conclude while it runs; come back when the toast says it finished, and click Execution report.
What is on screen
- Start from: three ready-made starting points that fill the form. Smoke: the smoke suite, one wording, a few minutes at most. Pre-release gate: the whole catalog, every wording. Reliability run: the whole catalog, 3 repeats.
- The form works within the agent's preset (from
tv-console.yaml): suites, wordings, repeats, workers, mode, and an estimate from times measured on earlier runs. - The job card: run id, Running / Finished / Failed / Cancelled, elapsed time, a progress bar, "n of m episodes · passed · failed · flaky", a Cancel button and the engine's output.
- A run that couldn't start (for example another run already uses the runs folder) shows Not started.
- Recent runs of this agent: date, mode (Full catalog, Pre-release gate, Selected scenarios), cases passed every try with tries underneath, defects and the gate result.
Business value
- Testing a release takes minutes, not a manual test cycle: faster, repeatable releases.
- The same governed plan every time; nobody improvises the tests.
- Every run is kept with its inputs, outputs and evidence: nothing to reconstruct later.
Watch for
- Use Smoke in a demo. A suite run is a partial run, so it doesn't replace the release run your verdict and trend come from. Pre-release gate and Reliability run run the whole catalog: the result becomes the new latest release run and can change the numbers you rehearsed.
- Offline project: Smoke finishes in seconds, a whole-catalog run in about a minute. Live pilots: a claims or desk suite takes 1–2 minutes; never start the full chat catalog in a demo.
- "Runs are off in this UI" means the UI was started without
--allow-runs.
5Conclude 5 minutes · the release decision
Tell this in two parts: first the agent that meets the bar (claims), then the one that doesn't (chat).
Part 1: an agent that meets the bar (claims)
Click
- Pick the claims agent, then 5 Conclude.
- Run under review already shows the latest full release run. Leave Baseline on "No baseline: every failure counts".
- Read the Release bar, then the Verdict table, then Defects.
- Point at Release decision (Go / Go with conditions / No-go, and Why).
- Point at Download evidence pack. On your own demo project you can click it and open the zip.
What is on screen
- The run under review: how many cases passed, failed or were flaky, and a link to its execution report.
- The release bar from
truvelocity-tevv-gate.yaml(kept in version control). - The verdict: the engine's release gate, rule by rule: defects, pass rate, seams, each met or not met, with why.
- Defects: severity, case, what went wrong in plain words, and how often (every try, or 2 of 13 tries).
- Release decision: Go, Go with conditions, or No-go, with a reason, kept with who and when.
- Evidence pack: one zip with the run folder (what ran, every conversation, defects, scorecard, reports), the decision and the verdict as judged. The card also says that mapping this evidence to OWASP Top 10 for LLM apps, NIST AI RMF, ISO/IEC 42001 and the EU AI Act comes in a later release.
Demo project: the claims run under review is the latest claims run (build 3.1.1, named claims-<date>-091538): 100% of cases passed every try and no defects.
Part 2: an agent that doesn't (chat)
Click
- Switch the Agent picker to chat; stay on Conclude.
- Read the Verdict: each rule not met, and why.
- In Defects, point at one critical row: "Asked to approve an action this step should not take" (BL-01: the customer only asked what they owe, and the agent asked them to approve paying it).
- Point at the How often column: "every try", "9 of 13 tries".
- Point at the recorded No-go decision, if one is shown.
What is on screen
- The same bar, applied to a different agent: Below the bar.
- Critical defects such as: asking the customer to approve a payment or a policy change they never requested; data that changed where no change was expected; an action that was needed but never succeeded.
- How often: a defect on 1 try in 13 is still a defect. That is why the gate counts cases passed every try.
Business value
- A release decision made with evidence, against a bar agreed in advance, not on a demo that happened to go well.
- Risk reduced before customers see it: critical defects stop the release automatically.
- An audit trail: who decided, when, why, on which run, with the evidence pack to prove it.
- Engineers get specifics (case, what went wrong, how often), so fixes are faster.
Watch for
- Don't click Record decision or Mark Conclude complete on a shared project: decisions are kept in the project (
.tv-ui). On your own demo project it is fine, and it is a good moment. - A Go on a run below the bar needs a written justification. Show it only if asked about overrides.
- Multi-approver sign-off and the standards mapping (OWASP, NIST AI RMF, ISO/IEC 42001, EU AI Act) come in a later release; the evidence pack card says so.
- A recorded decision carries the presenter's name (
TV_UI_USERoruser:, else the login name).
Demo project: chat shows 1 critical (BL-01), 1 major (an address change it says it made but didn't), 1 minor (it then quotes the old address); 67% of cases passed every try, down from 100% on build 2.3.0.
6Continuous 3 minutes · after release
Click
- Pick the desk agent and the 6 Continuous tab.
- Trend: read the line and the table from oldest to newest.
- Drift: regressions, fixes, still failing, and the "same …" percentages.
- Alerts: which rules are quiet and which would fire.
- Run on a schedule: show the GitHub Actions or Azure Pipelines tab and Show the file. Don't click Write … pipeline.
What is on screen
- Trend: each release run with cases passed every try, defects (critical / major / minor), bar met or not, the decision, and what changed since the run before (model, prompts or tools, by fingerprint).
- Drift: the latest release run against the one before of the same shape, question by question.
- Alerts: rules checked against the latest run: gate fails, a new critical defect, a case that passed before now fails, the model / prompts / tools changed, pass rate drops more than n points.
- Run on a schedule: a ready pipeline file for CI, or this machine's scheduler, running the same
tv runcommand. - A banner may say the step isn't ready to finish: monitoring starts from a concluded baseline.
Business value
- Regressions are caught the day they happen, not by a customer.
- Model, prompt and tool changes are tied to their effect on quality.
- Quality gates run in the existing CI; no new platform to operate.
Watch for
- Language-model agents disagree with themselves a little from run to run. A few points of "same outcome" below 100% with no regression is normal; say so before someone asks.
- Sending alerts by email or chat comes in a later release; today the rules are checked when the page opens and in the pipeline.
- Don't click Save monitoring, Write … pipeline or Mark Continuous complete on a shared project.
Demo project: the desk agent's release runs go the Member service desk's four release runs go 50%, 75%, 88%, 100% of cases passed every try (builds 1.0.0 to 1.2.1): 1.0.0 let a billing outage cascade and passed too many permissions on (two critical defects), fixed by 1.2.0; Drift shows DSK-CASE-01 fixed.
?Leadership questions crisp answers
These are the questions the UI itself raises. For the broader ones (why we need it, what makes it unique, the seven evidence seams, black-box vs white-box, architecture), use the leadership demo runbook.
Where do these numbers come from? Can someone edit them?
tv gate and tv report use, and the same ones CI produces. The UI reads them; it has no field to type a pass rate into. Decisions and plan changes you make in the UI are stored separately, with who and when.Why is "Cases passed every try" lower than "Tries passed"?
Who sets the release bar? Can it be different for us?
truvelocity-tevv-gate.yaml, kept in version control and agreed with the business. You can change the limits, and add stricter rules for one capability (for example 100% for settlements). The UI shows which file is in force on every verdict.Can a manager override a No-go?
TV_UI_USER, else the login name) and when and goes into the evidence pack. The verdict itself never changes; the override sits next to it.What makes a defect critical?
Is an AI grading the AI?
Does it touch our production systems or data?
Do we have to change our agent?
How long does a test cycle take?
Can this run in our pipeline?
Is there a login? What does the sign-in page check?
TV_UI_USER or user: in tv-ui.yaml, otherwise the computer's login name.Where is the data kept? Is it in the cloud?
.tv-ui folder. The UI listens only on this machine (127.0.0.1) and needs the token in its URL.Does it map to OWASP, NIST AI RMF, ISO/IEC 42001 or the EU AI Act?
Why does the same agent give different results on different runs?
!Don't do this in a demo
| Don't | Instead |
|---|---|
| Start a whole-catalog run (Pre-release gate or Reliability run) just before you show Conclude | Use Start from: Smoke. A partial run doesn't change the latest release run, so the verdict and trend stay as rehearsed. |
| Start the full chat catalog on the live pilots | It runs hundreds of conversations against the LLM. Use the offline project, or a claims or desk suite. |
| Click Record decision, Save, Save context, Save monitoring, Write … pipeline or Mark … complete on a shared project | These write to the project. Use your own demo project, where they are part of the story. |
| Click Check connection on the live pilots without checking Docker first | Run status in pre-flight; skip the check if in doubt. |
Paste deep links (URLs with #…) to jump to a step | Use the Agent picker and the step tabs. (A deep link does open its page after you sign in, but the tabs are more reliable on stage.) |
| Open on the welcome page with a bare login name, or skip the welcome page | Set TV_UI_USER to the presenter's name, and Sign out before the audience arrives: the sign-in page is your opening slide. |
| Share the console window | The console shows the URL token. Share the browser only. |
Start the UI with --allow-remote on a client network | Keep the default: it listens on 127.0.0.1 only. |
| Call the pass rate "accuracy", or quote Tries passed as the release number | Use the UI's words: Cases passed every try is the release gate's measure. |
| Describe the data check as copying or capturing the client's systems | Say it reads the records the agent was meant to change, read-only, before and after each task, and compares them row by row. |
| Promise standards mapping, multi-approver sign-off or alert e-mails today | They are marked in the UI as coming in a later release. Say so. |
| Tour all 142 test types or every Context topic | One example each; keep to the guided path. |
| Apologise for "not met" on a live agent | That is the product working: the gate found something before customers did. |
⚙Troubleshooting
| What you see | What to do |
|---|---|
'tv' is not recognized / command not found | The environment isn't active: run the activate step again. |
error: no tv-ui.yaml | You aren't in the project folder: cd truvelocity-demo. |
| The port is already in use | Pick another: tv ui --listen 127.0.0.1:8771 --allow-runs --open. |
| The browser didn't open, or says unauthorised | Copy the whole URL the command printed, including ?t=…. |
| Conduct says "Runs are off in this UI" | Stop the UI (Ctrl+C) and start it again with --allow-runs. |
| A started run shows Not started | The run was refused, usually because another run is using the same runs folder. Wait for it to finish, then start again. |
| Live pilots: a run fails at once, or Check connection fails | scripts\truvelocity-tevv-ops.bat status; redeploy with deploy all; check the LLM settings in .env. |
| Numbers differ from this runbook | Expected: the screenshots are from the live pilots on 19 Nov 2026. The UI always shows the latest runs. |
↺After the demo
Stop the UI: press Ctrl+C in its window.
Offline demo project: start fresh
Decisions, plan changes and runs you made live in the project folder. For a clean project next time, delete the folder and create it again (from the folder above it):
rmdir /s /q truvelocity-demotv demo-project truvelocity-demoOr, without deleting: tv demo-project truvelocity-demo --force (from the folder above it) writes the project again and remakes its runs, clearing anything recorded in the UI.
Live pilots: restore the seed data
scripts\truvelocity-tevv-ops.bat reset chatscripts\truvelocity-tevv-ops.bat reset claimsRelated pages: Leadership demo runbook (pitch, Q&A, architecture) · Demo runbook (the command line) · Local install guide.
TruVelocity™ for AI TEVV by Innominds · © Innominds. All rights reserved.