Test suites · review
TruVelocity™ for AI TEVV test suites
Which questions of the pilot catalogs belong to which test suite, and what each suite costs to run. Filter, search, and click a suite on any question to change it: the counts and costs update as you go, and Changes lists your edits to copy or download. Nothing leaves this page.
18 Nov 2026 · TruVelocity™ 2026.11.18.2 · data 7df289138899
What each suite costs
One pass: the original wording, one repeat (phrasings and repeats multiply it). Turns sent count a conversation's earlier turns, sent as context, so they show the real run time. Parallel workers split by whole conversation: the longest one is the least a run can take. Click a suite to filter by it.
The suites
How a suite runs
A conversation runs up to its last selected question. Its earlier turns are sent as context, because later
questions depend on them (“pay the rest of it”, ids created earlier), but they are not judged and not
counted toward the next question's evidence. Conversations with no selected question are left out; task and
workflow scenarios are selected whole. A catalog may add its own suites (for example accessibility) in a
top-level suites: list; any other name is a validation error.
Running one
tv validate pilots/chat/catalog-v2.yaml lists questions and turns per suite
tv run pilots/chat/catalog-v2.yaml --env ... --suite smoke only the smoke questions
tv run ... --suite negative --suite boundary several suites: their union
In the console (tv console --allow-runs), each preset has a suites box. Tags
live in the catalogs as suites: [...] (in the question-catalog sources under docs/,
test_suites: [...]); pack scenarios are tagged in the pack files.
TruVelocity™ for AI TEVV by Innominds · © Innominds. All rights reserved.