tdfs4ds.benchmark.runner¶
tdfs4ds.benchmark.runner¶
End-to-end latency / throughput benchmark for a tdfs4ds feature dataset view.
Uses raw teradatasql (NOT teradataml) on purpose: teradataml routes every
statement through a single shared SQLAlchemy connection pool, which (a) corrupts
under the concurrent-QPS thread fan-out ("N is not a valid connection pool
handle") and (b) adds per-call wrapping latency that pollutes a latency
benchmark. Connections are opened from the same TDFS4DS_TD_* environment the
serve layer uses (tdfs4ds.utils.reconnect._connect_kwargs).
WHAT THIS MEASURES¶
CLIENT-PERCEIVED latency — wall-clock around cursor.execute() — which
includes driver overhead, network round-trip, server parse/plan/AMP execution
and result serialization. The optional DBQL pass isolates pure server time.
Two entry points
- programmatic — :func:
run_benchmark(returns the report dict), and - CLI —
python -m tdfs4ds.benchmark.runner --host … --dataset … --entity-cols ….
module-attribute
¶
_PHASE_WEIGHTS = {'parse': 1, 'explain': 1, 'warm': 2, 'cold': 3, 'batch': 3, 'large_batch': 6, 'batch_strategies': 6, 'concurrent': 5, 'time_travel': 2, 'server_timing': 2}
module-attribute
¶
_TYPE_NAMES = {'I': 'INTEGER', 'I1': 'BYTEINT', 'I2': 'SMALLINT', 'I8': 'BIGINT', 'D': 'DECIMAL', 'N': 'NUMBER', 'F': 'FLOAT', 'CF': 'CHAR', 'CV': 'VARCHAR', 'CO': 'CLOB', 'DA': 'DATE', 'AT': 'TIME', 'TS': 'TIMESTAMP', 'TZ': 'TIME WITH TZ', 'SZ': 'TIMESTAMP WITH TZ', 'BF': 'BYTE', 'BV': 'VARBYTE', 'BO': 'BLOB', 'YR': 'INTERVAL YEAR', 'MO': 'INTERVAL MONTH', 'DY': 'INTERVAL DAY', 'HR': 'INTERVAL HOUR', 'MI': 'INTERVAL MINUTE', 'SC': 'INTERVAL SECOND', 'PD': 'PERIOD(DATE)', 'PT': 'PERIOD(TIME)', 'PS': 'PERIOD(TIMESTAMP)', 'JN': 'JSON', 'XM': 'XML', 'UT': 'UDT'}
module-attribute
¶
Build a SQL WHERE for ONE composite key (AND of equalities).
Teradata-valid batch predicate for a list of (composite) key tuples.
Single-column key -> col IN (…); composite key ->
(a=1 AND b=2) OR (a=3 AND b=4) OR ….
Render one key value as a SQL literal (quote+escape strings, NULL keyword).
Dates and timestamps get typed literals. They used to fall through to
str(v), so a date key rendered as col = 2016-01-01 - integer
arithmetic, which never equals a date: every lookup on a date-keyed dataset
silently returned no row (the benchmark then timed empty lookups).
True when the IN/OR predicate text stays safely under the parser limit.
UTF-8 byte length of the IN/OR predicate for key_tuples.
Resolve raw-connection kwargs from the TDFS4DS_TD_* environment.
Reuses the same resolver the serve bootstrap / reconnect use, so the benchmark works wherever the package is already configured.
Run a statement on a raw teradatasql cursor; return the cursor.
Render a Teradata type code into a SQL type. Falls back to the raw code (never an opaque '?') when unmapped.
Resolve a view/table's columns via HELP COLUMN
Probe the dataset's resolved columns → feature count + per-column types.
Resolution order: HELP COLUMN first (authoritative for VIEWS), DBC.ColumnsV fallback. 'features' are the non-entity columns; entity columns are the grain keys. Returns a metadata dict or {error:…}.
Parse + optimise cost of the dataset view, measured with EXPLAIN.
EXPLAIN SELECT * FROM <dataset> parses the view, expands it and builds
the plan, but executes nothing — so it isolates the cost a wide multi-join
dataset pays on every request whose plan is not cached. Runs before the key
sampling, which already executes the full view. Also reports the size of
the view's DDL and of the plan: a long DDL is what the grouped strategy
shortens.
The batch statement retrieve_batch would run (one SQL authority), or
None when it would exceed Teradata's ~1 MB request parser limit.
What the batch phases run: query shape and, per storage table, the primary index and the partitioning entity columns the keys are matched on.
test_batch_strategies(dataset, entity_cols, all_keys, sizes, n_runs=5, max_in_list_keys=None, pushdown='auto')
Compare the two batch-retrieval strategies side-by-side, per size.
For each size, time BOTH retrieve_batch(method='in_list') and
retrieve_batch(method='volatile_join') over the same key sample, reusing
the raw benchmark connection (_MAIN_CONN) so no teradataml pooling
latency enters the measurement. An in_list predicate that would exceed
the parser limit is recorded as {"error":"skipped_parser_limit"} rather
than crashing — exactly the case the volatile join is meant to cover.
max_in_list_keys caps only the in_list arm: above it the in_list arm
is recorded as {"error":"skipped_in_list_cap","cap":…} (a wide composite
predicate's parse time would dominate the measurement) while the
volatile_join arm still runs at the full size.
A statement-level VALIDTIME AS OF prefix is ignored when the view bakes in CURRENT VALIDTIME, so we rewrite the view body into a temp view and query it.
Server execution time of the benchmark's OWN single-key lookups (scoped to the same query shape so client−server is a real overhead, not a mix).
Derive the dataset's entity (grain) columns from the tdfs4ds catalog.
Requires an active teradataml context (the serve / programmatic case). Returns a list of column names, or None when it can't be resolved (the standalone CLI must then pass --entity-cols explicitly).
run_benchmark(dataset, entity_cols=None, *, sample_size=10000, skip=(), batch_sizes=(1, 5, 10, 50, 100, 500), large_sizes=(10000, 100000, 1000000), strategy_sizes=(500, 5000, 50000), concurrency=(1, 2, 4), time_travel_dates=(), max_in_list_keys=None, max_parse_ms=DEFAULT_MAX_PARSE_MS, batch_pushdown=True, conn_kwargs=None, progress=None)
Run the benchmark against dataset and return the report dict.
Parameters¶
dataset : str
Fully-qualified DB.dataset_view to benchmark.
entity_cols : list[str] | None
Grain/entity key columns. When None, auto-derived from the tdfs4ds
dataset catalog (needs an active teradataml context).
skip : set[str]
Any of {'parse','explain','warm','cold','batch','large_batch',
'batch_strategies','concurrent','time_travel','server_timing'} to skip.
max_parse_ms : float | None
The parse phase runs FIRST, before anything executes the view: it
times EXPLAIN SELECT * FROM <dataset> (parse + optimise, no
execution). When its median exceeds max_parse_ms the run stops
there — every request on such a view pays that cost before reading a
row, so the latency phases would mostly measure parsing and could take
very long. The report then explains why and what to try (the grouped
strategy). Default 5000 ms; None or 0 never stops.
strategy_sizes : tuple[int]
Batch sizes at which the batch_strategies phase compares the
in_list predicate vs the volatile_join retrieval side-by-side.
max_in_list_keys : int | None
Maximum number of keys put into an in_list predicate. When None
it is inferred from the entity-column count as
max(50, 500 // n_entity_cols) — wide composite keys produce large
per-key disjuncts whose parse time dominates the measurement, so the
cap keeps in_list batches cheap. It clamps batch_sizes and the
in_list arm of batch_strategies ONLY; the volatile_join arm
and large_batch keep their full sizes.
batch_pushdown : bool
True (default): the batch phases run what retrieve_batch runs -
the dataset's query regenerated with the keys inside every storage-table
read, so each FS_T_* table is read by primary index and only in the
partitions the keys fall in. False: the previous
SELECT * FROM <view> WHERE <keys> form, to measure the difference.
conn_kwargs : dict | None
Raw teradatasql kwargs; defaults to the TDFS4DS_TD_* environment.
progress : callable(str) | None
Receives one human-readable line per test phase (for a UI bar).
Returns the {"metadata":…, "results":…} report dict.