tdfs4ds.feature_store.valid_time_column¶
Valid time read from a column of the source (0.3.5.0).
A process registered with upload_features(..., valid_time_column='EVENT_DATE') does not
stamp its values with one valid time per run. Each source row carries its own: the row's
value is valid from that column's value until the entity's next row (or 9999-01-01
for the last one). A whole history is written by one run.
Rules (the MERGE's, applied per entity):
- A row is a change point. A later absence means "unchanged", never "gone".
- Equal consecutive values of an entity and feature are merged (NULL-safe), so no period is opened for an unchanged value, as the ingestion MERGE does.
- "From t0 onward": t0 = the entity's earliest valid time in the load. The load replaces the entity's stored history from t0 onward and leaves what is before untouched. A re-run is idempotent, an incremental load rewrites only from its own t0, a backfill needs no order.
The write is three statements per storage table, sent as ONE request (one transaction: a failure leaves the table as it was), verified on Vantage 20:
a. NONSEQUENCED VALIDTIME DELETE the rows of the entity starting at or after t0;
b. NONSEQUENCED VALIDTIME UPDATE the row spanning t0 so that it ends at t0;
c. one NONSEQUENCED VALIDTIME INSERT per feature of the computed periods (one per
feature, not a UNION ALL: a UNION takes its column type from its first branch, and two
features of one table may have VARCHAR lengths that differ).
The option is the process setting VALID_TIME_COLUMN (FS_PROCESS_SETTINGS), read
fail-closed by run(): a history load stamped with the run time would be silently wrong.
module-attribute
¶
_TS_RE = compile('^\\s*TIMESTAMP\\s*(?:\\(\\s*(\\d+)\\s*\\))?(\\s+WITH\\s+TIME\\s+ZONE)?', IGNORECASE)
Build db.obj with each part quoted only as needed.
qualified('TDB', 'FS_X') == 'TDB.FS_X' (unchanged for ordinary names);
qualified('SVC@CORP', 'FS_X') == '"SVC@CORP".FS_X'. An empty/None
database yields the bare object name, so unqualified callers keep working.
force=True quotes both parts unconditionally — only for the sites that
have always emitted quotes and whose exact text is already persisted (dataset
view DDL, returned volatile-table names).
Escape value for use inside a single-quoted SQL string literal.
For the DBC probes and catalog predicates that compare a name as data
(WHERE DatabaseName = '…'). The caller supplies the surrounding quotes.
'date', 'timestamp0' (no fractional seconds), 'timestamp' or None
(not a usable valid-time type). Pure.
The staged TDFS_VALID_TIME value, a TIMESTAMP(0) WITH TIME ZONE. Pure.
A DATE is midnight of that day. A timestamp with fractional seconds goes through its
text (CAST of a TIMESTAMP(6) to TIMESTAMP(0) is Error 7454), which keeps
the seconds and drops the fraction. A value without a time zone is read in the session
time zone, as FEATURE_STORE_TIME is.
A database timestamp as the text a VALIDTIME PERIOD literal takes. Pure.
The volatile table of each staged entity with its t0 (earliest valid time). Pure.
a. The entity's stored rows starting at or after its t0. Pure.
b. The stored row in force at t0 ends at t0. Pure.
insert_sql(target: str, volatile_ref: str, entity_columns, feature_id, feature_name, feature_version, period: bool = False) -> str
c. The periods of one feature: unchanged values dropped, each period ending at the
next kept row of the entity, the last at 9999-01-01. Pure.
PREV_V is NULL both for "previous value NULL" and "no previous row": RN = 1
tells them apart.
period: the feature holds a PERIOD value. Teradata refuses PERIOD data in a query with
a window function (Error 6976), so the value travels as its bounds and is rebuilt
outside the window queries (:func:_period_insert_sql).
_period_insert_sql(target: str, volatile_ref: str, entity_columns, feature_id, feature_name, feature_version) -> str
:func:insert_sql for a PERIOD feature. Pure.
Same periods and the same change rule, computed on BEGIN / END of the value (a
PERIOD never has a NULL bound, so a NULL BEGIN means a NULL value). No query block
that holds a window function carries a PERIOD column; the value is rebuilt last.
A PERIOD type as TYPE() prints it (PERIOD(TIMESTAMP(0))). Pure.
The one request writing a storage table: a, b, then c per feature. Pure.
features: [(feature_id, feature_name, feature_version)] of that table, with an
optional fourth item True for a PERIOD feature.
Write the staged batch, each row at its own valid time. Raises on a database error (the caller records the batch FAILED; each table's request is all or nothing).
Returns {"t_min", "t_max", "rows", "tables"}: the earliest and latest valid time of
the batch (text, None for an empty batch), its staged rows, the tables written.
The upper-cased names of the staged features holding PERIOD values, read with
TYPE() on one staged row (the batch is not empty here). A failed read raises: a
PERIOD feature written with the plain SQL fails anyway (Error 6976).
The VALID_TIME_COLUMN setting of process_id, or None.
Fail-closed, unlike the other settings: a table that exists but cannot be read raises, because running such a process with the run's valid time would write a wrong history. No table means no process of this store could have recorded the option.