Skip to content

Temporal model & time travel

Every catalog and every feature value is VALIDTIME temporal. That gives you four capabilities for free:

  1. Audit — every change is preserved with a validity range.
  2. Reproducibility — build a dataset as_of a past timestamp and obtain bit-for-bit identical features.
  3. What-if — register a new process today, ingest yesterday's data with FEATURE_STORE_TIME, compare.
  4. Recovery — undelete by re-querying at a timestamp before the corruption.

The global handle

tdfs4ds.FEATURE_STORE_TIME = '2024-06-01 00:00:00'   # point-in-time
tdfs4ds.FEATURE_STORE_TIME = None                     # current

Every read goes through CURRENT VALIDTIME when FEATURE_STORE_TIME is None and VALIDTIME AS OF <ts> otherwise. The catalogs, the consumer agent and the query optimizer honour this global directly.

Datasets are the exception: a dataset is a permanent view, so its point in time is part of its definition and is passed to build_dataset explicitly — as_of= for a fixed date, time_manager= to follow a TimeManager (see below). Setting FEATURE_STORE_TIME alone builds a current dataset.

Set it before reading, not before writing

FEATURE_STORE_TIME controls reads. Writes always reflect the current transaction time. To replay yesterday's ingestion at yesterday's state, use TimeManager and roll_out instead (see User guide / TimeManager).

Two clocks

tdfs4ds distinguishes two notions of time:

Clock What it represents How to drive it
System time When a row was inserted / updated Implicit — Teradata's VALIDTIME
Business time When the event happened in the real world Explicit — TimeManager advances a "current business date" view that drives source queries

For nightly ingestion, advance both: use TimeManager to step through business dates, and set FEATURE_STORE_TIME = business_time.get_date_in_the_past() so reads stay aligned with writes.

Reading at a past timestamp

tdfs4ds.FEATURE_STORE_TIME = '2024-01-01 00:00:00'

tdfs4ds.process_catalog()   # processes registered on or before that date
tdfs4ds.feature_catalog()   # features that were live then

tdfs4ds.FEATURE_STORE_TIME = None

Datasets at a point in time

A dataset view embeds its temporal qualifier in every feature sub-query. Three modes, all compatible with grouped=True:

Mode build_dataset argument Sub-query qualifier
Current (default) — SEQUENCED VALIDTIME (outer CURRENT VALIDTIME)
Fixed date as_of='2024-01-01' VALIDTIME AS OF TIMESTAMP '2024-01-01 00:00:00'
TimeManager time_manager=bt NONSEQUENCED VALIDTIME, filtered by ValidStart <= (SELECT … BUSINESS_DATE …) AND ValidEnd > (…)
# Point-in-time-correct training set, frozen at a date
past = tdfs4ds.build_dataset(
    entity_id         = ['CUSTOMER_ID'],
    selected_features = selected_features,
    view_name         = 'DS_CUSTOMER_2024_01_01',
    grouped           = True,
    as_of             = '2024-01-01',
)

# A dataset that moves with the business clock: bt.update(i) (or roll_out)
# shifts it to step i without rebuilding the view
rolling = tdfs4ds.build_dataset(
    entity_id         = ['CUSTOMER_ID'],
    selected_features = selected_features,
    view_name         = 'DS_CUSTOMER_ROLLING',
    time_manager      = bt,
)

The historical form, feature_store_time=True with FEATURE_STORE_TIME set, still builds a fixed-date dataset. An existing dataset can be re-timed with rebuild_dataset(view, as_of=…) or rebuild_dataset(view, time_manager=…). See Datasets.

How a write works

  1. upload_features opens a follow-up row (STATUS=RUNNING).
  2. The feature engineering view is queried.
  3. Values are MERGE'd into FS_T_<ENT>_<TYPE> (or UPDATE+INSERT if STORE_FEATURE='UPDATE_INSERT').
  4. The follow-up row is closed (STATUS=COMPLETED) — or remains RUNNING if the process crashed. A stale RUNNING row is the canonical signal that an ingestion run is incomplete.

See the Inspect & monitor page for the follow-up table and recovery patterns.

End period

tdfs4ds.END_PERIOD (default 'UNTIL_CHANGED') controls the validity end-date inserted when a feature value first appears. The default sentinel keeps the row open until a newer value supersedes it, mirroring slowly changing dimension semantics.