tdfs4ds.dataset.temporal¶
Temporal mode of a dataset view: CURRENT, a literal AS OF, or a TimeManager.
Pure module (no database access), so every rule here is testable offline.
A dataset view reads temporal feature tables, so it has to say at which valid time it reads them. Three modes exist:
CURRENT (the default)
Outer CURRENT VALIDTIME, feature sub-queries SEQUENCED VALIDTIME.
The view always shows today's values.
AS_OF
A fixed point in time, given as a literal. The outer qualifier is empty and
each feature sub-query reads VALIDTIME AS OF TIMESTAMP '<literal>'. The
view is frozen at that instant.
TIME_MANAGER
A point in time that MOVES with a TimeManager. Each feature sub-query is
NONSEQUENCED VALIDTIME and filters the valid period with the manager's
business date as a scalar::
WHERE B1.ValidStart <= (SELECT CAST(BUSINESS_DATE AS TIMESTAMP(6) WITH TIME ZONE) FROM <tm view>)
AND B1.ValidEnd > (SELECT CAST(BUSINESS_DATE AS TIMESTAMP(6) WITH TIME ZONE) FROM <tm view>)
AND ...
Calling ``time_manager.update(time_id)`` moves the dataset without
recreating it.
**Why not ``VALIDTIME AS OF (SELECT BUSINESS_DATE FROM <tm view>)``** — the
first shape shipped, and the obvious one: in the GROUPED strategy (``FEATURE_ID
IN (...)``) it makes Teradata read every storage table "by way of an all-rows
scan", losing the ``FEATURE_ID`` partition elimination a literal date keeps
("from 3 partitions"). The scalar ``WHERE`` keeps it, in both strategies.
Measured on Vantage 20 (EXPLAIN, 10-feature dataset). The predicate is the
one Teradata itself writes when it rewrites ``AS OF``
(``ValidStart <= CAST(x AS TIMESTAMP(0) WITH TIME ZONE) AND ValidEnd > ...``),
so the instant selected is the same; rows were identical to the literal
form, and the scalar is evaluated once per query, not once per feature.
**The scalar is cast UP to ``TIMESTAMP(6)``, never down to ``TIMESTAMP(0)``**
(0.3.4.1): a TimeManager loaded from a timestamp column stores its business
date as ``TIMESTAMP(6) WITH TIME ZONE``, and Teradata refuses
``CAST(<TIMESTAMP(6)> AS TIMESTAMP(0))`` even with zero fractions (Error 7454,
verified on Vantage 20) - the view was created and every read failed. A
``DATE`` manager casts up just as well, and the valid-period columns
(``TIMESTAMP(0)``) compare against a ``TIMESTAMP(6)`` value unchanged.
Never wrap it in an aggregate (``MAX``): a manager view that wrongly
returned two rows must fail (Error 3669), not silently pick one.
module-attribute
¶
_ISO_RE = compile('^\\s*\n (?P<date>\\d{4}-\\d{2}-\\d{2})\n (?:[ T](?P<time>\\d{2}:\\d{2}(?::\\d{2})?)(?P<frac>\\.\\d{1,6})?)?\n \\s*(?P<tz>Z|[+-]\\d{2}(?::?\\d{2})?)?\n \\s*$', VERBOSE)
module-attribute
¶
_FORMAT_HINT = "Use an ISO date or timestamp such as '2024-03-31', '2024-03-31 23:59:59', '2024-03-31T23:59:59+02:00', or a datetime / date / pandas.Timestamp object."
The resolved temporal mode of one dataset view.
Holds what the SQL needs (outer_qualifier, sub_qualifier) and what
the catalog records (mode, as_of, the TimeManager's location).
property
¶
The manager's business date as a scalar sub-query (TIME_MANAGER only).
property
¶
Leading WHERE conjuncts selecting the valid instant, or ''.
Only the TIME_MANAGER mode needs one (see the module docstring for why
it is not an AS OF sub-query); for the other modes the qualifier
alone selects the instant, and the SQL is byte-identical to before.
The dataset's valid instant moved by offset, as a constant expression.
CURRENT: the query time; AS_OF: the literal; TIME_MANAGER: the manager's
business date, the shift inside the scalar sub-query so that it stays one
single-row read resolved once. Shifting the constant (never the
ValidStart / ValidEnd columns) keeps partition elimination: the
arithmetic is folded into a literal in the plan (verified on Vantage 20).
The value written to DATASET_VALIDTIME (VARCHAR(255)).
CURRENT VALIDTIME or VALIDTIME AS OF TIMESTAMP '...' — the
qualifier the view carries — or TIME_MANAGER <db>.<view> for a
dataset following a TimeManager. A very long quoted manager name could
overflow the column, so it then degrades to the mode name; the build
table holds the full location anyway.
Normalise value to the body of a Teradata TIMESTAMP '...' literal.
Returns 'YYYY-MM-DD HH:MI:SS[.ffffff][+HH:MM]'. A time part is always
present: Teradata rejects TIMESTAMP '2024-03-31' (Error 3706, verified
live), so a bare date becomes midnight. Fractional seconds and a time-zone
offset are kept when given (both verified accepted in VALIDTIME AS OF);
without an offset the literal is read in the session time zone, exactly as
tdfs4ds.FEATURE_STORE_TIME is when features are ingested.
Accepted inputs: ISO strings ('2024-03-31', '2024-03-31 23:59',
'2024-03-31T23:59:59.5Z', '2024-03-31 23:59:59+0200'),
datetime.datetime, datetime.date, pandas.Timestamp and
numpy.datetime64. Anything else — including ambiguous forms such as
'03/31/2024' — raises ValueError naming the accepted formats, rather
than being guessed.
(database, view, hidden table) of a TimeManager object or a 'db.view' name.
resolve_dataset_time(feature_store_time=False, as_of=None, time_manager=None, current_feature_store_time=None) -> DatasetTime
Resolve build_dataset's temporal arguments into one DatasetTime.
- nothing set →
CURRENT(the default, unchanged). as_of=<literal>→AS_OFthat literal, normalised.time_manager=<TimeManager or 'db.view'>→TIME_MANAGER.feature_store_time=True(historical flag) →AS_OFtdfs4ds.FEATURE_STORE_TIMEwhen it is set, elseCURRENT— exactly as before. Its value is now normalised too, which leaves a well-formed timestamp untouched and turns a bare date (rejected by Teradata) into midnight.
The three are mutually exclusive: asking for two different points in time is a mistake to report, not to arbitrate silently.