Skip to content

tdfs4ds.feature_store.history_start

tdfs4ds.feature_store.history_start

Where a process's stored history starts, and the roll_out guard built on it (0.3.5.16).

A run at business date t writes its values from t onward: it rewrites what is stored after t, but it never inserts anything before the first value already stored. So once a process has values from date T (a first run in real time, a run at a later TimeManager date, a roll-out started at a later step), rolling it out over dates before T adds nothing and raises nothing: every step rewrites the period from T onward, and the history ends up as one period holding the last date's value.

roll_out reads, before step 1, where each process's stored values start (:func:history_blockers) and refuses when that is after the first date it would roll out. roll_out(..., rebuild_history=True) deletes those processes' stored values (:func:purge_process_history) and replays every date instead.

Layers: the SQL builders are pure; the readers take an injectable run_sql and are fail-open (a read that fails is reported as unverified, never as a blocker).

qualified

qualified(database, object_name, force=False)

Build db.obj with each part quoted only as needed.

qualified('TDB', 'FS_X') == 'TDB.FS_X' (unchanged for ordinary names); qualified('SVC@CORP', 'FS_X') == '"SVC@CORP".FS_X'. An empty/None database yields the bare object name, so unqualified callers keep working.

force=True quotes both parts unconditionally — only for the sites that have always emitted quotes and whose exact text is already persisted (dataset view DDL, returned volatile-table names).

sql_literal

sql_literal(value)

Escape value for use inside a single-quoted SQL string literal.

For the DBC probes and catalog predicates that compare a name as data (WHERE DatabaseName = '…'). The caller supplies the surrounding quotes.

_table_ref

_table_ref(table: str) -> str

"DB.TABLE" (as :func:process_storage_tables returns it) as an identifier.

history_start_sql

history_start_sql(process_id: str, tables: Sequence[str], first_date: str) -> str

One statement: the earliest stored ValidStart of process_id over tables, as text, and 1 when it is after first_date (the body of a TIMESTAMP '...' literal, read in the session time zone like the run's own valid time), else 0.

Each table is read in its own NONSEQUENCED derived table: a temporal qualifier cannot open a UNION branch (Error 9345).

purge_history_sql

purge_history_sql(process_id: str, tables: Sequence[str], since: str) -> List[str]

The statements deleting the stored values of process_id that start at or after since (the first date rolled out). The guard only purges a process whose values all start after it, so in practice this is all of them; the bound states the rule and keeps anything earlier safe if a run wrote it between the check and the purge.

_default_run_sql

_default_run_sql(sql: str)

_skips_guard

_skips_guard(process_id: str) -> bool

A process whose valid time comes from a source column writes its own history in one run (valid_time_column): the roll-out dates do not stamp its values.

history_blockers

history_blockers(process_list: Iterable[str], first_date: str, *, tables: Optional[Dict[str, set]] = None, run_sql: Optional[Callable[[str], list]] = None) -> dict

{"blocked": [{"process_id", "start", "tables"}], "unverified": [{"process_id", "error"}]}.

blocked: processes whose stored values start after first_date. A process with no stored value (registered with run_now=False, or features not registered yet) is not blocked. tables: {PROCESS_ID: {"DB.TABLE", ...}}; read from the catalogs when None.

blocked_message

blocked_message(blocked: List[dict], first_date: str) -> str

The refusal: what is stored, why the roll-out would add nothing, the two ways out.

purge_process_history

purge_process_history(process_id: str, tables: Sequence[str], since: str, *, run_sql: Optional[Callable[[str], object]] = None) -> None

Delete the stored values of process_id starting at or after since (and its entity-refresh rows), logged in FS_REMOVAL_LOG as a VALUES removal. Raises on a failed DELETE.