tdfs4ds.lifecycle¶
Lifecycle management for the tdfs4ds feature store.
Contains setup() and connect() — the two initialization functions that must be called once at the start of every session.
property
¶
True when the build table and its view exist (checked once, then cached).
property
¶
True when the window table and its view exist (checked once, then cached).
property
¶
True when the offsets table and its view exist (checked once, then cached).
__init__(schema_name=None, name='DATASET', existing_objects=None, schema_name_view=None, existing_view_objects=None)
Parameters¶
existing_objects : set, optional
A pre-computed set of lower-cased table/view names already present in
schema_name. When provided, the constructor skips its own
db_list_tables() call (saves one round-trip). Pass the set
returned by connect()'s DBC.TablesV query.
schema_name_view : str, optional
Database that holds the catalog VIEW objects. When None it defaults to
tdfs4ds.SCHEMA_VIEW if a dedicated view database is configured, else
to schema_name (co-located with the tables — byte-identical to the
historical single-database behaviour).
existing_view_objects : set, optional
Pre-computed set of lower-cased view names present in schema_name_view.
Only consulted when the view database differs from the table database.
Create whatever part of the catalog is missing. Idempotent.
Creates only the absent objects and leaves the existing ones alone, returning the list of names it created (empty when there was nothing to do). Creation order is preserved, so a table is always created before the view that reads it.
It used to refuse — logging an ERROR and raising — as soon as any
of the six objects existed, while _exists only reported True when
all six did. Those two predicates disagree on every partial catalog,
which left it permanently unrepairable: the constructor sees "not all
present", calls this, and this rejects "some present". Because the
constructor self-heals on every instantiation, that fired on ordinary
READ paths (dataset_catalog(), the agent's _resolve_dataset) and
on connect()/setup() — an ERROR-level log and an exception for a
condition the caller cannot act on. Self-healing is this constructor's
long-standing contract; it was simply incomplete.
Register (or update) dataset in the catalog. Returns its DATASET_ID.
Parameters¶
build : dict, optional
How the view was built — {'strategy', 'join_type', 'time'
(DatasetTime), 'source_dataset_id'} — written to the build table
when it exists. Omitted: the build table is left untouched.
dataset_id : str, optional
Force the DATASET_ID to write. Used by restore_dataset_catalog
when a dataset was registered several times and one id must be kept;
ordinary callers let it be looked up by name.
offsets : list of dict, optional
One row per column read at an offset (lags and leads), written to the
offsets table when it exists; [] closes the rows of an earlier
build. None (default) leaves the table untouched.
Build details (strategy, join type, temporal mode) — None when unavailable.
Offset columns (one row per lag / lead column) — None when unavailable.
Join it to get_dataset_features on (DATASET_ID, FEATURE_NAME): a
column without offset has no row.
History windows (one row per historical dataset) — None when unavailable.
Join it to the catalog on DATASET_ID with a LEFT OUTER JOIN: a
snapshot dataset has no row. WINDOW_START / WINDOW_END are the
bounds at query time (WINDOW_START NULL = from the earliest stored
value); the other columns are the definition.
Remove a dataset (catalog + entity + feature rows) by DATASET_ID.
Parameters¶
dataset_id : str
The DATASET_ID to remove.
purge : bool, optional
When False (default), a CURRENT VALIDTIME DELETE valid-time-closes
the currently-valid rows (temporal soft delete; history is retained).
When True, a NONSEQUENCED VALIDTIME DELETE physically removes every
temporal row for the dataset — use this to fully reset a dataset that
carries stale/closed history before re-registering it.
close_lineage : bool, optional
True (default): the dataset view's edges in FS_LINEAGE_GRAPH are
closed as well. False when the view itself stays (a duplicate
registration removed by restore_dataset_catalog).
Raises¶
ValueError
If no dataset with this dataset_id currently exists in the catalog.
The error message without the driver's stack trace.
Keeps EVERY message line — the half that names the fault is often not the first line — and drops only the Go frames teradatasql appends and the SQLAlchemy "Background on this error" link. Falls back to the raw text when nothing would be left.
Say so, once, when the store's database needs quoted identifiers.
A logon database such as SVC@CORP or DEINF.CARLOSN is only legal in
SQL when double-quoted. tdfs4ds quotes it automatically everywhere it builds
an identifier, so the store itself works -- but a few peripheral features
still assume a bare-legal name, and meeting those as an obscure failure much
later is worse than reading one line here. Never raises.
Probe VALIDTIME support once, before setup() creates anything.
Every catalog and storage table is a VALIDTIME table. On a system without the temporal feature each CREATE fails with "Temporal operations are not supported on this system", and setup() used to carry on regardless: one error per table, then a secondary index on each missing table (Error 3807), each with the driver's stack trace — a long log whose cause sat in the middle of it. The probe is a session-only volatile table (no perm space, nothing left behind).
Returns False only when the probe failed for a TEMPORAL reason; any other probe failure (or no probe at all) lets setup() proceed, so a transient issue in the probe can never block a store that would have worked.
Say whether the core catalog tables really exist, rather than assume it.
The creation helpers log each object they create (or find) and each failure with its message; this closes the run with one line that cannot claim a success that did not happen. Returns the list of missing tables, or None when they could not be listed.
Initialize the feature store environment by creating catalog tables and views.
Parameters¶
database : str Database that holds the physical feature-store TABLES. if_exists : str 'fail' (default) or 'replace' — drop existing catalog objects first. database_view : str, optional Dedicated database for VIEW objects (process views, dataset views, and all framework public FS_V_ views). When None (default) views are co-located with the tables in database — byte-identical to the historical behaviour. When set, every view is created in this database while tables stay in database* (the view database must be granted SELECT WITH GRANT on the table database).
connect(database=None, database_view=None, feature_catalog_name=None, process_catalog_name=None, data_distribution_name=None, filter_manager_name=None, followup_name=None, feature_catalog_name_view=None, process_catalog_name_view=None, dataset_catalog_name=None, documentation_process_business_logic=None, documentation_process_features=None, documentation_process_explain=None, documentation_business_dictionary_objects=None, documentation_business_dictionary_columns=None, documentation_business_dictionary_sections=None, databases_readonly=None, create_if_missing=False)
Connect the session to an existing feature store.
Needs an open teradataml context (tdml.create_context, tdfs4ds.open_session or
tdfs4ds.reconnect_teradata). Sets tdfs4ds.SCHEMA (and SCHEMA_VIEW), checks
which catalog objects exist (one DBC read), refreshes the catalog views, creates the
optional tables a newer version adds when they are missing (process options and
settings, placement, removal log, documentation tracking, data domain settings...;
a refused CREATE is a warning, never a failure), and configures the Vector Store
authentication when TDFS4DS_VECTOR_STORE_AUTH_* is set.
Parameters¶
database : str, optional
Database of the store's tables. Default: tdfs4ds.SCHEMA.
database_view : str, optional
A separate database for the store's views (see setup). Default: the same.
databases_readonly : list, optional
Other feature stores to read, never write (cross-store read federation): a database
name, or {'schema', 'schema_view', 'vector_prefix'}. None keeps none.
create_if_missing : bool, optional
Create the missing catalog objects (as setup would) instead of reporting them.
Default False: a store with missing catalogs is reported and not connected.
feature_catalog_name, process_catalog_name, ... : str, optional
Override the catalog object names (default: the tdfs4ds module constants).
Returns¶
bool True when the store is connected, False when its catalogs are missing.