tdfs4ds.process_store.process_store_catalog_management¶

tdfs4ds.process_store.process_store_catalog_management ¶

quote_ident ¶

quote_ident(name, force=False)

Return name as a Teradata identifier, quoting it only if required.

Idempotent: an already-quoted name is returned unchanged rather than being wrapped a second time. Embedded quotes are escaped as "".

Parameters¶

name : str The raw object/database name. force : bool, default False Quote even when the name does not need it. Use only at the few sites that have always emitted unconditional quotes and whose exact output is already baked into persisted DDL (see feature_store/feature_query_retrieval.py) — elsewhere forcing would change the SQL of existing stores for no reason.

qualified ¶

qualified(database, object_name, force=False)

Build db.obj with each part quoted only as needed.

qualified('TDB', 'FS_X') == 'TDB.FS_X' (unchanged for ordinary names); qualified('SVC@CORP', 'FS_X') == '"SVC@CORP".FS_X'. An empty/None database yields the bare object name, so unqualified callers keep working.

force=True quotes both parts unconditionally — only for the sites that have always emitted quotes and whose exact text is already persisted (dataset view DDL, returned volatile-table names).

split_qualified ¶

split_qualified(name)

Split db.obj into (database, object), both de-quoted.

Quote-aware, so a dot inside a quoted part does not split the name: split_qualified('"DEINF.CARLOSN"."FS_X"') == ('DEINF.CARLOSN', 'FS_X'). The naive rsplit('.', 1) this replaces returns ('"DEINF.CARLOSN"', '"FS_X"') only by luck and ('"DEINF', 'CARLOSN"."FS_X"') in the split('.') variant.

An unqualified name returns ('', name).

execute_query ¶

execute_query(query, raise_error=False, fetch_results=False, quiet_errors=(), log_errors=True)

Execute a SQL query or a list of queries using the tdml module.

Parameters:

Name	Type	Description	Default
`query`	`str or list`	A single SQL query string or a list of SQL query strings.	required
`raise_error`	`bool`	If True, re-raise exceptions after printing them. Default is False.	`False`
`fetch_results`	`bool`	If True, fetch and return the results of the SQL execution. Default is False.	`False`
`quiet_errors`	`tuple`	Lowercase substrings identifying failures the caller EXPECTS and will handle itself. A matching failure is logged at DEBUG instead of ERROR; whether it is re-raised is still governed by `raise_error` alone. Default `()` — every existing call site keeps byte-identical behaviour.	`()`
`log_errors`	`bool`	False when the caller re-raises (`raise_error=True`) and logs the failure itself, so each failure is logged once.	`True`

Returns: The result of the SQL execution if a single query is passed. None if a list of queries is passed or an exception occurs.

execute_query_wrapper ¶

execute_query_wrapper(f)

Decorator to execute a query. It wraps around the function and adds exception handling.

This decorator is used to wrap around a function that generates a query and execute that query. It handles exceptions during query execution and prints the error message along with the query. The decorator works with both TeradataML's tdml and legacy tdml.get_context() execution methods, depending on the TeradataML version.

Parameters:

Name	Type	Description	Default
`f`	`function`	Function to be decorated.	required

Returns:

Name	Type	Description
`function`		Decorated function.

Example

To decorate a function that generates and executes a query:

@execute_query_wrapper def my_query_function(): return "SELECT * FROM my_table"

This decorator can be applied to functions that generate and execute queries with ease.

create_object_if_missing ¶

create_object_if_missing(create_query, label, follow_up=())

Run an idempotent CREATE and report honestly what happened.

This exists because of a structural trap: execute_query defaults to raise_error=False, so it logs the failure itself and returns normally — an except wrapped around it never sees anything. Every catalog-creation block written that way therefore logged an ERROR for a store that was simply already provisioned, then went on to log "has been created" for a table it had not created, while its own already-exists handling sat unreachable.

Parameters¶

create_query : str The CREATE statement. label : str Human-readable object name for the log lines (e.g. "TABLE DB.FS_X"). follow_up : iterable of str Statements to run once the object is known to be present, whether this call created it or found it. They must be idempotent — COMMENT ON and REPLACE VIEW are the intended uses.

Returns¶

bool True if this call created the object, False if it was already there.

Raises¶

Exception The original driver exception for any failure that is NOT "already exists" — logged at ERROR with the full message, because the half that names the fault is often the later half.

describe_db_error ¶

describe_db_error(exc) -> str

The error message without the driver's stack trace.

Keeps EVERY message line — the half that names the fault is often not the first line — and drops only the Go frames teradatasql appends and the SQLAlchemy "Background on this error" link. Falls back to the raw text when nothing would be left.

logger_safe ¶

logger_safe(level, message, *args, **kwargs)

Wrapper around the global logger that only emits logs when tdfs4ds.DISPLAY_LOGS is True. level is a string like "info", "error", etc.

upgrade_process_catalog ¶

upgrade_process_catalog()

process_catalog_select_sql ¶

process_catalog_select_sql()

The SELECT behind FS_V_PROCESS_CATALOG, without its temporal qualifier.

The view prefixes it with CURRENT VALIDTIME; the admin app's catalog snapshot prefixes it with VALIDTIME AS OF TIMESTAMP '...' to read the processes as they were registered at another valid time. One text for both.

process_store_catalog_view_creation ¶

process_store_catalog_view_creation()

Constructs SQL queries to replace views related to the process store catalog.

This function generates two SQL queries:

A query to replace a view in the process store catalog with the content of the process catalog table. The query uses the REPLACE VIEW statement and locks rows for access during execution.
A query to replace a view that contains split features from the process catalog table. This query utilizes a text processing function (NGramSplitter) to extract n-grams from a specified text column and creates a view based on the processed data.

Returns:

Name	Type	Description
`tuple`		A pair of SQL query strings: - `query_1`: SQL to replace the primary process catalog view. - `query_2`: SQL to replace the split features view with processed data.

process_store_catalog_creation ¶

process_store_catalog_creation(if_exists='fail', comment='this table is a process catalog')

Create or replace a feature store catalog table in Teradata database.

This function creates a catalog table in the specified schema of the Teradata database. The catalog table is used to store information about features, including their names, associated tables, databases, validity periods, etc.

if_exists (str, optional): Specifies the behavior if the catalog table already exists. Default is 'fail': an existing table is left untouched. 'replace' DROPS it and recreates it empty.

The default was 'replace' until 0.3.1.211, but that branch could never run: execute_query swallowed the "already exists" failure, so the except guarding it was dead code. Fixing the log noise made the failure visible, which would have turned every setup() on a provisioned store into a catalog wipe — setup() calls this function with no argument. 'fail' therefore preserves the observable behaviour exactly; setup() still does its own explicit dropping when the caller asks for it. - comment (str, optional): A comment to describe the catalog table. Default is 'this table is a process catalog'.

Returns: str: The name of the created or replaced catalog table.

Notes: - The schema in which the catalog table will be created should be specified separately. - If the catalog table already exists and if_exists is set to 'replace', the existing table will be dropped and recreated. - This function also creates a secondary index on the 'PROCESS_TYPE' column for faster querying. - The catalog table structure includes columns for process ID, process type, view name, entity ID, feature names, feature version, data domain, metadata, and validity period. - This function relies on external global variables and configurations like 'tdfs4ds' and 'tdml' which should be set in the environment. - It also depends on an 'execute_query' function for executing SQL queries.

Dependencies: - 'tdfs4ds' global variables for schema and table name. - 'tdml' for displaying SQL queries (if configured). - 'execute_query' for executing SQL queries.

time_manager_catalog_creation ¶

time_manager_catalog_creation()

Create the FS_TIME_MANAGER catalog table if it does not already exist.

Non-destructive, idempotent counterpart to the FS_TIME_MANAGER block inside :func:process_store_catalog_creation — it NEVER drops an existing table. Used by :func:tdfs4ds.connect to self-heal installs whose process catalog predates v0.3.1.147 (so the FS_V_PROCESS_CATALOG view's LEFT JOIN FS_TIME_MANAGER resolves). The table mirrors FS_FILTER_MANAGER exactly.

Returns:

Name	Type	Description
`str`		the FS_TIME_MANAGER table name.

record_process_time_manager ¶

record_process_time_manager(process_id, time_manager)

Upsert a process → TimeManager association into FS_TIME_MANAGER (v0.3.1.147).

Called from :func:tdfs4ds.roll_out as it drives a time_manager across a process_list — the natural home for the association, since a TimeManager is a roll-out driver (one timeline, many processes), not a registration-time property of a single process like the FilterManager. Running an existing rollout therefore auto-populates the catalog, including for processes registered by tdfs4ds versions predating this table.

Idempotent (UPDATE … ELSE INSERT keyed on PROCESS_ID) and non-fatal — any failure is logged and swallowed so it can never abort a rollout.

get_process_info ¶

get_process_info(process_id: str) -> dict

Retrieve process information including SQL and columns from the process catalog.

Parameters:

Name	Type	Description	Default
`process_id`	`str`	The unique identifier of the process.	required

Returns: dict: A dictionary containing process SQL and columns.