tdfs4ds.scheduling.unify¶

tdfs4ds.scheduling.unify ¶

Opt-in unification of filter / time managers across parallel-candidate processes (v0.3.1.147).

When :func:tdfs4ds.analyze_parallel_plan reports unify_candidates — distinct FilterManager / TimeManager objects that hold equivalent content — these helpers repoint a set of processes onto a single canonical manager so one driver (one update() / one roll_out timeline) can drive the whole wave.

Repointing a process means two coordinated writes:

Rewrite the process view's manager reference — the process feature view references the manager's public view by name (a scalar subquery or JOIN); _repoint_process_* swaps that fully-qualified reference for the canonical manager's view and re-issues REPLACE VIEW.
Update the catalog row — FS_FILTER_MANAGER / FS_TIME_MANAGER so run() reconstructs the canonical manager.

Everything is opt-in and defaults to dry_run=True (returns the SQL it would run without touching the store). _add_filter_values / _add_time_steps are the "edit an existing manager" primitives used when the managers are compatible (same columns / timeline shape) but hold different rows — they extend the canonical to the union before repointing.

logger_safe ¶

logger_safe(level, message, *args, **kwargs)

Wrapper around the global logger that only emits logs when tdfs4ds.DISPLAY_LOGS is True. level is a string like "info", "error", etc.

execute_query ¶

execute_query(query, raise_error=False, fetch_results=False, quiet_errors=(), log_errors=True)

Execute a SQL query or a list of queries using the tdml module.

Parameters:

Name	Type	Description	Default
`query`	`str or list`	A single SQL query string or a list of SQL query strings.	required
`raise_error`	`bool`	If True, re-raise exceptions after printing them. Default is False.	`False`
`fetch_results`	`bool`	If True, fetch and return the results of the SQL execution. Default is False.	`False`
`quiet_errors`	`tuple`	Lowercase substrings identifying failures the caller EXPECTS and will handle itself. A matching failure is logged at DEBUG instead of ERROR; whether it is re-raised is still governed by `raise_error` alone. Default `()` — every existing call site keeps byte-identical behaviour.	`()`
`log_errors`	`bool`	False when the caller re-raises (`raise_error=True`) and logs the failure itself, so each failure is logged once.	`True`

Returns: The result of the SQL execution if a single query is passed. None if a list of queries is passed or an exception occurs.

_catalog_upsert_sql ¶

_catalog_upsert_sql(table_name: str, process_id: str, mgr) -> str

CURRENT VALIDTIME UPDATE ... ELSE INSERT for a manager catalog row.

_rewrite_view_reference ¶

_rewrite_view_reference(ddl: str, old_db: str, old_view: str, new_db: str, new_view: str) -> str

Replace every fully-qualified reference to old_db.old_view in a view DDL with new_db.new_view — tolerant of quoting and case ("db"."view" and db.view) - as a whole name: db.X never matches inside db.X_COPY or db.X_HIDDEN. Same pattern as the migration rewrite (one rule for both).

_get_view_ddl ¶

_get_view_ddl(view_name: str) -> Optional[str]

SHOW VIEW DDL for a fully/partly-qualified process view name.

_repoint_process_manager ¶

_repoint_process_manager(process_id: str, view_name: str, current, canonical, catalog_table: str, rewrite_ddl: bool, dry_run: bool) -> Dict[str, Any]

Repoint one process from current manager to canonical (both dicts with database/view/table).

Returns the planned statements; executes them when dry_run is False.

_repoint_process_filter_manager ¶

_repoint_process_filter_manager(process_id, view_name, current, canonical, dry_run=True)

_repoint_process_time_manager ¶

_repoint_process_time_manager(process_id, view_name, current, canonical, dry_run=True)

_add_filter_values ¶

_add_filter_values(database: str, view: str, rows, time_column: Optional[str] = None, dry_run: bool = True) -> Dict[str, Any]

Extend a FilterManager to the union of its current rows and rows.

rows is a teradataml/pandas frame with the manager's data columns (and the time_column when time-filtering). Reuses the manager's own load_filter so the deterministic filter_id re-assignment (ROW_NUMBER + UNIQUE PRIMARY INDEX + COLLECT STATISTICS) invariant is preserved. Dry-run only reports the intent.

_add_time_steps ¶

_add_time_steps(database: str, view: str, rows, time_column: str, dry_run: bool = True) -> Dict[str, Any]

Extend a TimeManager to the union of its current steps and rows (see _add_filter_values).

_unify ¶

_unify(process_ids, canonical, kind, repoint_fn, dry_run)

Shared driver for filter/time unification over a set of processes.

unify_filter_managers ¶

unify_filter_managers(process_ids, canonical=None, dry_run: bool = True) -> Dict[str, Any]

Repoint process_ids onto a single canonical FilterManager.

Parameters¶

process_ids : list[str] Processes to unify (typically one unify_candidates['filter'] group's processes from :func:analyze_parallel_plan). canonical : dict | None {'database','view','table'} of the manager to keep. When None the first process that already has a filter manager is chosen. dry_run : bool When True (default) returns the SQL that would run without executing.

unify_time_managers ¶

unify_time_managers(process_ids, canonical=None, dry_run: bool = True) -> Dict[str, Any]

Repoint process_ids onto a single canonical TimeManager (see unify_filter_managers): each process view is rewritten to read canonical (default: the first process's manager) and its FS_TIME_MANAGER row updated. dry_run=True (default) returns the statements without executing them.

split_time_manager ¶

split_time_manager(process_ids, table_name: str, schema_name: Optional[str] = None, clone_mode: str = 'soft', dry_run: bool = True) -> Dict[str, Any]

Give process_ids their own TimeManager cursor (0.3.1.240) - the inverse of :func:unify_time_managers.

The processes must share one TimeManager today. A clone named table_name is made with TimeManager.clone_timer: clone_mode='soft' (default) keeps the same dates (the clone reads the source's hidden table) with its own cursor; 'hard' copies the dates into a table of its own. Each process view is then rewritten onto the clone and the pairing is recorded in FS_TIME_MANAGER. The processes can then run at another business date than the ones left on the source manager, for example in parallel.

A process that reads the manager only through an intermediate view cannot be rewritten here (its own view does not name the manager); it is reported as not_direct and left alone.

dry_run=True (default) creates nothing and returns the statements. schema_name: the database of the clone (default the source manager's). With a domain clock (0.3.6.0), upload_features makes the clones by itself and tdfs4ds.align_domain_time_managers adopts processes registered before it.