tdfs4ds.scheduling.unify¶

tdfs4ds.scheduling.unify ¶

Opt-in unification of filter / time managers across parallel-candidate processes (v0.3.1.147).

When :func:tdfs4ds.analyze_parallel_plan reports unify_candidates — distinct FilterManager / TimeManager objects that hold equivalent content — these helpers repoint a set of processes onto a single canonical manager so one driver (one update() / one roll_out timeline) can drive the whole wave.

Repointing a process means two coordinated writes:

Rewrite the process view's manager reference — the process feature view references the manager's public view by name (a scalar subquery or JOIN); _repoint_process_* swaps that fully-qualified reference for the canonical manager's view and re-issues REPLACE VIEW.
Update the catalog row — FS_FILTER_MANAGER / FS_TIME_MANAGER so run() reconstructs the canonical manager.

Everything is opt-in and defaults to dry_run=True (returns the SQL it would run without touching the store). _add_filter_values / _add_time_steps are the "edit an existing manager" primitives used when the managers are compatible (same columns / timeline shape) but hold different rows — they extend the canonical to the union before repointing.

_CREATE_VIEW `module-attribute` ¶

_CREATE_VIEW = compile('^\\s*(?:CREATE|REPLACE|CV|RV)\\s+VIEW\\b', IGNORECASE)

logger_safe ¶

logger_safe(level, message, *args, **kwargs)

Wrapper around the global logger that only emits logs when tdfs4ds.DISPLAY_LOGS is True. level is a string like "info", "error", etc.

execute_query ¶

execute_query(query, raise_error=False, fetch_results=False, quiet_errors=(), log_errors=True)

Execute a SQL query or a list of queries using the tdml module.

Parameters:

Name	Type	Description	Default
`query`	`str or list`	A single SQL query string or a list of SQL query strings.	required
`raise_error`	`bool`	If True, re-raise exceptions after printing them. Default is False.	`False`
`fetch_results`	`bool`	If True, fetch and return the results of the SQL execution. Default is False.	`False`
`quiet_errors`	`tuple`	Lowercase substrings identifying failures the caller EXPECTS and will handle itself. A matching failure is logged at DEBUG instead of ERROR; whether it is re-raised is still governed by `raise_error` alone. Default `()` — every existing call site keeps byte-identical behaviour.	`()`
`log_errors`	`bool`	False when the caller re-raises (`raise_error=True`) and logs the failure itself, so each failure is logged once.	`True`

Returns: The result of the SQL execution if a single query is passed. None if a list of queries is passed or an exception occurs.

_catalog_upsert_sql ¶

_catalog_upsert_sql(table_name: str, process_id: str, mgr) -> str

CURRENT VALIDTIME UPDATE ... ELSE INSERT for a manager catalog row.

_rewrite_view_reference ¶

_rewrite_view_reference(ddl: str, old_db: str, old_view: str, new_db: str, new_view: str) -> str

Replace every fully-qualified reference to old_db.old_view in a view DDL with new_db.new_view — tolerant of quoting and case ("db"."view" and db.view) - as a whole name: db.X never matches inside db.X_COPY or db.X_HIDDEN. Same pattern as the migration rewrite (one rule for both).

_get_view_ddl ¶

_get_view_ddl(view_name: str) -> Optional[str]

SHOW VIEW DDL for a fully/partly-qualified process view name.

_repoint_process_manager ¶

_repoint_process_manager(process_id: str, view_name: str, current, canonical, catalog_table: str, rewrite_ddl: bool, dry_run: bool) -> Dict[str, Any]

Repoint one process from current manager to canonical (both dicts with database/view/table).

Returns the planned statements; executes them when dry_run is False.

_repoint_process_filter_manager ¶

_repoint_process_filter_manager(process_id, view_name, current, canonical, dry_run=True)

_repoint_process_time_manager ¶

_repoint_process_time_manager(process_id, view_name, current, canonical, dry_run=True)

_add_filter_values ¶

_add_filter_values(database: str, view: str, rows, time_column: Optional[str] = None, dry_run: bool = True) -> Dict[str, Any]

Extend a FilterManager to the union of its current rows and rows.

rows is a teradataml/pandas frame with the manager's data columns (and the time_column when time-filtering). Reuses the manager's own load_filter so the deterministic filter_id re-assignment (ROW_NUMBER + UNIQUE PRIMARY INDEX + COLLECT STATISTICS) invariant is preserved. Dry-run only reports the intent.

_add_time_steps ¶

_add_time_steps(database: str, view: str, rows, time_column: str, dry_run: bool = True) -> Dict[str, Any]

Extend a TimeManager to the union of its current steps and rows (see _add_filter_values).

_unify ¶

_unify(process_ids, canonical, kind, repoint_fn, dry_run)

Shared driver for filter/time unification over a set of processes.

unify_filter_managers ¶

unify_filter_managers(process_ids, canonical=None, dry_run: bool = True) -> Dict[str, Any]

Repoint process_ids onto a single canonical FilterManager.

Parameters¶

process_ids : list[str] Processes to unify (typically one unify_candidates['filter'] group's processes from :func:analyze_parallel_plan). canonical : dict | None {'database','view','table'} of the manager to keep. When None the first process that already has a filter manager is chosen. dry_run : bool When True (default) returns the SQL that would run without executing.

unify_time_managers ¶

unify_time_managers(process_ids, canonical=None, dry_run: bool = True) -> Dict[str, Any]

Repoint process_ids onto a single canonical TimeManager (see unify_filter_managers): each process view is rewritten to read canonical (default: the first process's manager) and its FS_TIME_MANAGER row updated. dry_run=True (default) returns the statements without executing them.

split_time_manager ¶

split_time_manager(process_ids, table_name: str, schema_name: Optional[str] = None, clone_mode: str = 'soft', dry_run: bool = True) -> Dict[str, Any]

Give process_ids their own TimeManager cursor (0.3.1.240) - the inverse of :func:unify_time_managers.

The processes must share one TimeManager today. A clone named table_name is made with TimeManager.clone_timer: clone_mode='soft' (default) keeps the same dates (the clone reads the source's hidden table) with its own cursor; 'hard' copies the dates into a table of its own. Each process view is then rewritten onto the clone and the pairing is recorded in FS_TIME_MANAGER. The processes can then run at another business date than the ones left on the source manager, for example in parallel.

A process that reads the manager only through an intermediate view cannot be rewritten here (its own view does not name the manager); it is reported as not_direct and left alone.

dry_run=True (default) creates nothing and returns the statements. schema_name: the database of the clone (default the source manager's). With a domain clock (0.3.6.0), upload_features makes the clones by itself and tdfs4ds.align_domain_time_managers adopts processes registered before it.

_as_replace ¶

_as_replace(ddl: str) -> str

A view text as REPLACE VIEW (SHOW VIEW returns what was typed: CREATE or REPLACE).

clone_shared_view_time_manager ¶

clone_shared_view_time_manager(view: str, *, clone_name: Optional[str] = None, dry_run: bool = True) -> Dict[str, Any]

Give the processes that read a TimeManager through a shared view one clone of their own (0.3.6.8).

view ('db.view', not a process view) filters its source with a TimeManager, for example the domain clock, and several processes read it. None of them can get its own soft clone, because rewriting view for one rewrites it for all. This makes one soft clone of that manager next to view (<view>_TM or clone_name: same dates, own cursor, at the manager's current step), rewrites only the manager reference inside view (nothing else of its text changes, and it is never copied), and records the clone as the TimeManager of every registered process reading view (FS_TIME_MANAGER + lineage). The readers then move together, apart from the manager and from other processes; roll_out(data_domain=...) keeps the clone in step with the clock.

Copying view per process, or moving its date filter into each process view, gives each process a cursor of its own; that changes your SQL and is left to you.

An existing object with the clone's name is reused only when it is already a soft clone over the same dates; anything else is refused (nothing is created or rewritten).

dry_run=True (default) returns the statements and the readers without changing anything. Returns {view, manager, clone, readers, skipped, statements, dry_run} plus error and hint when refused; never raises for a refusal.

tdfs4ds.scheduling.unify¶