tdfs4ds.migration.planner¶
Cross-store migration planner (v0.3.1.150) — the read-only phase.
plan_migration(seeds, source) resolves a mixed list of process / dataset /
feature names in a distant (lab) store, expands the upstream process
closure (the self-contained migration unit — the same closure the parallel
scheduler computes), inventories every object the closure touches (process views,
datasets, intermediate views, raw source tables), classifies them, detects
name conflicts against the primary (production) store, runs the replace
downstream-safety gate, and inventories the filter/time managers — returning
a JSON-serialisable manifest that requires approval before anything is written.
This phase never writes. It reuses the pure graph core of
:mod:tdfs4ds.scheduling.parallel_plan (_resolve_seeds / _build_nodeid_to_pid
/ _process_dependency_edges / _build_process_dag / _upstream_closure /
_compute_waves) — only the catalog reads are re-scoped to the source store.
Wrapper around the global logger that only emits logs when
tdfs4ds.DISPLAY_LOGS is True. level is a string like "info", "error", etc.
Uppercase, de-quoted, stripped — the canonical comparison form.
Split 'db.obj' / '"db"."obj"' / 'obj' into (db, obj) (raw,
de-quoted but original case preserved). db is '' when unqualified.
build_placement_map(objects, source_desc, primary_schema, primary_view_schema, renames=None) -> Dict[Tuple[str, str], Tuple[str, str]]
Build the source→primary placement map for a list of classified objects.
Parameters¶
objects : iterable of dict
Each {'database','object','object_type'} where object_type is one
of 'table' | 'view' | 'dataset' | 'process_view' | 'intermediate_view'
| 'source_table' (anything containing 'table' lands in the table
schema, everything else in the view schema).
source_desc : dict
The read-only descriptor {'schema','schema_view',...} of the source
store (used only as the default source DB when an object's DB is blank).
primary_schema : str
Primary table database.
primary_view_schema : str
Primary view database (view_schema()).
renames : dict, optional
{(SRC_DB_UP, SRC_OBJ_UP): NEW_OBJ_NAME} for objects renamed to avoid a
conflict. The destination DB is still chosen by object type.
Returns¶
dict
{(SRC_DB_UP, SRC_OBJ_UP): (DST_DB, DST_OBJ)}.
A source store's public view as a pandas frame (fail-open to empty).
A PRIMARY store public view (NOT federated) — used for conflict checks.
Walk every closure process view's DDL and inventory + classify all objects it
reaches: process_view / dataset / intermediate_view / source_table.
build_teradata_dependency_graph already skips FS-internal technical tables
and stops at dataset / process-view level, so the remaining tables are raw
sources and the remaining views are intermediate.
(DB_UP, OBJ_UP) of objects already present in the primary schema/view schema.
_build_downstream_gate(dst_db, dst_obj, otype, incoming_names, primary_graph, primary_proc_by_key, primary_ds_features_by_node) -> Dict[str, Any]
Compute the downstream feature-loss verdict for replacing a primary object.
{data_domain: clock} read from the SOURCE store's FS_DATA_DOMAIN_SETTINGS
(explicit schema, read-only), with the clock's current cursor. Fail-open: a store
without the table (before 0.3.6.0) or a domain without a clock gives nothing.
plan_migration(seeds=None, source=None, *, all_domains: bool = True, target_data_domain: Optional[str] = None, data_strategy: str = 'recompute', attach: bool = True) -> Dict[str, Any]
_plan_migration(seeds=None, source=None, *, all_domains: bool = True, target_data_domain: Optional[str] = None, data_strategy: str = 'recompute', attach: bool = True) -> Dict[str, Any]
Plan a cross-store migration (READ-ONLY — nothing is written).
Parameters¶
seeds : list[str] | None
Mixed process (id/view) / dataset / feature names in the source store.
None = the whole source store.
source : str | dict
The source store (a schema string or a {'schema','schema_view',...}
descriptor). Attached read-only for the duration when attach=True.
all_domains : bool
Consider all source data domains (default).
target_data_domain : str | None
Re-route every migrated feature under this DATA_DOMAIN in the primary
(avoids feature-name collisions with existing production domains). None
keeps each feature's source domain.
data_strategy : 'recompute' | 'copy' | 'none'
Default per-process data strategy (recompute recreates + re-runs, at
the business date of the process's TimeManager when it has one;
copy snapshots the FS_T_* values; none registers the processes
without computing, for a primary that replays its history itself with
roll_out - a value computed first cannot be preceded by earlier
valid times afterwards). Per-process overrides can be applied at
execution.
attach : bool
Attach source read-only if not already attached (default True).
Returns¶
dict
The migration manifest (see module docstring). Pass it to
:func:tdfs4ds.migrate_objects with an approval map.