tdfs4ds.migration.rewrite

tdfs4ds.migration.rewrite

Pure DB-reference / object-name rewriting helpers for cross-store migration (v0.3.1.150).

Every persisted catalog / business-dictionary / lineage row in tdfs4ds carries its own absolute database name (FEATURE_DATABASE / DATABASE_NAME / PARENT|CHILD_DATABASE_NAME / a qualified VIEW_NAME). Migrating an object from a source store into the primary therefore means recreating it in the primary schema and rewriting every reference to the source store so it points at the migrated (primary) placement — a missed reference would silently read the source store forever.

These helpers are pure (no Teradata, no tdfs4ds globals) so the migration executor's linchpin transformation is unit-testable in isolation. They generalise :func:tdfs4ds.scheduling.unify._rewrite_view_reference (which rewrites a single manager reference) to an arbitrary object-placement map.

Key concepts

• A placement map is {(SRC_DB_UP, SRC_OBJ_UP): (DST_DB, DST_OBJ)} — for every object being migrated, where it lands in the primary (tables land in the primary table schema, views in the primary view schema; a renamed object gets its new name here). rewrite_references swaps each fully-qualified src_db.src_obj in a DDL / catalog string for its dst_db.dst_obj.
• remap_process_ids rewrites the FEATURE_VERSION='<uuid>' process references baked into a dataset view's DDL when process ids are re-minted.

_IDENT module-attribute

_IDENT = '[A-Za-z_][A-Za-z0-9_$#]*'

_NOT_BEFORE module-attribute

_NOT_BEFORE = '(?<![A-Za-z0-9_$#])'

_NOT_AFTER module-attribute

_NOT_AFTER = '(?![A-Za-z0-9_$#])'

MAPPABLE_KINDS module-attribute

MAPPABLE_KINDS = ('source_table', 'intermediate_view', 'process_view', 'dataset')

norm_name

norm_name(name: Optional[str]) -> str

Uppercase, de-quoted, stripped — the canonical comparison form.

split_qualified

split_qualified(name: Optional[str]) -> Tuple[str, str]

Split 'db.obj' / '"db"."obj"' / 'obj' into (db, obj) (raw, de-quoted but original case preserved). db is '' when unqualified.

_ref_pattern

_ref_pattern(db: str, obj: str) -> re.Pattern

Case/quote-tolerant regex matching a fully-qualified db.obj reference, as a whole name (never a prefix or a suffix of a longer one).

_schema_prefix_pattern

_schema_prefix_pattern(schema: str) -> re.Pattern

Match schema.<object> capturing the (de-quoted) object token.

rewrite_references

rewrite_references(text: str, placement_map: Dict[Tuple[str, str], Tuple[str, str]]) -> str

Rewrite every fully-qualified reference named in placement_map.

Parameters

text : str A DDL string or a catalog value (e.g. a VIEW_NAME). placement_map : dict {(SRC_DB_UPPER, SRC_OBJ_UPPER): (DST_DB, DST_OBJ)}. Keys must be normalised (uppercase, de-quoted) — use :func:norm_name.

Returns

str text with each src_db.src_obj replaced by "dst_db"."dst_obj". Precise: only fully-qualified references are touched (never bare column tokens), so it is safe to run over an entire view DDL.

rewrite_schema_prefix

rewrite_schema_prefix(text: str, src_schema: str, dst_schema: str) -> str

Swap every src_schema.<object> prefix for dst_schema.<object>.

Use for a whole-object move where every reference to a source schema should follow to one destination schema (e.g. a manager backing view, or a leftover reference not covered by a placement map). Precise to qualified references.

remap_process_ids

remap_process_ids(text: str, pid_map: Dict[str, str]) -> str

Replace each source PROCESS_ID with its primary id everywhere in text.

Process ids are UUIDs, so a literal, word-boundary-guarded string replacement is unambiguous. Used to rewrite a dataset view's FEATURE_VERSION='<uuid>' references and any id baked into a catalog value.

normalise_database_map

normalise_database_map(database_map) -> Dict[str, Dict[str, str]]

Validate a migration database_map and return {kind: {SRC_DB_UP or '*': dst_db}}.

Keys are object kinds (:data:MAPPABLE_KINDS); a value is either a database name (every object of that kind goes there) or {source_db: target_db} ('*' = any other source database). The feature store's own tables and views (FS_T_*, catalogs) are not migrated objects and cannot be mapped: their database is the target store itself.

mapped_database

mapped_database(kind: str, src_db: str, database_map) -> Optional[str]

The target database database_map (normalised) gives an object, or None.

build_placement_map

build_placement_map(objects, source_desc, primary_schema, primary_view_schema, renames=None, database_map=None) -> Dict[Tuple[str, str], Tuple[str, str]]

Build the source→primary placement map for a list of classified objects.

Parameters

objects : iterable of dict Each {'database','object','object_type'} where object_type is one of 'table' | 'view' | 'dataset' | 'process_view' | 'intermediate_view' | 'source_table' (anything containing 'table' lands in the table schema, everything else in the view schema). source_desc : dict The read-only descriptor {'schema','schema_view',...} of the source store (used only as the default source DB when an object's DB is blank). primary_schema : str Primary table database. primary_view_schema : str Primary view database (view_schema()). renames : dict, optional {(SRC_DB_UP, SRC_OBJ_UP): NEW_OBJ_NAME} for objects renamed to avoid a conflict. The destination DB is still chosen by object type. database_map : dict, optional Output of :func:normalise_database_map: the target database per kind (and source database), over the defaults above.

Returns

dict {(SRC_DB_UP, SRC_OBJ_UP): (DST_DB, DST_OBJ)}.