tdfs4ds.read_federation¶
Cross-store READ federation (v0.3.1.148).
Attach one or more OTHER feature stores to the current session in READ-ONLY
mode. upload_features / documentation / lineage writes still target only
tdfs4ds.SCHEMA; when read-only stores are attached, catalog /
business-dictionary / lineage reads become a UNION ALL across the primary
+ the read-only stores, and every unioned row is tagged with a synthetic
FEATURE_STORE provenance column naming the store it came from (primary first).
Empty tdfs4ds.SCHEMA_READONLY (the default) ⇒ every read funnel is
byte-identical to the historical single-store behaviour — this is the same opt-in
shape as the database_view / view_schema() split.
Design notes¶
- Every persisted catalog / BD / lineage row already carries its own absolute
database name (
FEATURE_DATABASE/DATABASE_NAME/PARENT/CHILD_DATABASE_NAME), so aUNION ALLof each store's public views composes cleanly with no SQL rewriting — a read-only store's rows describe that store's own physical tables, sobuild_datasetresolves them cross-store given SELECT grants. - A read-only store is described by
{'schema','schema_view','vector_prefix'}(see :func:tdfs4ds._ro_descriptor):schema_viewis where itsFS_V_*views live (what reads consult),schemais its table database (the vectortarget_database). A bare string'DB'is shorthand forschema == schema_view == 'DB'with the default vector prefix.
Public helpers¶
- :func:
extend_read_stores— attach one or more read-only stores. - :func:
detach_read_stores— detach named stores (or all). - :func:
list_read_stores— current read-only descriptors.
Internal¶
- :func:
catalog_union_sql— the sharedUNION ALLbuilder used by every federated read funnel. ReturnsNonewhen only the primary is attached, so the caller keeps its existing single-store read (byte-identical).
module-attribute
¶
CAPABILITIES = ('has_bd', 'has_lineage', 'has_followup', 'has_process_docs', 'has_process_explain', 'has_provenance', 'has_manager_registry')
Return name as a Teradata identifier, quoting it only if required.
Idempotent: an already-quoted name is returned unchanged rather than being
wrapped a second time. Embedded quotes are escaped as "".
Parameters¶
name : str
The raw object/database name.
force : bool, default False
Quote even when the name does not need it. Use only at the few sites
that have always emitted unconditional quotes and whose exact output is
already baked into persisted DDL (see
feature_store/feature_query_retrieval.py) — elsewhere forcing would
change the SQL of existing stores for no reason.
Build db.obj with each part quoted only as needed.
qualified('TDB', 'FS_X') == 'TDB.FS_X' (unchanged for ordinary names);
qualified('SVC@CORP', 'FS_X') == '"SVC@CORP".FS_X'. An empty/None
database yields the bare object name, so unqualified callers keep working.
force=True quotes both parts unconditionally — only for the sites that
have always emitted quotes and whose exact text is already persisted (dataset
view DDL, returned volatile-table names).
Split db.obj into (database, object), both de-quoted.
Quote-aware, so a dot inside a quoted part does not split the name:
split_qualified('"DEINF.CARLOSN"."FS_X"') == ('DEINF.CARLOSN', 'FS_X').
The naive rsplit('.', 1) this replaces returns ('"DEINF.CARLOSN"',
'"FS_X"') only by luck and ('"DEINF', 'CARLOSN"."FS_X"') in the
split('.') variant.
An unqualified name returns ('', name).
Escape value for use inside a single-quoted SQL string literal.
For the DBC probes and catalog predicates that compare a name as data
(WHERE DatabaseName = '…'). The caller supplies the surrounding quotes.
Column names of the PRIMARY store's view_const view (cached).
Used to project an explicit column list into every UNION ALL leg so the
union has matching degree/names and tolerates version skew across stores.
Upper-cased names of the JSON columns of the primary view_const (cached).
Teradata refuses a set operation over a JSON column (Error 3600 "Comparisons are not allowed for the type JSON"), even a UNION ALL; the process catalog has two (METADATA, ENTITY_NULL_SUBSTITUTE), so its federated read failed on every live system. Fail-open: an unreadable type list gives an empty set.
Build a UNION ALL over view_const across the federated view schemas.
Parameters¶
view_const : str
The public view name (e.g. FS_V_FEATURE_CATALOG) — the SAME name in
every store, qualified per leg by that store's view schema.
columns : list[str], optional
Explicit column list projected into every leg. When None it is
discovered once from the primary view (a lazy metadata read).
provenance_col : str
Name of the synthetic provenance column (default FEATURE_STORE).
schemas : list[str], optional
Override the schema list (defaults to tdfs4ds.read_view_schemas());
primarily for testing.
Returns¶
str | None
The UNION ALL SQL when ≥2 stores are federated, else None — a
None return tells the caller to keep its existing single-store read
(byte-identical default behaviour).
Return a teradataml DataFrame over view_const federated across the read
schemas, or None when only the primary is attached (so the caller keeps its
existing single-store read → byte-identical). Convenience wrapper over
:func:catalog_union_sql for the plain browse choke-points.
Keep the rows of the primary store in a frame read across stores.
A frame without the provenance column (a single store) is returned as it is. For what only the primary may do: run its processes, plan their execution, write their documentation.
View schemas (primary first) that support an OPTIONAL capability
('has_bd' / 'has_lineage'). The primary is always included; a read-only
store is included only when its attach-time probe set the flag True (a store
with features but no documentation/lineage is skipped so its missing view can't
black-hole the whole union). Fail-open: an unprobed/legacy entry counts as
capable.
Return the SELECT <cols> FROM ( <UNION ALL of store VIEWS> ) t prefix for
a federated business-dictionary / lineage read (the store views already apply
CURRENT VALIDTIME, so the caller drops its own outer prefix), or None
when only the primary supports the capability — then the caller keeps its
byte-identical single-store base-table read (with its own CURRENT VALIDTIME).
"( <UNION ALL of each capable store's view> ) alias", or None.
For a read that must cover every attached store offering capability
(CAPABILITIES): the primary first, each leg tagged FEATURE_STORE.
None when only the primary offers it, so the caller keeps its single-store
SQL byte-identical. columns None = the primary view's columns.
Return a derived-table FROM fragment "( <UNION ALL> ) alias" over
view_const across the federated schemas, or None when only the primary
is attached (caller keeps its single-store FROM → byte-identical).
Lets a domain-scoped <validtime> SELECT ... WHERE ... keep its outer
valid-time prefix while the source becomes a cross-store union of VIEWS (the
prefix applies at the statement level across the unioned temporal tables). The
FEATURE_STORE provenance column is available inside the derived table for
joins that must not cross store boundaries.
Return True if the read-only store at desc['schema_view'] exposes the
catalog views, is SELECT-able, and is column-compatible with the primary.
Logs a prescriptive error and returns False otherwise (fail-open: the caller
skips this store and keeps the good ones).
Set desc['has_bd'] / desc['has_lineage'] by cheaply probing whether
the store exposes the business-dictionary objects view and the lineage view.
A store that lacks them still federates its catalogs; only BD / lineage
federation skips it. Fail-open (unprobed ⇒ treated as capable).
Best-effort: if the descriptor's table schema was defaulted to its view
schema (bare-string attach) but the store actually uses a table/view split,
learn the real table DB from a DISTINCT FEATURE_DATABASE probe so the vector
target_database is correct. Fail-open — leaves desc unchanged on error.
Attach one or more feature stores to the session in READ-ONLY mode.
Parameters¶
databases : str | dict | list
A store (or list of stores). Each is either a string ('DB_B' ⇒
schema == schema_view == 'DB_B', default vector prefix) or a dict
descriptor ({'schema':..., 'schema_view':..., 'vector_prefix':...}) for
a table/view split or a custom vector prefix.
validate : bool
When True (default), each store is probed for the catalog views + SELECT
access + column compatibility; a store that fails is logged and skipped
(the good ones still attach).
Returns¶
list[dict] The full read-only descriptor list after the attach.
Detach named read-only stores (matched on schema or view schema,
case-insensitively), or all of them when databases is None.
Return the current read-only store descriptors (normalised dicts, including
the has_bd / has_lineage capability flags discovered at attach).