tdfs4ds.serve.efs_map¶
EFS map: every object of the feature store, metadata plane vs data plane.
For architects and DBAs. Three layers, so the model and the HTML can be tested without a database:
collect_efs_map/collect_table_feature_rows— the only functions that query. Catalogs are materialised once; the dictionary (DBC.TablesV,IndicesV,PartitioningConstraintsV,StatsV,TableSizeV) is read in batchedINlists; exact row counts are oneUNION ALLper 25 tables.build_efs_model/build_feature_rows_model— pure, plain dicts.render_efs_map/render_feature_rows— self-contained HTML (scoped CSS + inline SVG, no script), Teradata palette.
Metadata plane: the catalogs that describe, govern and drive the data —
feature, process and dataset catalogs, run history, the Filter/Time managers,
the business dictionary and the lineage graph. Each hidden table is shown with
its public FS_V_* view.
Data plane: the feature values. Storage tables FS_T_* (and their
FS_V_* companion views) grouped by entity — one table per data domain,
entity and value type — with the processes that write them and the datasets
that read them.
module-attribute
¶
_META_GROUPS = [('feature_catalog', 'Feature catalog', 'Every feature: its id, type, entity, data domain and the storage table that holds it.', ('FEATURE_CATALOG',)), ('process_catalog', 'Process catalog', 'Registered feature-engineering processes: source view, entity, features, primary index and partitioning of their storage.', ('PROCESS_CATALOG', 'DATA_DISTRIBUTION')), ('execution', 'Run history & run drivers', 'Every run with its status, the FilterManager / TimeManager objects that drive segmented and time-stepped runs, and the clock of each data domain.', ('FOLLOW_UP', 'FILTER_MANAGER', 'TIME_MANAGER', 'DATA_DOMAIN_SETTINGS')), ('dataset_catalog', 'Dataset catalog', 'Datasets (the serving views): entity keys, exposed features, build options.', ('{dataset}_',)), ('documentation', 'Business dictionary & documentation', 'Business descriptions of objects, sections and columns; generated process documentation.', ('BUSINESS_DICTIONARY', 'PROCESS_DOCUMENTATION')), ('entity_refresh', 'Entity refresh (admin)', 'When each process last confirmed each entity (track_entity_refresh): one FS_R_* table per domain, entity and primary index. Never a feature, in no catalog.', ('FS_R_',)), ('lineage', 'Lineage graph', 'Object-to-object dependency edges (sources → process views → datasets).', ('LINEAGE_GRAPH',))]
module-attribute
¶
_OTHER_GROUP = ('other', 'Other feature-store objects', 'Objects with the FS_ prefix not classified above.')
module-attribute
¶
_CSS = '\n<style>\n.efs-map{--navy:#00233C;--orange:#FF5F00;--slate:#5A6B7B;--chip:#F1F3F6;--border:#E2E8F0;\n --meta:#EEF2F8;--data:#FFF5EE;color:var(--navy);font-size:.9rem}\n.efs-map *{box-sizing:border-box}\n.efs-kpis{display:grid;grid-template-columns:repeat(auto-fit,minmax(140px,1fr));gap:10px;margin:4px 0 18px}\n.efs-kpi{background:#fff;border:1px solid var(--border);border-radius:10px;padding:10px 12px}\n.efs-kpi b{display:block;font-size:1.35rem;font-weight:600}\n.efs-kpi span{color:var(--slate);font-size:.78rem}\n.efs-kpi span.efs-kpi-sub{display:block;color:#B54708;font-size:.72rem;margin-top:2px}\n.efs-plane{border-radius:14px;padding:16px 16px 6px;margin-bottom:18px;border:1px solid var(--border)}\n.efs-plane.meta{background:var(--meta);border-left:6px solid var(--navy)}\n.efs-plane.data{background:var(--data);border-left:6px solid var(--orange)}\n.efs-plane>h2{margin:0 0 2px;font-size:1.15rem}\n.efs-plane>p.lead{margin:0 0 14px;color:var(--slate)}\n.efs-grid{display:grid;grid-template-columns:repeat(auto-fill,minmax(330px,1fr));gap:12px;margin-bottom:10px}\n.efs-box{background:#fff;border:1px solid var(--border);border-radius:10px;padding:12px}\n.efs-box h3{margin:0 0 2px;font-size:.98rem}\n.efs-box p{margin:0 0 8px;color:var(--slate);font-size:.8rem}\n.efs-obj{display:flex;gap:6px;align-items:center;flex-wrap:wrap;padding:5px 0;border-top:1px dashed var(--border);font-size:.8rem}\n.efs-obj code{font-size:.78rem;word-break:break-all}\n.efs-tag{display:inline-block;border-radius:4px;padding:0 6px;font-size:.7rem;font-weight:700;line-height:1.6}\n.efs-tag.t{background:var(--navy);color:#fff}.efs-tag.v{background:#fff;color:var(--navy);border:1px solid var(--navy)}\n.efs-tag.ty{background:var(--chip);color:var(--navy)}\n.efs-tag.warn{background:#dc2626;color:#fff}\n.efs-num{color:var(--slate);margin-left:auto;white-space:nowrap}\n.efs-entity{background:#fff;border:1px solid var(--border);border-radius:12px;margin-bottom:12px}\n.efs-entity>summary{cursor:pointer;padding:12px 14px;list-style:none;display:flex;flex-wrap:wrap;gap:8px 16px;align-items:baseline}\n.efs-entity>summary::-webkit-details-marker{display:none}\n.efs-entity>summary h3{margin:0;font-size:1rem}\n.efs-entity>summary .efs-num{margin-left:0}\n.efs-flow{display:grid;grid-template-columns:minmax(150px,1fr) minmax(320px,3fr) minmax(150px,1fr);gap:12px;padding:0 14px 14px;align-items:start}\n@media (max-width:900px){.efs-flow{grid-template-columns:1fr}}\n.efs-lane h4{margin:0 0 6px;font-size:.75rem;text-transform:uppercase;letter-spacing:.04em;color:var(--slate)}\n.efs-lane ul{margin:0;padding:0;list-style:none}\n.efs-lane li{background:var(--chip);border-radius:6px;padding:4px 8px;margin-bottom:4px;font-size:.78rem;word-break:break-all}\n.efs-tables{display:grid;grid-template-columns:repeat(auto-fill,minmax(260px,1fr));gap:10px}\n.efs-table{border:1px solid var(--border);border-top:4px solid var(--orange);border-radius:8px;padding:10px;background:#fff}\n.efs-table.missing{border-top-color:#dc2626}\n.efs-table h5{margin:0 0 6px;font-size:.82rem;word-break:break-all}\n.efs-metrics{display:flex;gap:6px;margin:6px 0}\n.efs-metric{flex:1;background:var(--chip);border-radius:6px;padding:4px 6px;text-align:center}\n.efs-metric b{display:block;font-size:1.05rem}\n.efs-metric span{font-size:.68rem;color:var(--slate)}\n.efs-bar{height:6px;background:var(--chip);border-radius:3px;overflow:hidden;margin:4px 0 6px}\n.efs-bar i{display:block;height:100%;background:var(--orange)}\n.efs-kv{font-size:.74rem;color:var(--slate);margin:2px 0;word-break:break-word}\n.efs-kv b{color:var(--navy);font-weight:600}\n.efs-btn{margin-top:6px;background:var(--navy);color:#fff;border:0;border-radius:6px;padding:5px 10px;font-size:.76rem;cursor:pointer}\n.efs-btn:hover{background:var(--orange)}\n.efs-arrow{text-align:center;color:var(--slate);font-size:.8rem;margin:-6px 0 12px}\n.efs-legend{font-size:.78rem;color:var(--slate);margin:6px 0 0}\n.efs-legend i{display:inline-block;width:10px;height:10px;border-radius:2px;margin:0 4px 0 10px;vertical-align:middle}\n</style>\n'
Wrapper around the global logger that only emits logs when
tdfs4ds.DISPLAY_LOGS is True. level is a string like "info", "error", etc.
Build db.obj with each part quoted only as needed.
qualified('TDB', 'FS_X') == 'TDB.FS_X' (unchanged for ordinary names);
qualified('SVC@CORP', 'FS_X') == '"SVC@CORP".FS_X'. An empty/None
database yields the bare object name, so unqualified callers keep working.
force=True quotes both parts unconditionally — only for the sites that
have always emitted quotes and whose exact text is already persisted (dataset
view DDL, returned volatile-table names).
Escape value for use inside a single-quoted SQL string literal.
For the DBC probes and catalog predicates that compare a name as data
(WHERE DatabaseName = '…'). The caller supplies the surrounding quotes.
Pair a hidden table with its public view: FS_V_X / X_HIDDEN → FS_X.
Group key of a metadata object, from its name.
The partitioning expression of DBC.PartitioningConstraintsV.ConstraintText.
The dictionary stores it wrapped: CHECK (/*01 02 00*/ RANGE_N(...) /*...*/
IS NOT NULL). Keep only RANGE_N(...) / CASE_N(...).
build_efs_model(feat_df, proc_df=None, ds_cat_df=None, ds_feat_df=None, *, storage_info=None, exact_counts=None, metadata_objects=None, process_tables=None, schema=None, view_schema=None, consistency=None, valid_time=None) -> dict
Assemble the map from materialised catalogs and dictionary facts.
storage_info: {(DB, TABLE): {exists, pi, partitioning, stats_rows,
perm_bytes, skew_pct}} (upper-cased keys). exact_counts: {(DB,
TABLE): (current, total)} or None. metadata_objects: [{database,
name, kind, perm_bytes, rows}]. process_tables: {PROCESS_ID:
{"DB.TABLE"}}. consistency: store_consistency.build_consistency_model
output (orphan IDs per table, unreferenced tables, KPI notes).
valid_time: the catalog valid-time label every count is read at.
build_feature_rows_model(database, table, current_rows, total_rows, feat_df=None, proc_df=None) -> dict
Rows per (feature, version) of one storage table.
current_rows / total_rows: [(FEATURE_ID, FEATURE_VERSION, n)]
from CURRENT / NONSEQUENCED VALIDTIME group-bys. A FEATURE_ID absent
from the catalog for this table is flagged orphan.
The db.table label of a UNION ALL count branch.
Cast, because a UNION's column type comes from its FIRST branch: an uncast literal is a CHAR of the first label's length, and every longer label is truncated and no longer matches its table.
{(DB, TABLE): (current, total)}, one UNION ALL per chunk; per table on error.
FS_ objects of the store's databases + the Filter/Time manager objects.
Read the store and build the map model (see build_efs_model).
The catalogs are read at as_of (None = current valid time) through the
shared snapshot; orphan_scan is the last storage scan, used when it was
taken at the same valid time.
Rows per feature (and version) of one storage table, current and history.
The Report's Time drivers section, inside the metadata plane's colours. Empty when no process exists or the model could not be built.