tdfs4ds.serve.doc_coverage¶
Documentation coverage: what is documented, what is missing, what is in the vector index.
Backs the admin app's Documentation section (Coverage and Objects panels).
Three layers, like efs_map.py:
collect_doc_inventory()-- the only function that reads the database or the vector index (one read per catalog / business-dictionary table / collection);build_doc_model(...)-- pure: one row per object, plus the overview;render_*-- scoped CSS + HTML, no<script>.
Objects counted: process views and their feature columns, datasets (description
and sections), intermediate views and source tables of the lineage graph. The
feature store's own technical objects (FS_T_* storage tables, _HIDDEN
tables) are left out, with the rule documentation itself uses.
Vector index status:
- Chroma -- exact. The expected documents come from the builders
build_vector_indexuses (vector_index.chroma_index_status), so a row reportedindexedis one the next build would skip. - Teradata Vector Store -- inferred. No per-row membership can be read safely
(the embeddings are keyed by
TD_ID, a row position recomputed on refresh). A row counts as indexed when its collection isreadyand the row was documented (ValidStart) before the collection'slast_updated; a row documented after it ispending.
module-attribute
¶
DATASET_SECTIONS = ['OVERVIEW', 'ENTITY', 'FEATURE_THEMES', 'BUSINESS_QUESTIONS', 'INTENDED_AUDIENCE']
module-attribute
¶
module-attribute
¶
KIND_LABELS = {'process': 'Processes', 'feature': 'Features (process outputs)', 'dataset': 'Datasets', 'intermediate_view': 'Intermediate views', 'source_table': 'Source tables'}
module-attribute
¶
KIND_SINGULAR = {'process': 'Process', 'feature': 'Feature', 'dataset': 'Dataset', 'intermediate_view': 'Intermediate view', 'source_table': 'Source table'}
module-attribute
¶
_INDEX_ORDER = ['collection_failed', 'missing', 'stale', 'pending', 'building', 'unknown', 'indexed_inferred', 'indexed']
module-attribute
¶
INDEX_LABELS = {'indexed': 'indexed', 'indexed_inferred': 'indexed (inferred)', 'stale': 'stale (text changed)', 'pending': 'pending (documented after the last build)', 'missing': 'not indexed', 'collection_failed': 'collection failed', 'building': 'collection building', 'unknown': 'unknown', 'n/a': 'nothing to index'}
module-attribute
¶
_UPLOAD_HINT = 'tdfs4ds never generates documentation for source tables: write it in Objects > Edit, with tdfs4ds.set_object_documentation(database, name, description=..., columns={...}), or load an existing data dictionary with tdfs4ds.upload_business_dictionary_objects (and _columns).'
module-attribute
¶
_CAUSE_TEXT = {'definition': 'replaced', 'upstream definition': 'reads {o}, replaced', 'upstream documentation': 'documentation of {o} changed'}
module-attribute
¶
_CSS = '<style>\n.doc-cov{--navy:#00233C;--orange:#FF5F00;--ok:#16a34a;--warn:#d97706;--fail:#dc2626;\n --muted:#64748b;--line:#E2E8F0;--chip:#F1F3F6;color:#1e293b;font-size:14px}\n.doc-cov h3{color:var(--navy);font-weight:600;margin:18px 0 8px;font-size:16px}\n.doc-cov .doc-kpis{display:grid;grid-template-columns:repeat(auto-fit,minmax(190px,1fr));gap:12px}\n.doc-cov .doc-kpi{border:1px solid var(--line);border-radius:10px;padding:12px 14px;background:#fff;min-width:0}\n.doc-cov .doc-kpi .lbl{font-size:12px;text-transform:uppercase;letter-spacing:.04em;color:var(--muted)}\n.doc-cov .doc-kpi .val{font-size:24px;font-weight:600;color:var(--navy);font-variant-numeric:tabular-nums}\n.doc-cov .doc-kpi .sub{font-size:12px;color:var(--muted)}\n.doc-cov .doc-bar{display:flex;height:8px;border-radius:4px;overflow:hidden;background:var(--chip);margin:8px 0 4px}\n.doc-cov .doc-bar span{display:block;height:100%}\n.doc-cov .b-ok{background:var(--ok)}.doc-cov .b-warn{background:var(--warn)}.doc-cov .b-fail{background:var(--fail)}\n.doc-cov .doc-chip{display:inline-block;padding:1px 8px;border-radius:999px;font-size:12px;\n border:1px solid var(--line);background:var(--chip);color:#334155;white-space:nowrap}\n.doc-cov .doc-chip.ok{background:#ecfdf5;border-color:#a7f3d0;color:#065f46}\n.doc-cov .doc-chip.warn{background:#fffbeb;border-color:#fde68a;color:#92400e}\n.doc-cov .doc-chip.fail{background:#fef2f2;border-color:#fecaca;color:#991b1b}\n.doc-cov .doc-wrap{overflow-x:auto}\n.doc-cov table{border-collapse:collapse;width:100%;font-size:13px}\n.doc-cov th{background:var(--navy);color:#fff;text-align:left;padding:6px 8px;font-weight:500}\n.doc-cov td{border-bottom:1px solid var(--line);padding:5px 8px;vertical-align:top}\n.doc-cov tr:nth-child(even) td{background:#F7F9FC}\n.doc-cov .doc-link{color:var(--navy);text-decoration:underline;cursor:pointer;background:none;border:0;\n padding:0;font:inherit;text-align:left}\n.doc-cov .doc-note{color:var(--muted);font-size:12px;margin:6px 0}\n.doc-cov .num{text-align:right;font-variant-numeric:tabular-nums}\n</style>'
Keep the primary store's rows (see read_federation.primary_rows).
One business-dictionary view read per attached read-only store (0.3.3.15),
rows tagged FEATURE_STORE. Their views carry no ValidStart: the date
a row was documented is then unknown. A store whose view cannot be read is
skipped (logged): its objects show as undocumented rather than failing the
panel.
Read everything the model needs. Index reads are fail-open (reported, not raised).
{(DB, NAME): {'layout', 'physical'}} of the tables declared deep (few; DBC read per table).
(existing, definition_times) from one DBC.TablesV read.
existing: {(DB, NAME)} of the objects of the lineage databases, or None
if unknown. The lineage builder sometimes stores SQL tokens as objects (seen
live: 1, APPLY, POSITION); without this check they would be
reported as views missing documentation.
definition_times: {(DB, NAME): Timestamp} -- when each VIEW was last
replaced (LastAlterTimeStamp: moved by REPLACE VIEW, not by COMMENT ON,
same clock as CURRENT_TIMESTAMP; checked live on Vantage 20), or None if
unknown. Tables are left out: their stamp also moves on operations that do not
change what they mean.
Per source: {'collection', 'exists', 'status', 'last_updated', 'rows', 'orphans', 'error'}.
With several stores read (frames tagged FEATURE_STORE), the primary's
state stays at the top level and each read-only store's is under
'_stores' ({STORE: {source: state}}): on Chroma each store has its
own collections. On the Teradata backend a read-only store's index state is
not read (its collections live in its own database): 'unknown'.
The id a business-dictionary row has in the vector index (same convention everywhere).
Index status of one documented row (store: a read-only store's view
database, whose own index state is in index['_stores']).
build_doc_model(processes=None, datasets=None, graph=None, fs_tables=None, bd_objects=None, bd_sections=None, bd_columns=None, index=None, backend='chroma', schema=None, view_schema=None, existing_objects=None, catalog_features=None, deep_tables=None, stores=None, definition_times=None) -> dict
One row per object plus the overview. Pure: no database access.
existing_objects ({(DB, NAME)} from DBC, or None when unknown): lineage
nodes absent from it are not objects and are reported as phantom nodes.
catalog_features: the feature catalog, to reconcile the feature count
(process outputs here, (domain, entity, name) triplets in the catalog).
definition_times ({(DB, NAME): Timestamp} of views, or None when
unknown): documentation written before its object, or an object upstream,
was replaced is reported outdated (see :func:outdated_reasons).
{(DB, NAME): {(DB, NAME), ...}}: every object each lineage node reads,
transitively (edges go reader -> input). Manager views and tables are left
out: a TimeManager view is replaced at every step by design, and a manager's
_HIDDEN table is data, not definition.
outdated_reasons(key, documented_since, definition_times, upstream, doc_latest_of=None, upstream_docs=False, max_listed=3) -> list
Why documentation written at documented_since may no longer describe
key ((DB, NAME)). Pure.
- the object itself was replaced after it was documented;
- an object it reads (upstream:
{key: {upstream keys}}) was replaced after it was documented; - upstream_docs (a dataset: its documentation is propagated from its processes): the documentation of an upstream object changed after.
One entry per cause: {'cause', 'object', 'at'} (the latest objects first,
at most max_listed upstream ones, plus a count of the rest).
'replaced 2026-11-02 09:15 · reads DB.STG, replaced 2026-11-03 08:00'.
Set freshness (current / outdated / unknown), outdated (reasons,
text) and documented_since on each row, in place.
The comparison uses the OLDEST documented part: a feature description written before the view was replaced is out of date even if the object's description was edited later. Unknown when the documentation date is unknown (another store's rows carry none) or no definition date could be read (DBC refused).
'362 in the catalog · 16 without a current producer · ...' - why the process-output count differs from the catalog's. Same rule as the Report.
Objects whose documentation or index entry is not complete, or out of date.
Summary block above an object's documentation.