tdfs4ds.genai.attached_objects¶
Views and tables attached to the feature store (0.3.5.2).
A view joining a registered dataset with a source table, or a table brought in by hand,
is read by no process: the agent saw it "outside the feature store" and a full
rebuild_lineage_graph() closed the edges an extra_views= call had recorded.
attach_objects declares such objects part of the feature store's documented
perimeter:
- Marker: the reserved business-dictionary section
ATTACHEDon the object (JSON: kind, optional data domain, fingerprint, documentation time). No new table; every section reader skips it (deep_tables.RESERVED_SECTION_NAMES). - Lineage: an attached view's edges are recorded at once and every full rebuild seeds it, like a process view or a dataset.
- Documentation: a column the view passes through unchanged takes the description of
the column it comes from (PROPAGATED); computed columns and the object description go
to the LLM. A table is documented by its owner (
set_object_documentation). - Refresh:
refresh_attached_objectscompares the fingerprint (normalised view text and one hash per column, or the table's column types) and re-documents only what changed; an object gone from the database is detached, its documentation ended (still readable at a past date).
Parts written by a person (IMPORTED / MANUAL provenance) are never overwritten.
Wrapper around the global logger that only emits logs when
tdfs4ds.DISPLAY_LOGS is True. level is a string like "info", "error", etc.
Escape value for use inside a single-quoted SQL string literal.
For the DBC probes and catalog predicates that compare a name as data
(WHERE DatabaseName = '…'). The caller supplies the surrounding quotes.
{(DB, OBJ): 'V' | 'T'} of the attached objects (the agent's scope). Fail-open: {}.
The attached objects: DATABASE_NAME, OBJECT_NAME, KIND ('V' / 'T'), DATA_DOMAIN,
DOCUMENTED_AT (optionally of one data domain).
[(db, view)] of the attached views (the lineage rebuild seeds them).
End the ATTACHED marker. The documentation stays; the lineage edges of a view
are closed by the next full rebuild_lineage_graph (it is no longer seeded).
The relations a view reads and, for each output column, its source. Pure.
Returns {"ok": True, "view": [db, name], "sources": [[db, obj], ...], "columns":
{view column: [db, obj, column]}, "star": [[db, obj], ...], "items": {view column: select
item text}, "computed": [view columns]} or {"ok": False, "reason"}. A column is
a source column when its select item is a plain column of one named relation (renamed
or not); anything else is computed. A set operation at the top level makes every
column computed. Several relations are fine (unlike parse_exposed_view).
{"text", "columns": {column: hash}, "sources": [...]} of a view. Pure.
text hashes the normalised view text (process_sql.normalise_sql, the package's
one SQL normaliser); each column hashes its select item and its source.
{"text", "columns": {column: type}} of a table. Pure.
{"same", "added", "changed", "removed", "sources_changed"}. Pure.
render_marker(kind: str, data_domain: Optional[str], fingerprint: dict, documented_at: Optional[str]) -> str
propagation_rows(view: Tuple[str, str], info: dict, source_docs: Dict[Tuple[str, str, str], str], view_columns: Optional[List[str]] = None, only: Optional[Iterable[str]] = None) -> List[dict]
BD_COLUMNS rows for the view's pass-through columns. Pure.
source_docs: {(DB, OBJ, COLUMN) upper: description} of the source columns. A
column listed by a * of a source takes the description of the same-named column
of that source (only for a column of view_columns). only: restrict to these
view columns (upper).
'V' (view) / 'T' (table) / None (absent or not visible).
{(DB, OBJ): 'process' | 'dataset'} of the registered process views and datasets.
Column descriptions of the source objects, one BD_COLUMNS read. A process view's or dataset's columns are its features: their text is there too.
Whether a source object's documentation (columns or description) was written after since.
End the BD_COLUMNS rows of columns (removed from the view), except protected ones.
End the object's BD rows (a disappeared object), protected parts aside.
Object description and computed columns by the LLM (_document_intermediate_view
with the view's sources as upstream context). Returns a status text; never raises.
_document_view(db, obj, ddl, info, report, *, llm: bool, only_columns=None, author=None, event_id=None)
Propagate the pass-through columns (always), the LLM part when llm.
attach_objects(objects, data_domain: Optional[str] = None, *, document: bool = True, refresh_lineage: bool = True, refresh_index: bool = True, author: Optional[str] = None, dry_run: bool = False) -> List[dict]
Attach views and tables to the feature store's documented perimeter.
objects: 'DB.NAME' or a list. data_domain: optional; an object with one shows
in that domain's summaries. A view: its lineage edges are recorded (and kept by every
full rebuild_lineage_graph), its pass-through columns take their sources'
documentation, and with document the LLM writes its description and computed
columns. A table: registered as it is (document it with set_object_documentation).
A process view, a dataset or an FS_* object is refused (already in the feature
store); so is a name not found in the database. Returns one report per object.
refresh_attached_objects(objects=None, *, dry_run: bool = True, document: bool = True, refresh_index: bool = True, author: Optional[str] = None) -> List[dict]
Re-document the attached objects that changed; detach the ones that disappeared.
States: unchanged (skipped, no write, no LLM); sources_changed (a source's
documentation changed: pass-through columns copied again, no LLM); changed (the
view text or the table columns changed: added / changed columns re-documented, removed
ones ended, the LLM pass only when a computed column or the sources changed, lineage
re-recorded); broken (exists but no longer compiles: reported, nothing written);
disappeared (gone: marker ended, lineage closed, documentation ended - still
readable at a past date -, a person's parts kept and reported). dry_run=True
(default) only classifies.
Views of databases that read a registered dataset (or process view) and are not
attached. Read only: VIEW, READS, ATTACHED. Matching on whole qualified names.