tdfs4ds.genai.documentation_tracking¶
Who wrote each piece of documentation, and what was done to it (0.3.2.1).
Two optional tables, created when missing; the six documentation tables (business dictionary + process documentation) are never altered:
FS_DOCUMENTATION_PROVENANCE (VALIDTIME)
One row per human-authored or copied documentation part:
(DATABASE_NAME, OBJECT_NAME, PART, PART_NAME) + DOC_SOURCE
(IMPORTED | MANUAL | PROPAGATED), DB_USER (the database
USER, authoritative), EDITOR_NAME (declared by the person: the admin
app connects with one account for everybody) and the EVENT_ID of the log
row that set it. No row = written by the LLM, or before 0.3.2.1. The LLM
pipelines write no provenance, and keep every IMPORTED / MANUAL part
(:func:protected_parts).
FS_DOCUMENTATION_LOG (plain, append-only)
One row per documentation action, opened RUNNING and closed with its status
like FS_FOLLOW_UP, with RUN_ID / PROCESS_ID so the two join. It
is deliberately NOT the follow-up table: every follow-up reader treats a row
as a process execution (run() skips a computation that already has a
COMPLETED row at its valid time), and that table cannot take a column.
A part is keyed on the object it documents, with the business dictionary's
identity: a feature is (view db, process view, 'COLUMN', feature) - one
key for its two storage places - a process's business logic is
(view db, process view, 'BUSINESS_LOGIC', ''), an object's description
(db, object, 'DESCRIPTION', ''), a section ('SECTION', name).
Everything here is fail-open: a missing or unwritable table gives one warning and the documentation action itself still happens.
module-attribute
¶
module-attribute
¶
ACTIONS = ('MANUAL_EDIT', 'IMPORT', 'PROPAGATE', 'PROTECT', 'RELEASE', 'LLM_DOCUMENT', 'CASCADE', 'INDEX_REFRESH')
module-attribute
¶
Wrapper around the global logger that only emits logs when
tdfs4ds.DISPLAY_LOGS is True. level is a string like "info", "error", etc.
Escape value for use inside a single-quoted SQL string literal.
For the DBC probes and catalog predicates that compare a name as data
(WHERE DatabaseName = '…'). The caller supplies the surrounding quotes.
Forget what was learnt about the two tables (tests, or after setup()).
Within the block, LLM writers may replace IMPORTED / MANUAL parts.
Label the actions of the block (ORIGIN), and their declared editor.
The declared editor: author=, else the block's editor, else the OS user.
Create the provenance and log tables (and their views) when missing.
Never raises: a failure is remembered (no retry on reads) and reported once.
Returns {'provenance': bool, 'log': bool}.
connect() hook: create the tracking tables that are missing.
existing is the lower-cased object set connect() already read from
DBC; a table found there is only marked available (its view is refreshed,
REPLACE VIEW being idempotent), so an up-to-date store pays no CREATE.
Never raises.
Whether the provenance / log table can be used (checked once per process).
log_open(action: str, target_kind: str, *, database=None, obj=None, part=None, part_name=None, process_id=None, editor=None, detail=None, parent_event_id=None) -> Optional[str]
Insert a RUNNING row; returns its EVENT_ID, or None when not logged.
Open a log row for the block and close it COMPLETED or FAILED,
Yields a dict: event_id, detail (filled by the block and stored when
it closes) and status (overridable, e.g. 'COMPLETED,2 kept').
The documentation activity log as a teradataml DataFrame (sort it with .sort).
target: an object name ('DB.OBJ' or 'OBJ'); since: a timestamp
string; actions: a list of ACTION values.
Latest log rows of one object, as pandas (newest first). Empty when unavailable.
Current provenance rows (all, or of one object), as pandas.
{normalised key: row dict} of the current provenance rows.
{(PART, PART_NAME_UPPER)} of the IMPORTED / MANUAL parts of an object.
Empty when overwriting is allowed (overwriting_manual) or the table is
unavailable - LLM runs then behave as before 0.3.2.1.
record_provenance(keys: Iterable[Tuple[str, str, str, str]], source: str, editor: Optional[str] = None, event_id: Optional[str] = None) -> int
Mark parts as IMPORTED / MANUAL / PROPAGATED (one MERGE per 50 parts).
Returns the number of parts marked; 0 when the table is unavailable.
Close the provenance rows of these parts: the LLM may rewrite them again.