tdfs4ds.removal¶
What a removed process or feature leaves behind, and its removal (0.3.3.12).
remove_process and remove_feature used to delete the catalog rows (and,
for a process, its documentation tables) and nothing else. Left behind were:
- the vector-store entries of the documentation (the agent could still find a removed process or feature by search);
- the documentation provenance rows;
- the stored SQL versions (
FS_PROCESS_SQL) and the FilterManager / TimeManager catalog rows of a process; - the lineage edges of a dropped process view.
The helpers here remove them. Every step is independent and non-fatal: a failure is logged and reported, never raised, so a partial cleanup never stops the removal itself.
Lineage. The lineage graph holds objects, not processes. A process view kept
after its process is removed is still a real view that other objects may read,
so its edges stay; they are removed when the view is gone (remove_process(...,
drop_view=True)). A feature is not a node of the graph: removing one leaves the
graph unchanged.
Vector store. Chroma: the entries of the object are deleted by id (the ids
build_vector_index gives them). Teradata Vector Store: a collection reads the
business-dictionary view live, but its embeddings are keyed by row position, so
deleted rows must be followed by a refresh (build_vector_index, which uses
update(data_refresh=True)); refresh_index=False skips it and says so.
Split db.obj into (database, object), both de-quoted.
Quote-aware, so a dot inside a quoted part does not split the name:
split_qualified('"DEINF.CARLOSN"."FS_X"') == ('DEINF.CARLOSN', 'FS_X').
The naive rsplit('.', 1) this replaces returns ('"DEINF.CARLOSN"',
'"FS_X"') only by luck and ('"DEINF', 'CARLOSN"."FS_X"') in the
split('.') variant.
An unqualified name returns ('', name).
Escape value for use inside a single-quoted SQL string literal.
For the DBC probes and catalog predicates that compare a name as data
(WHERE DatabaseName = '…'). The caller supplies the surrounding quotes.
The ids of source ('objects' / 'sections' / 'columns') that belong to the object, among ids. With columns, only those columns (columns collection).
Ids are DB__OBJ (objects), DB__OBJ__SECTION and DB__OBJ__COLUMN,
as build_vector_index writes them; compared case-insensitively.
remove_from_vector_index(database: str, object_name: str, *, columns=None, refresh_index: bool = True) -> dict
Remove an object's documentation (or only some of its columns) from the
vector index. Never raises. Returns {"backend", "removed" | "refreshed", ...}.
remove_process_leftovers(process_id: str, view_name: Optional[str], *, purge: bool = True, refresh_index: bool = True) -> dict
Everything a process leaves outside its catalog and documentation tables.
Called by remove_process after the documentation is deleted and before the
catalog rows are. Never raises; returns a report.
{"dataset_id", "database", "name"} of a registered dataset of the
primary store, by DATASET_ID, db.name or bare name; None when unknown.
Current business-dictionary rows of an object, per table.
remove_dataset(dataset: str, *, drop_view: bool = False, purge: bool = False, dry_run: bool = True, refresh_index: bool = True, force: bool = False) -> dict
Remove a dataset from the catalog, with what it leaves behind (0.3.3.19).
dataset: its DATASET_ID, database.name or name. Removes the catalog,
entity, feature, build and window rows and closes its lineage edges
(drop_dataset), then its business-dictionary rows (description,
sections, columns), their provenance and its vector-index entries. The view
itself is kept unless drop_view=True.
dry_run=True (default) changes nothing and returns what would go, with the
processes whose view reads it (dependents). Only a dataset of the primary
store can be removed. A dataset that process views read is not removed unless
force=True (0.3.3.27): tdfs4ds.dependencies.DependentObjectsError
lists them in the order they must be removed. Every step after the catalog is
non-fatal and reported; the call raises only when the dataset is unknown or
ambiguous, or when the catalog rows cannot be removed.
[(PROCESS_ID, VIEW_NAME)] of the processes that compute the feature.
What remove_feature removes (0.3.3.23, pure).
A feature (data domain, entity, name) has ONE catalog entry, whatever the
number of processes computing it. Without process_id the whole feature goes
("full", the historical behaviour). With it, the catalog entry goes only
when no OTHER process computes the feature; otherwise only that process's part
goes ("process": its documentation of the feature), and the catalog entry
stays for the others.
A process_id absent from producer_ids (already removed, or never a producer) is not an error: the decision rests on the other producers.
remove_feature_leftovers(feature_name: str, entity_id, data_domain: str, feature_ids, *, refresh_index: bool = True, process_ids=None, producers=None) -> dict
The feature's documentation and index entries, for every process computing it.
feature_ids: the catalog FEATURE_IDs of the feature, read before its catalog
rows are deleted. process_ids: only these processes' documentation of the
feature (remove_feature(..., process_id=) on a shared feature, 0.3.3.23);
None = every producer. producers: [(PROCESS_ID, VIEW_NAME)] when the
caller already read them. Never raises; returns a report.