tdfs4ds.catalog¶
Read-only catalog accessor functions for the tdfs4ds feature store.
These functions let you browse what is registered in the feature store: features, processes, and datasets. For discovery of entity IDs and feature versions before calling build_dataset(), the lower-level helpers from feature_store.feature_query_retrieval are also re-exported here.
property
¶
True when the build table and its view exist (checked once, then cached).
property
¶
True when the window table and its view exist (checked once, then cached).
property
¶
True when the offsets table and its view exist (checked once, then cached).
__init__(schema_name=None, name='DATASET', existing_objects=None, schema_name_view=None, existing_view_objects=None)
Parameters¶
existing_objects : set, optional
A pre-computed set of lower-cased table/view names already present in
schema_name. When provided, the constructor skips its own
db_list_tables() call (saves one round-trip). Pass the set
returned by connect()'s DBC.TablesV query.
schema_name_view : str, optional
Database that holds the catalog VIEW objects. When None it defaults to
tdfs4ds.SCHEMA_VIEW if a dedicated view database is configured, else
to schema_name (co-located with the tables — byte-identical to the
historical single-database behaviour).
existing_view_objects : set, optional
Pre-computed set of lower-cased view names present in schema_name_view.
Only consulted when the view database differs from the table database.
Create whatever part of the catalog is missing. Idempotent.
Creates only the absent objects and leaves the existing ones alone, returning the list of names it created (empty when there was nothing to do). Creation order is preserved, so a table is always created before the view that reads it.
It used to refuse — logging an ERROR and raising — as soon as any
of the six objects existed, while _exists only reported True when
all six did. Those two predicates disagree on every partial catalog,
which left it permanently unrepairable: the constructor sees "not all
present", calls this, and this rejects "some present". Because the
constructor self-heals on every instantiation, that fired on ordinary
READ paths (dataset_catalog(), the agent's _resolve_dataset) and
on connect()/setup() — an ERROR-level log and an exception for a
condition the caller cannot act on. Self-healing is this constructor's
long-standing contract; it was simply incomplete.
Register (or update) dataset in the catalog. Returns its DATASET_ID.
Parameters¶
build : dict, optional
How the view was built — {'strategy', 'join_type', 'time'
(DatasetTime), 'source_dataset_id'} — written to the build table
when it exists. Omitted: the build table is left untouched.
dataset_id : str, optional
Force the DATASET_ID to write. Used by restore_dataset_catalog
when a dataset was registered several times and one id must be kept;
ordinary callers let it be looked up by name.
offsets : list of dict, optional
One row per column read at an offset (lags and leads), written to the
offsets table when it exists; [] closes the rows of an earlier
build. None (default) leaves the table untouched.
Build details (strategy, join type, temporal mode) — None when unavailable.
Offset columns (one row per lag / lead column) — None when unavailable.
Join it to get_dataset_features on (DATASET_ID, FEATURE_NAME): a
column without offset has no row.
History windows (one row per historical dataset) — None when unavailable.
Join it to the catalog on DATASET_ID with a LEFT OUTER JOIN: a
snapshot dataset has no row. WINDOW_START / WINDOW_END are the
bounds at query time (WINDOW_START NULL = from the earliest stored
value); the other columns are the definition.
Remove a dataset (catalog + entity + feature rows) by DATASET_ID.
Parameters¶
dataset_id : str
The DATASET_ID to remove.
purge : bool, optional
When False (default), a CURRENT VALIDTIME DELETE valid-time-closes
the currently-valid rows (temporal soft delete; history is retained).
When True, a NONSEQUENCED VALIDTIME DELETE physically removes every
temporal row for the dataset — use this to fully reset a dataset that
carries stale/closed history before re-registering it.
close_lineage : bool, optional
True (default): the dataset view's edges in FS_LINEAGE_GRAPH are
closed as well. False when the view itself stays (a duplicate
registration removed by restore_dataset_catalog).
Raises¶
ValueError
If no dataset with this dataset_id currently exists in the catalog.
Retrieve a list of unique entity names from a specified data domain.
This function executes a database query to extract distinct entity names from a feature catalog, filtered by the provided data domain. If no domain is specified, it defaults to a predefined data domain.
domain (str, optional): The data domain to filter the entity names. Defaults to None, in which case a predefined domain is used.
Returns: DataFrame: A pandas-like DataFrame containing the unique entity names.
Retrieves the names of available features for a given entity ID from a feature store.
This function constructs and executes a SQL query to select feature names from a specific catalog within a schema
for a given entity ID, optionally using a specific valid time if set in tdfs4ds.FEATURE_STORE_TIME. The function
supports querying features as of a certain timestamp or the current valid time if no timestamp is specified. It
also provides an option to display detailed information about the features.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
entity_id
|
str, list, or dict
|
The ID of the entity for which to retrieve available features. This can be a single ID (str), a list of IDs, or a dictionary of IDs where keys are entity names and values are their corresponding IDs. |
required |
display_details
|
bool
|
If True, prints detailed information about all features in the catalog. Defaults to False. |
False
|
Returns:
| Name | Type | Description |
|---|---|---|
list |
A list of feature names available for the specified entity ID. |
Note
tdfs4dsis assumed to be a configuration module or object containing settings for the feature store, includingFEATURE_STORE_TIME,SCHEMA,FEATURE_CATALOG_NAME,DATA_DOMAIN, andDEBUG_MODE.- The entity ID(s) are used to filter the features specific to the entity. If
entity_idis a dictionary, the keys are considered as entity names, and their values are IDs. If it's a list, it's assumed to be a list of entity names or IDs. For a single string, it is treated as a single entity name or ID. FEATURE_STORE_TIMEis used to determine the temporal context for querying the feature data. If it is None, the query defaults to the current valid time.- If
DEBUG_MODEis enabled intdfs4ds, the constructed query is printed to the console. - The function uses
tdml.DataFrame.from_queryto execute the SQL query andto_pandasto convert the result into a pandas DataFrame for further processing.
Retrieve version identifiers for one or more features belonging to a given entity.
The function queries the underlying metadata tables to find the process (i.e., feature‑version) records that match the supplied entity and feature names. It returns a mapping from each requested feature name to either:
- None – if no matching rows were found.
- A single UUID string – if exactly one matching row exists for the feature.
- A list of dictionaries – if more than one matching row is found; each
dictionary contains:
process_id– the UUID of the process that produced the version,process_view_name– the human‑readable view name associated with that process.
Parameters¶
entity_name : str | list[str] The name (or names) of the entity whose features we are querying. If a single string is supplied it is treated as a singleton list.
str | list[str]
One or more feature names to look up. Accepts a single string or an iterable of strings; if a single string is provided it is wrapped in a list internally.
str, optional
The data‑domain partition to filter on. If omitted the default
tdfs4ds.DATA_DOMAIN constant is used.
Returns¶
dict[str, str | None | list[dict]]
A dictionary keyed by feature name. Each value is either:
* None – no records were found for that feature.
* str – a single UUID string when exactly one row matched.
* list[dict] – multiple matches; each dict has keys
process_id and process_view_name.
Notes¶
- The query joins the feature catalog view with the process catalog (specifically the “feature split” view) on data domain, entity ID, and feature name.
- SQL string literals are escaped by doubling single quotes; this is a lightweight escape that suffices for the current use‑case.
- The function preserves insertion order of features in the returned dictionary (Python 3.7+ guarantees dict order).
- When
tdfs4ds.DEBUG_MODEis true, the generated SQL statement is printed to stdout – useful for troubleshooting.
Example¶
get_feature_versions('user', ['age', 'income'])
Retrieve a Teradata DataFrame of features registered in the feature store at the current (or time-travelled) valid time.
Parameters¶
all_data_domains : bool, optional If False (default), only features belonging to the current DATA_DOMAIN are returned. Set to True to return features from all data domains.
Retrieve a Teradata DataFrame of processes registered in the process store at the current (or time-travelled) valid time.
Parameters¶
all_data_domains : bool, optional If False (default), only processes belonging to the current DATA_DOMAIN are returned. Set to True to return processes from all data domains.
Retrieve a Teradata DataFrame of all datasets registered in the dataset catalog.
Cross-store federation (v0.3.1.148): when read-only stores are attached, browses
a UNION ALL of every store's dataset-catalog view (with a FEATURE_STORE
provenance column); otherwise byte-identical. Federating here — at the read-only
accessor — keeps DatasetCatalog's own write/registration logic primary-only.
Return the entity definition (columns and types) for the given dataset ID. If dataset_id is None, returns entity info for all registered datasets. Federated across read-only stores when attached (dataset IDs are globally unique UUIDs, so the dataset_id filter selects the owning store's rows).
Return the feature list for the given dataset ID. If dataset_id is None, returns feature info for all registered datasets. Federated across read-only stores when attached (see get_dataset_entity).
Return how each dataset was built: BUILD_STRATEGY (MULTIJOIN / GROUPED),
JOIN_TYPE, TIME_MODE (CURRENT / AS_OF / TIME_MANAGER), the
AS_OF_TIMESTAMP or the TimeManager's location, and SOURCE_DATASET_ID
for a dataset rebuilt from another one.
Primary store only. Returns None when the build table does not exist
(a store whose catalog was created before 0.3.1.216 and never reconnected by
a user allowed to create it). A dataset registered before that version has
no row until it is rebuilt or restore_dataset_catalog(dry_run=False) runs.
Return the window of each historical dataset (build_historical_dataset),
one row per dataset, from FS_V_<catalog>_WINDOW:
WINDOW_START/WINDOW_END: the bounds when queried (a sliding window moves);WINDOW_STARTNULL means from the earliest stored value;START_KIND(FIXED/END_MINUS_DURATION/EARLIEST) andEND_KIND(FIXED/START_PLUS_DURATION/NOW);- the definition:
START_TS,END_TS,DURATION_ISO,DELAY_ISO(plus months/seconds),SLIDINGand the readableWINDOW_TEXT.
A snapshot dataset has no row, so join it to the catalog with a LEFT OUTER
JOIN on DATASET_ID. Primary store only. Returns None when the
window table does not exist (needs one connect() by a user allowed to
create it).
Return the columns of each dataset that read a feature at an offset (lags and
leads, build_dataset(feature_offsets=...)), one row per column, from
FS_V_<catalog>_OFFSETS:
FEATURE_NAME(the dataset column, e.g.AMOUNT_LAG_1D),BASE_FEATURE_NAME,FEATURE_ID,FEATURE_VERSION(the process);OFFSET_DIRECTION(LAG/LEAD/NONEfor '0'),OFFSET_MONTHS/OFFSET_SECONDS(signed),OFFSET_ISO(-P1D),OFFSET_TEXT("1 day before");OFFSET_STRATEGY(PIVOT/PER_OFFSET) andLEAD_HORIZON(NOW: a lead is NULL until its instant is reached).
A column without offset has no row: join it to get_dataset_features with a
LEFT OUTER JOIN on (DATASET_ID, FEATURE_NAME). Primary store only. Returns
None when the table does not exist (needs one connect() by a user
allowed to create it).