tdfs4ds.feature_store.entity_management¶
feature_store_table_creation(entity_id, feature_type, if_exists='fail', primary_index=None, partitioning='', table_tag=None)
Creates a table and a corresponding view for feature storage in a Teradata database schema, based on specified entity ID and feature type.
table_tag (optional, see tdfs4ds.feature_store.placement) creates the table the features placed under
that tag live in: the historical name with __<TAG> appended. Without it nothing changes.
Build db.obj with each part quoted only as needed.
qualified('TDB', 'FS_X') == 'TDB.FS_X' (unchanged for ordinary names);
qualified('SVC@CORP', 'FS_X') == '"SVC@CORP".FS_X'. An empty/None
database yields the bare object name, so unqualified callers keep working.
force=True quotes both parts unconditionally — only for the sites that
have always emitted quotes and whose exact text is already persisted (dataset
view DDL, returned volatile-table names).
Wrapper around the global logger that only emits logs when
tdfs4ds.DISPLAY_LOGS is True. level is a string like "info", "error", etc.
The (feature_type, tag) pairs whose storage table is not in the database.
One DBC read for all of them. Fail-open: when it cannot be read every pair is
returned and feature_store_table_creation checks each table itself, as before.
Registers an entity within the feature store by facilitating the creation of feature store tables across multiple data types.
This function leverages the feature_store_table_creation function to create feature store tables tailored for different data types: 'FLOAT', 'BIGINT', 'VARCHAR_UNICODE', and 'VARCHAR_LATIN'. By doing so, it ensures a comprehensive setup for the specified entity, allowing for a diverse range of feature data types to be stored efficiently.
Parameters: - entity_id (str): Unique identifier for the entity. This ID is utilized to construct table names and associate them with the corresponding entity within the feature store. - primary_index (list, optional): Specifies the columns to be used as the primary index for the tables. A well-chosen primary index can significantly enhance query performance. - partitioning (str, optional): Defines the partitioning strategy for the tables, which can help manage and optimize the storage and retrieval of large datasets.
Returns: tuple: Contains the names of the created feature store tables. This tuple includes names for tables designed to store 'FLOAT', 'BIGINT', 'VARCHAR_UNICODE', and 'VARCHAR_LATIN' feature types, in the order they were created.
Example Usage:
register_entity("customer123") ('customer123_float_table', 'customer123_bigint_table', 'customer123_varchar_unicode_table', 'customer123_varchar_latin_table')
Note:
- This function is integral to initializing the feature storage for a new entity, ensuring that data of various types can be accommodated.
- The function assumes that the feature_store_table_creation function handles the intricacies of table creation, including dealing with existing tables based on the 'if_exists' parameter and applying specified partitioning and indexing strategies.
Generate a dictionary mapping entity IDs to their respective data types in a given tdstone2 model.
This function analyzes the provided tdstone2 model object to determine the type of the model ('model scoring' or 'feature engineering'). Depending on the model type, it retrieves the list of entity IDs from the appropriate mapper attribute ('mapper_scoring' or 'mapper'). It then constructs a dictionary where each key is an entity ID, and its corresponding value is the data type of that entity ID, as specified in the model's mapper attributes ('types').
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
existing_model
|
object
|
A model object created using the tdstone2 package. This object should contain the necessary mapper attributes ('mapper_scoring' or 'mapper') with 'id_row', 'id_partition', and 'types'. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
dict |
A dictionary mapping entity IDs to their data types. Keys are entity IDs, and values are data types (e.g., 'BIGINT'). |
Raises:
| Type | Description |
|---|---|
TypeError
|
If the relevant ID attribute ('id_row' or 'id_partition') in the model is neither a list nor a single value. |
Note
- The function dynamically determines the type of the model (scoring or feature engineering) based on the presence of specific attributes.
- 'id_row' or 'id_partition' is converted to a list if it is not already one.
- This function assumes the model is correctly instantiated and the necessary attributes are properly defined.
Removes all database entities (tables and views) associated with a given entity name within a specific data domain, and then removes the entity's metadata from the feature catalog.
This function first constructs a SQL query to select distinct table and view names associated with the given entity name and data domain from the feature catalog. It then iterates over the list of tables and views, attempting to drop each one. Progress and status of these operations are tracked using a progress bar. If an error occurs during the drop operation, the error message is displayed in the progress bar description. Finally, the function constructs and executes another SQL query to delete the entity's metadata from the feature catalog.
Parameters:
- entity (str): The name of the entity to be removed.
- data_domain (str, optional): The data domain where the entity is located. If not provided,
the default data domain from tdfs4ds.DATA_DOMAIN is used.
The function utilizes the tdfs4ds.DEBUG_MODE flag to control the printing of SQL queries for debugging purposes. If
DEBUG_MODE is set to True, the queries for selecting and deleting entities are printed to the console.
Dependencies:
- Requires the tdfs4ds module for configuration constants like DATA_DOMAIN, SCHEMA, and FEATURE_CATALOG_NAME,
as well as the DEBUG_MODE flag.
- Requires the tdml module or object with an execute_sql method for database operations.
- Uses the tqdm module to display a progress bar for the operations.
Note: - This function assumes that the necessary permissions are in place for dropping tables and views, as well as deleting metadata entries from the feature catalog. - Exception handling within the dropping operations is limited to displaying the first line of any encountered error messages in the progress bar description. Further error handling or cleanup may be necessary depending on the specific requirements and database configuration.
- None. The primary effect of this function is to alter the state of the database by removing entities and their metadata, not to return data to the caller.