tdfs4ds.feature_store.storage_definition¶
Check a storage-table definition before anything depends on it (0.3.4.3).
A process's storage tables (FS_T_*) are created from the entity columns (types
inferred from the source), the primary index and the partitioning. The database is
the only authority on whether such a definition is valid: RANGE_N refuses a
DECIMAL column (Error 3732), a TIMESTAMP column with DATE bounds
(Error 5714), a column that is not in the table, and more.
Before 0.3.4.3 a refused definition surfaced only when the first table was created, after the process was registered; the error was logged and skipped, the features were registered anyway, and every later run failed on a table that never existed.
:func:check_storage_definition tries the same columns, primary index and
partitioning as a volatile table (no data, own session, no rights needed), drops it,
and returns None or the problem: the full database message plus what to change.
module-attribute
¶
_RANGE_N_TYPES = ('BYTEINT', 'SMALLINT', 'INTEGER', 'BIGINT', 'DATE', 'TIMESTAMP', 'CHAR', 'VARCHAR')
Bases: ValueError
The database refused the definition of a process's storage tables.
Wrapper around the global logger that only emits logs when
tdfs4ds.DISPLAY_LOGS is True. level is a string like "info", "error", etc.
Column list of a storage table: the text feature_store_table_creation writes.
The volatile table that stands for the storage tables of entity_id.
What to change, for a refused definition. message is kept whole by the caller.
check_storage_definition(entity_id: dict, primary_index=None, partitioning: str = '') -> Optional[str]
None when the database accepts the storage definition, else the problem.
Fail-open: when the probe cannot reach the database (no context, a lost connection), nothing is reported and table creation decides as before.