tdfs4ds.feature_store.value_types

tdfs4ds.feature_store.value_types

Which SQL types a feature value can have, and where each one can go (0.3.5.17).

A feature value keeps the type the process view computes. Measured on Vantage 20, every type goes through the ingestion's temporal MERGE, LOBs included; the limits are elsewhere:

• comparison: CLOB, BLOB, JSON and XML cannot be compared (Errors 5689, 3600), so an unchanged value cannot be recognised (every run would open a new period) and a history dataset cannot merge equal periods. They are refused at registration (:func:refused_types); CLOB/BLOB/JSON also at run time, as before. XML, refused at registration since 0.3.5.17, is only a warning at run time so that a process registered earlier keeps running.
• aggregation: the grouped dataset pivot and the lag/lead pivot read MAX(CASE ...), which Teradata refuses on periods (6976), user-defined types such as ST_GEOMETRY, XML, JSON (6945) and LOBs (5664). A dataset holding such a feature is built with the forms that do not aggregate (:func:fit_strategy): multi-join instead of grouped, one sub-query per offset column instead of the pivot.

Pure helpers, plus one reader (HELP COLUMN on the storage table's FEATURE_VALUE) with an injectable run_sql; every reader is fail-open.

_REFUSED_RE module-attribute

_REFUSED_RE = compile('\\b(CLOB|BLOB|JSON|XML)\\b', IGNORECASE)

RUN_REFUSED_SUBSTRINGS module-attribute

RUN_REFUSED_SUBSTRINGS = ('clob', 'blob', 'json')

NOT_AGGREGATABLE_CODES module-attribute

NOT_AGGREGATABLE_CODES = frozenset({'PD', 'PT', 'PZ', 'PS', 'PM', 'UT', 'XM', 'JN', 'CO', 'BO', 'A1', 'AN'})

refused_types

refused_types(types: Dict[str, str]) -> Dict[str, str]

{feature: type} of the feature types refused at registration.

refusal_message

refusal_message(bad: Dict[str, str]) -> str

Why the registration is refused, and what to do instead.

run_time_refused

run_time_refused(types: Dict[str, str]) -> Dict[str, str]

The run-time refusal, unchanged since before 0.3.5.17 (CLOB, BLOB, JSON).

run_time_warned

run_time_warned(types: Dict[str, str]) -> Dict[str, str]

Types the run lets through with a warning: XML (refused at registration only).

_default_run_sql

_default_run_sql(sql: str)

value_type_code

value_type_code(table: str, run_sql: Optional[Callable[[str], tuple]] = None) -> Optional[str]

The HELP COLUMN type code of <table>.FEATURE_VALUE ('I', 'PM', 'UT'...), or None when it cannot be read. table is the storage table as written in SQL.

not_aggregatable

not_aggregatable(specs: Iterable, read_code: Optional[Callable[[str], Optional[str]]] = None) -> List[str]

Aliases of the dataset columns whose stored type MAX() refuses. One read per distinct storage table; an unreadable type is not flagged (fail-open).

fit_strategy

fit_strategy(specs, grouped: bool, offset_pivot: bool, caller: str, read_code: Optional[Callable[[str], Optional[str]]] = None)

(grouped, offset_pivot) that the dataset's value types allow.

Nothing is read unless the requested form aggregates (grouped, or offsets read with the pivot). When a feature's type cannot be aggregated, the dataset is built multi-join and/or with one sub-query per offset column, and a warning names the features.