tdfs4ds.genai.operators.byom

tdfs4ds.genai.operators.byom

BYOM scoring operators (ONNXPredict, PMMLPredict, H2OPredict, ...): see operator_context.

A BYOM call scores every row of an input with a model STORED IN A TABLE of the database (ON <input> AS InputTable ... ON <model table> AS ModelTable DIMENSION USING ...). The model table holds the model itself (a BLOB, never read here) and often metadata (model id, training date, comment, metrics) which tells what the model is and which one is in use. detect parses the call, collect reads that metadata, render writes the fixed text block.

FUNCTIONS module-attribute

FUNCTIONS = ('ONNXPredict', 'PMMLPredict', 'H2OPredict', 'DataRobotPredict', 'DataikuPredict', 'MLeapPredict', 'ONNXEmbeddings', 'ONNXSeq2Seq', 'ONNXClassification')

_FUNC_BY_UPPER module-attribute

_FUNC_BY_UPPER = {(upper()): f for f in FUNCTIONS}

MAX_ROWS module-attribute

MAX_ROWS = 20

MAX_VALUE_CHARS module-attribute

MAX_VALUE_CHARS = 300

MAX_LEVELS module-attribute

MAX_LEVELS = 3

RENDER_TEXT_CAP module-attribute

RENDER_TEXT_CAP = 600

RENDER_ROWS module-attribute

RENDER_ROWS = 10

_SKIP_TYPES module-attribute

_SKIP_TYPES = {'BO', 'CO', 'BF', 'BV', 'JN', 'XM', 'UT', 'A1', 'AN'}

_TIME_TYPES module-attribute

_TIME_TYPES = {'DA', 'TS', 'SZ'}

_NAME module-attribute

_NAME = '(?:"[^"]*"|[\\w$#@]+)'

_FUNC_RE module-attribute

_FUNC_RE = compile('(?<![\\w$#@])((?:' + _NAME + '\\s*\\.\\s*)?)(' + join(FUNCTIONS) + ')\\s*\\(', I)

_FROM_RE module-attribute

_FROM_RE = compile('\\bFROM\\s+(' + _NAME + '(?:\\s*\\.\\s*' + _NAME + ')?)', I)

_mask

_mask(sql: str) -> str

sql with comments blanked and string / quoted-identifier contents neutralised.

Same length as the input, so positions found on the mask slice the original. Quoted identifiers keep word characters (x) so name patterns still match them.

_matching_paren

_matching_paren(masked: str, open_pos: int) -> int

_top_level

_top_level(masked: str, start: int, end: int)

Yield (pos, char) of every character of masked[start:end] at paren depth 0.

_split_top_level

_split_top_level(masked: str, start: int, end: int, keywords: Tuple[str, ...]) -> List[Tuple[str, int, int]]

Cut masked[start:end] at depth-0 keywords (whole words): [(keyword, from, to)].

_clean

_clean(text: str) -> str

_unquote_value

_unquote_value(v: str) -> str

_parse_using

_parse_using(sql: str, masked: str, start: int, end: int) -> Dict[str, List[str]]

Name(v1, v2) Name2(v) -> {Name: [v1, v2], ...} (order kept).

_parse_source

_parse_source(sql: str, masked: str, start: int, end: int) -> dict

One ON clause body <object | (subquery)> AS <alias> [PARTITION BY ... | DIMENSION ...].

detect

detect(sql: str) -> list

One dict per BYOM call: function, qualifier, input / model source, USING parameters.

_run

_run(sql: str) -> Tuple[List[str], list]

(column names, rows) of sql through the active teradataml context.

_split

_split(name: str, default_db: Optional[str]) -> Tuple[str, str]

_fq

_fq(db: str, obj: str) -> str

_default_db

_default_db() -> Optional[str]

_table_kind

_table_kind(db: str, obj: str) -> Optional[str]

TableKind from DBC.TablesV ('T', 'O', 'V', ...), None when unknown (no right, no row).

_view_text

_view_text(db: str, obj: str) -> Tuple[Optional[str], Optional[str]]

(view text, None) for a view, (None, None) for a table, (None, refusal) when SHOW VIEW was refused.

_single_from_object

_single_from_object(view_sql: str) -> Optional[str]

_resolve_table

_resolve_table(name: str, note: List[str]) -> Tuple[str, str, list]

Follow views (SHOW VIEW, one FROM object) down to the table. -> (db, table, chain).

_read_table

_read_table(db: str, obj: str, where: Optional[str] = None) -> Tuple[List[str], List[dict], List[str]]

Non-LOB columns and at most MAX_ROWS rows of a table. -> (columns, rows, notes).

where: a simple selection (see simple_selection) read first, so the model the SQL picks is in the list even when the table holds more than MAX_ROWS models.

collect

collect(found: list) -> list

Add the model table's metadata to each item. Fail-open: a problem becomes a note.

_cap

_cap(text: Optional[str], n: int = RENDER_TEXT_CAP) -> str

_outputs

_outputs(function: str) -> str

simple_selection

simple_selection(where: Optional[str]) -> Dict[str, set]

{column: {values}} when where is only col = 'v' / col IN ('a','b') joined by AND, else {}.

render

render(items: list) -> str