tdfs4ds.lineage.indexing¶

tdfs4ds.lineage.indexing ¶

_PARTITION_STOPWORDS `module-attribute` ¶

_PARTITION_STOPWORDS = {'range_n', 'case_n', 'columnar', 'case', 'between', 'and', 'or', 'not', 'is', 'null', 'no', 'range', 'in', 'like', 'exists', 'distinct', 'each', 'interval', 'day', 'month', 'year', 'from', 'to', 'every', 'cast', 'extract', 'coalesce', 'nullif', 'trim', 'substr', 'substring', 'current_date', 'current_timestamp', 'date', 'timestamp', 'integer', 'smallint', 'bigint', 'byteint', 'decimal', 'float', 'when', 'then', 'else', 'end', 'format', 'zone', 'as'}

_strip_sql_comments ¶

_strip_sql_comments(sql: str) -> str

Remove Teradata-style comments

/ ... / block comments
-- ... end-of-line comments

BUT do not treat comment markers inside single-quoted literals or double-quoted identifiers as comments.

_compress_whitespace ¶

_compress_whitespace(sql: str) -> str

_extract_parenthesized_list ¶

_extract_parenthesized_list(text: str, start_idx: int)

Given text and index pointing at an opening '(', return (content_inside_parens, index_after_closing_paren).

_split_top_level_commas ¶

_split_top_level_commas(expr: str) -> List[str]

Split by commas that are not inside parentheses.

_normalize_identifier ¶

_normalize_identifier(ident: str) -> str

_find_identifiers ¶

_find_identifiers(s: str) -> List[str]

_columns_from_chunk ¶

_columns_from_chunk(masked_chunk: str) -> List[str]

_parse_partition_elements ¶

_parse_partition_elements(partition_expr_masked: str) -> List[Dict[str, Any]]

Parse a (masked) PARTITION BY expression into ordered levels. Each level: {level, kind, columns, raw}

_partitioning_by_column ¶

_partitioning_by_column(levels: List[Dict[str, Any]]) -> Dict[str, List[Dict[str, Any]]]

_mask_single_quoted_literals_same_len ¶

_mask_single_quoted_literals_same_len(sql: str) -> str

Replace each single-quoted literal with spaces of the same length so

keywords inside literals can't be detected
string length stays identical (indexes still align)

Handles escaped quotes like 'It''s'.

_find_create_table_columns_block_end ¶

_find_create_table_columns_block_end(ddl_clean: str) -> int

Find index just after the closing ')' of the CREATE TABLE column-definition block. Robust: ignores parentheses inside single-quoted literals and double-quoted identifiers. Assumes ddl_clean is comment-stripped + whitespace-compressed.

analyze_teradata_ddl ¶

analyze_teradata_ddl(ddl: str) -> Dict[str, Any]

Analyse Teradata CREATE TABLE DDL and return

{ 'primary_index_columns': [...], 'partition_columns': [...], 'partitioning_levels': [...], 'partitioning_by_column': {...} }

Critical behavior: - Ignores keyword-like text inside single-quoted literals - Searches PRIMARY INDEX / PARTITION BY only in the table-options tail (after the column-definition block).

tdfs4ds.lineage.indexing¶