tdfs4ds.utils.lineage¶

tdfs4ds.utils.lineage ¶

quote_ident ¶

quote_ident(name, force=False)

Return name as a Teradata identifier, quoting it only if required.

Idempotent: an already-quoted name is returned unchanged rather than being wrapped a second time. Embedded quotes are escaped as "".

Parameters¶

name : str The raw object/database name. force : bool, default False Quote even when the name does not need it. Use only at the few sites that have always emitted unconditional quotes and whose exact output is already baked into persisted DDL (see feature_store/feature_query_retrieval.py) — elsewhere forcing would change the SQL of existing stores for no reason.

qualified ¶

qualified(database, object_name, force=False)

Build db.obj with each part quoted only as needed.

qualified('TDB', 'FS_X') == 'TDB.FS_X' (unchanged for ordinary names); qualified('SVC@CORP', 'FS_X') == '"SVC@CORP".FS_X'. An empty/None database yields the bare object name, so unqualified callers keep working.

force=True quotes both parts unconditionally — only for the sites that have always emitted quotes and whose exact text is already persisted (dataset view DDL, returned volatile-table names).

split_qualified ¶

split_qualified(name)

Split db.obj into (database, object), both de-quoted.

Quote-aware, so a dot inside a quoted part does not split the name: split_qualified('"DEINF.CARLOSN"."FS_X"') == ('DEINF.CARLOSN', 'FS_X'). The naive rsplit('.', 1) this replaces returns ('"DEINF.CARLOSN"', '"FS_X"') only by luck and ('"DEINF', 'CARLOSN"."FS_X"') in the split('.') variant.

An unqualified name returns ('', name).

logger_safe ¶

logger_safe(level, message, *args, **kwargs)

Wrapper around the global logger that only emits logs when tdfs4ds.DISPLAY_LOGS is True. level is a string like "info", "error", etc.

query_change_case ¶

query_change_case(query, case)

Change the case of alternate segments of a query string split by single quotes.

Parameters: query (str): The input query string to be processed. case (str): The case to change the segments to. Should be 'lower' or 'upper'.

Returns: str: The query string with alternate segments in the specified case.

Raises: ValueError: If 'case' is not 'lower' or 'upper'.

query_replace ¶

query_replace(query, word, substitute)

Replace occurrences of a specific word in alternating segments of a query string, delimited by single quotes.

Parameters: query (str): The input query string to be processed. word (str): The word to be replaced within the segments. substitute (str): The word to substitute in place of the 'word' parameter.

Returns: str: The query string with occurrences of 'word' replaced by 'substitute' in alternating segments.

Raises: ValueError: If either 'query', 'word', or 'substitute' is not a string.

_analyze_sql_query ¶

_analyze_sql_query(sql_query)

Analyzes a SQL query and extracts table and view names categorized as 'source' and 'target'.

This function takes a SQL query as input, removes comments from the query, and then identifies tables and views mentioned in various SQL components such as CREATE TABLE, INSERT INTO, CREATE VIEW, and SELECT statements. The identified table and view names are categorized into 'source' and 'target', representing the tables/views that are being referenced and the ones that are being created or inserted into.

Parameters:

Name	Type	Description	Default
`sql_query`	`str`	The SQL query to be analyzed.	required

Returns:

Name	Type	Description
`dict`		A dictionary containing two lists - 'source' and 'target', where 'source' contains the
		table and view names being referenced in the query, and 'target' contains the table and view names
		being created or inserted into in the query. The names are normalized with double quotes for
		consistency and may include schema references.

analyze_sql_query ¶

analyze_sql_query(sql_query, df=None, target=None, root_name='ml__', node_info=None)

Analyzes a SQL query and its relationships to target tables/views.

This function takes a SQL query as input and extracts source tables/views mentioned in the query. It also allows for recursively analyzing TeradataML (tdml) views to identify relationships and dependencies between the source tables/views and target tables/views. The analysis results are accumulated in a TeradataML DataFrame (df) and a list of node information (node_info) to capture target, columns, and query details.

Parameters:

Name	Type	Description	Default
`sql_query`	`str`	The SQL query to be analyzed.	required
`df`	`DataFrame`	An existing TeradataML DataFrame to append the analysis results to. Default is None.	`None`
`target`	`str`	The target table/view where the query is directed. Default is None.	`None`
`root_name`	`str`	A root name identifier for filtering TeradataML views. Default is 'ml__'.	`'ml__'`
`node_info`	`list`	A list of dictionaries containing node information. Default is None.	`None`

Returns:

Name	Type	Description
`tuple`		A tuple containing the analysis results - a TeradataML DataFrame (df) containing source and target
		table/view relationships, and a list of node information (node_info) capturing details about each node
		in the analysis.

Note

The 'target' parameter should be specified when analyzing queries directed at a specific table/view.
When analyzing TeradataML views, the function recursively extracts and analyzes the view's definition.

Example

To analyze a SQL query:

result_df, result_node_info = analyze_sql_query(sql_query)

To analyze a SQL query with a specific target table/view:

result_df, result_node_info = analyze_sql_query(sql_query, target='my_target_table')

To analyze a SQL query and append results to an existing TeradataML DataFrame:

existing_df = teradataml.DataFrame() result_df, result_node_info = analyze_sql_query(sql_query, df=existing_df)

crystallize_view ¶

crystallize_view(tddf, view_name, schema_name, output_view=True)

Crystallizes a TeradataML DataFrame into a database view.

This function takes a TeradataML DataFrame (tddf) and converts its SQL representation into a database view. It analyzes dependencies, creates sub-views if needed, and constructs the final view with the specified view_name and schema_name. The resulting view can optionally be returned as a TeradataML DataFrame.

Parameters:

Name	Type	Description	Default
`tddf`	`DataFrame`	The TeradataML DataFrame to be converted into a view.	required
`view_name`	`str`	The name of the main view to be created or replaced in the database.	required
`schema_name`	`str`	The schema where the view will be created.	required
`output_view`	`bool`	If `True`, returns a TeradataML DataFrame representing the created view. If `False`, the function performs the operation without returning a DataFrame. Default is `True`.	`True`

Returns:

Type	Description
	teradataml.DataFrame: A TeradataML DataFrame representation of the created or replaced view, if `output_view` is `True`. Otherwise, returns `None`.

Raises:

Type	Description
`Exception`	If there is an error during SQL execution for creating sub-views or the main view.

Notes

The tddf parameter must be a valid TeradataML DataFrame.
The function generates a dependency graph for the SQL query representing the input DataFrame. Sub-views are created or replaced with new names based on this graph.
Column names are replaced case-insensitively to ensure consistency in SQL queries.
The function replaces "CREATE VIEW" with "REPLACE VIEW" to handle updates.

Example

To crystallize a TeradataML DataFrame into a view:

result_df = crystallize_view(my_teradataml_df, 'my_view', 'my_schema')

generate_view_dependency_network ¶

generate_view_dependency_network(schema_name)

Generates a dependency network of views within a specified schema.

This function lists all views within the given schema, analyzes their SQL queries to identify dependencies, and constructs a dependency network represented as a pandas DataFrame. The DataFrame contains source and target view relationships, capturing how views in the schema are interconnected.

Parameters:

Name	Type	Description	Default
`schema_name`	`str`	The name of the schema containing the views for which the dependency network is generated.	required

Returns:

Type	Description
	pandas.DataFrame: A DataFrame representing the view dependency network with source and target view relationships.

Note

The 'schema_name' parameter should specify the name of the schema containing the views to be analyzed.
The resulting DataFrame shows how views in the schema depend on each other.

Example

To generate a view dependency network for a schema:

dependency_df = generate_view_dependency_network('my_schema')

generate_view_dependency_network_fs ¶

generate_view_dependency_network_fs(schema_name)

Generates a dependency network between views and features within a specified schema.

This function constructs a dependency network by analyzing relationships between views and features within the given schema. It performs a SQL query to identify dependencies and creates a pandas DataFrame representing the relationships between views (source) and features (target).

Parameters:

Name	Type	Description	Default
`schema_name`	`str`	The name of the schema containing the views and features for which the dependency	required

Returns:

Type	Description
	pandas.DataFrame: A DataFrame representing the dependency network between views (source) and features (target).

Note

The 'schema_name' parameter should specify the name of the schema containing the relevant views and features.
The resulting DataFrame shows how views within the schema depend on specific features.

Example

To generate a view-feature dependency network for a schema:

dependency_df = generate_view_dependency_network_fs('my_schema')

get_ddl ¶

get_ddl(view_name, schema_name, object_type='view')

Retrieves the Data Definition Language (DDL) for a view or table in a specified schema.

This function allows you to retrieve the DDL for a view or table within a specific schema. You can specify the object type ('view' or 'table') to determine whether to retrieve the DDL for a view or a table.

Parameters:

Name	Type	Description	Default
`view_name`	`str`	The name of the view or table for which to retrieve the DDL.	required
`schema_name`	`str`	The name of the schema containing the view or table.	required
`object_type`	`str`	The type of the object ('view' or 'table') for which to retrieve the DDL. Default is 'view'.	`'view'`

Returns:

Name	Type	Description
`str`		The DDL (Data Definition Language) statement for the specified view or table.

Raises:

Type	Description
`ValueError`	If the 'object_type' parameter is not recognized or is not 'view' or 'table'.

Note

The 'view_name' parameter should specify the name of the view or table.
The 'schema_name' parameter should specify the name of the schema containing the view or table.

Example

To retrieve the DDL for a view:

ddl_statement = get_ddl('my_view', 'my_schema', object_type='view')

To retrieve the DDL for a table:

ddl_statement = get_ddl('my_table', 'my_schema', object_type='table')

generate_process_report ¶

generate_process_report(format='html', output_file=None, collapsible=False, sort_by='view_name', theme_mode='light', company_name=None, company_logo_url=None)

Generate a process catalog report styled after the Teradata website.