Skip to content

Register your first feature

This walkthrough takes you from raw data to a registered, persisted, time-travel ready feature in three steps.

1 · Write the feature engineering view

Prefer raw SQL — it gives you the full Teradata expressivity (PERCENTILE_CONT, window functions, partition predicates). Persist the view directly with REPLACE VIEW:

feature_sql = """
SELECT
    CUSTOMER_ID,
    COUNT(*)        AS nb_transactions,
    SUM(AMOUNT)     AS total_amount,
    AVG(AMOUNT)     AS avg_amount
FROM SOURCE_DB.TRANSACTIONS
GROUP BY CUSTOMER_ID
"""

tdml.execute_sql(f"REPLACE VIEW {tdfs4ds.SCHEMA}.FEAT_ENG_CUSTOMER AS {feature_sql}")
df_features = tdml.DataFrame(tdml.in_schema(tdfs4ds.SCHEMA, 'FEAT_ENG_CUSTOMER'))

If you must use the teradataml DataFrame API and it produces a temporary view, crystallize it first:

from tdfs4ds.utils.lineage import crystallize_view

df_features = crystallize_view(
    source_data.groupby('CUSTOMER_ID').agg({'AMOUNT': ['sum', 'mean']}),
    view_name   = 'FEAT_ENG_CUSTOMER',
    schema_name = tdfs4ds.SCHEMA,
)

2 · Register and ingest with upload_features

tdfs4ds.upload_features(
    df_features,
    entity_id     = ['CUSTOMER_ID'],        # the key column(s); types come from df
    feature_names = ['nb_transactions', 'total_amount', 'avg_amount'],
    metadata      = {'project': 'customer_analytics'},
    dataset_view_name = 'ds_customer_spend',          # optional — also builds a dataset
)

The single call performs eight operations:

  1. Registers the entity (CUSTOMER_ID) if not already present.
  2. Registers each feature in the feature catalog.
  3. Registers the process (the view + entity + feature list) in the process catalog.
  4. Records the source distribution.
  5. (Optional) simplify_query — if tdfs4ds.simplify_query is configured.
  6. (Optional) document_process_incremental — if an instruct model is configured and document=True (default).
  7. run(process_id) — ingests the feature values into the temporal storage tables.
  8. (Optional) Builds and documents ds_customer_spend.

entity_id is a list of column names

Pass the key column(s) as a list, e.g. ['CUSTOMER_ID'] or ['CUSTOMER_ID', 'EVENT_DATE']. The SQL types are read from the source, with their exact length and character set. A dict of types is accepted only to override an inferred type.

3 · Confirm registration

tdfs4ds.process_catalog()       # PROCESS_ID, VIEW_NAME, ENTITY_NAME, ...
tdfs4ds.feature_catalog()       # FEATURE_NAME, ENTITY_NAME, FEATURE_TABLE, PROCESS_ID, ...
tdfs4ds.dataset_catalog()       # VIEW_NAME, ENTITY_NAME, COMMENT, ...

To re-run the same logic later (e.g. nightly refresh), look up the process ID and call run:

from tdfs4ds.process_store.process_query_administration import get_process_id

process_id = get_process_id('"my_db"."FEAT_ENG_CUSTOMER"')
tdfs4ds.run(process_id=process_id)

What's next