Hybrid date × partition filter¶
Iterate one (date, region) combination at a time. The hybrid FilterManager
view exposes both BUSINESS_DATE (the time column) and the partition
column.
from tdfs4ds.utils.filter_management import FilterManager
from tdfs4ds.utils.lineage import crystallize_view
source_data = tdml.DataFrame(tdml.in_schema('source_db', 'TRANSACTIONS'))
# 1. Build the hybrid manager (5 most-recent combinations for validation)
fm = FilterManager(table_name='FS_FILTER_REGION_DATE', schema_name=tdfs4ds.SCHEMA)
fm.load_filter(
tdml.DataFrame.from_query("""
SELECT REGION, TX_DATE
FROM (
SELECT DISTINCT REGION, TX_DATE
FROM source_db.TRANSACTIONS
QUALIFY ROW_NUMBER() OVER (ORDER BY TX_DATE DESC, REGION) <= 5
) t
"""),
time_column='TX_DATE',
)
print('Combinations:', fm.nb_filters)
# 2. Feature view uses scalar subqueries on both filter columns
df_features_raw = tdml.DataFrame.from_query(f"""
SELECT A.*
FROM source_db.TRANSACTIONS A
WHERE A.TX_DATE = (SELECT BUSINESS_DATE FROM {fm.schema_name}.{fm.view_name})
AND A.REGION = (SELECT REGION FROM {fm.schema_name}.{fm.view_name})
""")
df_features_proc = crystallize_view(
df_features_raw.groupby('CUSTOMER_ID').agg({'AMOUNT': ['sum', 'mean']}),
view_name = 'FEAT_ENG_TX_HYBRID',
schema_name = tdfs4ds.SCHEMA,
)
# 3. Register with the FilterManager attached — iterates automatically
tdfs4ds.upload_features(
df_features_proc,
entity_id = ['CUSTOMER_ID'],
feature_names = ['sum_AMOUNT', 'mean_AMOUNT'],
filtermanager = fm,
)
Why scalar subqueries: the filter view always exposes exactly one row, so the Teradata optimiser resolves the partition value before scanning the source table. This is what unlocks partition elimination at the storage level.