Upload, optimise, document — in one go

upload_features accepts three AI toggles. When an instruct model is configured, you can register a feature engineering view, let the query optimizer try to rewrite it, replace the view if the score improves, and generate the documentation — all from a single call.

tdfs4ds.upload_features(
    df,
    entity_id     = ['CUSTOMER_ID'],
    feature_names = ['nb_transactions', 'total_amount', 'avg_amount'],
    metadata      = {'project': 'customer_analytics'},

    # AI add-ons (silently skipped when no instruct model is configured)
    optimize_query = True,    # run the 9-step optimiser
    update_view    = True,    # REPLACE VIEW with the winner when score improves
    document       = True,    # populate process docs + EXPLAIN score
    max_candidates = 5,       # override tdfs4ds.QUERY_OPTIMIZER_MAX_CANDIDATES

    # Optional dataset build + documentation
    dataset_view_name = 'ds_customer_spend',
)

What runs (in order):

1. Register entity, features, and process.
2. Persist register_process_view SQL (process catalog, data distribution, optional FilterManager).
3. skill_optimize_query — generate candidates, score, optionally REPLACE VIEW.
4. document_process_incremental — business logic + per-feature + EXPLAIN docs.
5. run(process_id) — ingest values via the (now-optimized) view.
6. If dataset_view_name set: build the dataset view, then document_dataset_incremental.

Reports for the optimiser land under {QUERY_OPTIMIZER_REPORT_DIR}/{DATA_DOMAIN}/. Failures in steps 3 / 4 / 6 are non-fatal — the ingestion result is never affected.