{# Spec 151 - per-table layout recommendation detail. Renders: - Header (table name, region, size, window cost) - Current vs proposed layout (side-by-side) - Ranked candidate columns (table) - Apply SQL (copy panel) - Rollback SQL (copy panel) - Caveats list Empty state (no recommendation in latest scan) renders a single explanation card with a "Re-run scan" link instead of 404ing. #} {% extends "layout.html" %} {% block content %} {% from "components/_page_header.html" import back_link, header_card %}
{{ back_link("/recommendations/layout", "Partition & Cluster") }} {% set _path_parts = (table_full_name or '').split('.') %} {% set _dataset = _path_parts[1] if _path_parts | length >= 2 else '' %} {% set _table = _path_parts[2] if _path_parts | length >= 3 else (table_full_name or '') %} {% set _title %}{% if _dataset %}{{ _dataset }}.{% endif %}{{ _table }}{% endset %} {{ header_card(_title, caption="Partition & Cluster recommendation") }} {% if not recommendation %}

No recommendation in the latest scan for this table. The most common reasons:

  • The table is below 10 GB - clustering's benefit doesn't outweigh metadata cost.
  • Consumer queries don't filter / join / group on enough common columns to give a confident signal.
  • The table is already optimally laid out for the workload we see.

Recommendations are derived from the workload the latest scan observed. Run another scan if you suspect this changed recently.

{% else %} {# Three-card stat strip - reframed to answer "is this worth doing?": 1. Table size - magnitude check (is this a big table?) 2. Cost in scan window - what we'd potentially reduce 3. Affected downstreams - count of consumer models with active issues (the actionable signal - "Contributing jobs" was just evidence density and not user-meaningful) #} {% set _impacted_count = (lineage_payload.nodes | rejectattr('kind', 'equalto', 'target') | list | length) if lineage_payload else 0 %} {% from "components/_stat_card.html" import stat_card %}
{{ stat_card("Table size", recommendation.table_total_bytes | format_bytes) }} {{ stat_card( "Cost in scan window", recommendation.table_total_cost_in_window_usd | format_usd, meta="across " ~ recommendation.contributing_job_count ~ " consumer job" ~ ("s" if recommendation.contributing_job_count != 1 else "") ) }} {% if _impacted_count > 0 %} {{ stat_card( "Affected downstreams", _impacted_count, meta="model" ~ ("s" if _impacted_count != 1 else "") ~ " with active issues", accent="rose" ) }} {% else %} {{ stat_card("Affected downstreams", 0, meta="no flagged downstreams - preventative", accent="emerald") }} {% endif %}
{# US2 - BigQuery second-opinion panel. Renders when a LayoutBqSecondOpinion row exists for this recommendation; otherwise renders a banner explaining the IAM grant. #} {% if bq_second_opinion %}

BigQuery's recommendation (second opinion)

{% if bq_second_opinion.agrees_with_audit_top_pick %} Agreement on top column {% else %} Different top pick {% endif %}

BigQuery picks

{% if bq_second_opinion.bq_recommended_columns %}
{% for col in bq_second_opinion.bq_recommended_columns %} {{ col }} {% endfor %}
{% else %}

No specific columns surfaced

{% endif %}

Methodology: BigQuery's ML-based recommender - slot-time + scan-pattern weighting on the recent workload.

Audit picks

{% if recommendation.recommended_partition_column %} {{ recommendation.recommended_partition_column }} {% endif %} {% for c in cluster_candidates %} {{ c.column_name }} {% endfor %}

Methodology: cost-weighted filter / join / group-by frequency across {{ recommendation.contributing_job_count }} cached queries.

{% if bq_second_opinion.bq_description %}

{{ bq_second_opinion.bq_description }}

{% endif %}
{% else %}

BigQuery second opinion

Add roles/recommender.bigqueryPartitionClusterViewer to the principal that runs the scan to surface BigQuery's own recommendation alongside the audit's. The page works without it — this enables a second-opinion panel that increases confidence when both methodologies agree.

{% endif %} {# Why this recommendation - rationale panel. Explains the reasoning behind the picked partition / cluster columns and how a layout change ripples into the downstream issues shown in the lineage below. Replaces the deleted "Ranked candidates" table - same evidence, surfaced in plain English. #} {% set _part_pick = partition_candidate %} {% set _cluster_pick = (cluster_candidates | first) if cluster_candidates else None %} {% set _consumers = (lineage_payload.nodes | rejectattr('kind', 'equalto', 'target') | list) if lineage_payload else [] %}

Why this recommendation

Picked columns

    {% if _part_pick %}
  • PARTITION BY {{ _part_pick.column_name }} filter frequency {{ "%.0f%%" | format(_part_pick.cost_share * 100) }} - score {{ "%.2f" | format(_part_pick.score) }}
  • {% endif %} {% for c in cluster_candidates %}
  • CLUSTER #{{ c.cluster_rank }} {{ c.column_name }} {{ c.usage_type | replace('_', ' ') }} frequency {{ "%.0f%%" | format(c.cost_share * 100) }} - score {{ "%.2f" | format(c.score) }}
  • {% endfor %} {% if not _part_pick and not cluster_candidates %}
  • No picks - workload signal was below the partition / cluster gates.
  • {% endif %}

How it helps the downstream issues

{% if _consumers | length > 0 %} {{ _consumers | length }} downstream model{{ 's' if _consumers | length != 1 else '' }} reading from this table carries active issues today. {% if _cluster_pick %} Clustering by {{ _cluster_pick.column_name }} reorders the table's micro-files so reads filtered on it scan fewer bytes - the same scans that drive slot-contention and shuffle-spill on those downstreams. The lineage below names each impacted model. {% elif _part_pick %} Partitioning by {{ _part_pick.column_name }} lets BigQuery prune partitions at read time, reducing the byte volume that drives slot-contention and shuffle-spill on those downstreams. {% endif %} {% else %} No downstream models with active issues today. The recommendation is preventative - it locks in efficient layout before issues appear at scale. {% endif %}

Pick methodology: cost-weighted filter / join / group-by frequency across {{ recommendation.contributing_job_count }} cached consumer job{{ 's' if recommendation.contributing_job_count != 1 else '' }}, intersected with the table's schema. Columns must clear the materiality (≥ 1 GB), partition (DATE / TIMESTAMP, 70 % filter), and cluster (50 % combined) gates.

{# Downstream lineage - pure HTML/SVG dbt-style chip layout. Target chip on the left, every consumer model with active issues stacked on the right; curved SVG connectors drawn between them. Pure CSS + tiny inline SVG so it always renders (the previous ECharts attempt failed silently on the user's data). #}

Downstream lineage (impacted models)

Consumer models with active issues that read from this table. Click a chip to open its opportunity detail.

this table issue
{% if _consumers | length == 0 %}
No downstream models with active issues today. Models reading this table are either clean or only carry SQL-rewrite suggestions.
{% else %} {# Cap visible at 12 so the lineage stays a sensible height even when a base table fans out to 30+ downstreams. Top rows are picked by issue_count, then cost. #} {% set _max_visible = 12 %} {% set _sorted_consumers = _consumers | sort(attribute='total_cost_usd', reverse=true) | sort(attribute='issue_count', reverse=true) %} {% set _visible = _sorted_consumers[:_max_visible] %} {% set _hidden_count = (_consumers | length) - (_visible | length) %} {# ECharts ``tree`` series renders a clean dbt-style left-to- right hierarchical layout for free: chip-shaped nodes, smooth bezier edges, automatic spacing. We pre-compute the tree payload server-side and pass it via ``data-payload``; the script below feeds it straight into ECharts. #}
{% if _hidden_count > 0 %}

+ {{ _hidden_count }} more impacted model{{ 's' if _hidden_count != 1 else '' }} - view all

{% endif %} {% endif %}
{# Current / Proposed layout panel removed - the picks are already surfaced in the "Picked columns" sub-panel of the rationale, and the exact ALTER SQL is in the Apply panel below. Two restatements were redundant. #} {# Apply / Rollback SQL panels. #} {% from "components/_copy_pre.html" import copy_pre %} {% if recommendation.apply_sql %} {{ copy_pre("apply-sql", "Apply", recommendation.apply_sql) }} {% endif %} {% if recommendation.rollback_sql %} {{ copy_pre("rollback-sql", "Rollback", recommendation.rollback_sql) }} {% endif %} {# Caveats - use render_caveats (route-computed) so per-request warnings like PARTITION_CARDINALITY_EXCEEDED show alongside the persisted caveats. #} {% set _caveats = render_caveats if render_caveats is defined else recommendation.caveats %} {% if _caveats %}

Caveats

    {% for caveat in _caveats %}
  • {{ caveat.code }} {{ caveat.message }}
  • {% endfor %}
{% endif %} {% endif %}
{% endblock %}