SDR-2026-0425-DV-MESSY-PROD-WEB

Production Web Analytics

Grade F
Overall 51%
Components 487 evaluated
Adapter CJA · 3.5.17
Rubric strict@1.0
Generated Apr 25 2026 · 09:14 UTC
Summary

This implementation graded F (51%). The grader evaluated 487 components in this data view using the strict@1.0 rubric pack. The largest gap is in calc metric maint (11%); 4 of 5 rules in that category fired.

Category scores

schema hygiene
71% C−
naming consistency
100% A
segment complexity
60% D−
calc metric maint
11% F
attribution coverage
100% A
governance posture
17% F

Top remediations

  1. Designate one canonical metric per concept, document its choices, and deprecate the others by tagging them with a sunset date. CALC-014 +5 pts overall
  2. Establish a snapshot directory in version control and schedule regular runs of cja_auto_sdr / aa_auto_sdr to populate it. GOV-001 +5 pts overall
  3. Generate the SDR via cja_auto_sdr / aa_auto_sdr and commit it alongside this snapshot. Suppress this rule via .sdr-grader.yaml if the SDR lives somewhere the grader cannot see. GOV-003 +5 pts overall
  4. Re-create the missing component, or update the segment / calculated metric to point at the current canonical equivalent. SCH-002 +5 pts overall
  5. Update the formula to reference the current canonical component or retire the calculated metric. CALC-002 +3 pts overall

All findings

CALC-014 High calc metric maint

1 near-duplicate calculated metric group

1 group of calculated metrics have references overlapping at or above the Jaccard similarity threshold (0.85). Near-duplicates produce subtly different numbers in different reports.

calculatedMetrics/cm_rev_per_visit_final, calculatedMetrics/cm_rev_per_visit_v2, calculatedMetrics/cm_rev_visit_linear, calculatedMetrics/cm_revenue_per_visit, calculatedMetrics/cm_revenue_visit_corrected, calculatedMetrics/cm_revpv_lasttouch, calculatedMetrics/cm_rpv_marketing

Designate one canonical metric per concept, document its choices, and deprecate the others by tagging them with a sunset date.

GOV-001 High governance posture

No snapshot history detected for this data view

No prior snapshots of this implementation were detected alongside the input. Without snapshot history, configuration changes cannot be diffed, drift cannot be detected, and audit trails are unavailable for compliance review.

Establish a snapshot directory in version control and schedule regular runs of cja_auto_sdr / aa_auto_sdr to populate it.

GOV-003 High governance posture

No SDR documentation detected

No Solution Design Reference (SDR) document was supplied alongside this snapshot. Without an SDR, definitions live only in the platform UI, which makes peer review and onboarding harder and obscures the “why” behind component choices.

Generate the SDR via cja_auto_sdr / aa_auto_sdr and commit it alongside this snapshot. Suppress this rule via .sdr-grader.yaml if the SDR lives somewhere the grader cannot see.

SCH-002 High schema hygiene

60 broken references (showing first 10 of 60)

60 references are broken — segments or calculated metrics point at component IDs that don't exist in this data view. Broken references are usually a symptom of components renamed or deleted without updating their consumers.

calc_metric calculatedMetrics/cm_revenue_per_visit -> missing metrics/revenue
calc_metric calculatedMetrics/cm_revenue_per_visit -> missing metrics/visits
calc_metric calculatedMetrics/cm_rev_per_visit_v2 -> missing metrics/revenue
calc_metric calculatedMetrics/cm_rev_per_visit_v2 -> missing metrics/visits
calc_metric calculatedMetrics/cm_revpv_lasttouch -> missing metrics/revenue
calc_metric calculatedMetrics/cm_revpv_lasttouch -> missing metrics/visits
calc_metric calculatedMetrics/cm_rev_visit_linear -> missing metrics/revenue
calc_metric calculatedMetrics/cm_rev_visit_linear -> missing metrics/visits
calc_metric calculatedMetrics/cm_revenue_visit_corrected -> missing metrics/revenue
calc_metric calculatedMetrics/cm_revenue_visit_corrected -> missing metrics/visits

Re-create the missing component, or update the segment / calculated metric to point at the current canonical equivalent.

CALC-002 Medium calc metric maint

60 broken calculated metric references

60 calculated metric references are broken — the formula points at components, segments, or other calculated metrics that don't exist in this snapshot.

calculatedMetrics/cm_revenue_per_visit -> metrics/revenue
calculatedMetrics/cm_revenue_per_visit -> metrics/visits
calculatedMetrics/cm_rev_per_visit_v2 -> metrics/revenue
calculatedMetrics/cm_rev_per_visit_v2 -> metrics/visits
calculatedMetrics/cm_revpv_lasttouch -> metrics/revenue
calculatedMetrics/cm_revpv_lasttouch -> metrics/visits
calculatedMetrics/cm_rev_visit_linear -> metrics/revenue
calculatedMetrics/cm_rev_visit_linear -> metrics/visits
calculatedMetrics/cm_revenue_visit_corrected -> metrics/revenue
calculatedMetrics/cm_revenue_visit_corrected -> metrics/visits
calculatedMetrics/cm_rpv_marketing -> metrics/revenue
calculatedMetrics/cm_rpv_marketing -> metrics/visits
calculatedMetrics/cm_rev_per_visit_final -> metrics/revenue
calculatedMetrics/cm_rev_per_visit_final -> metrics/visits
calculatedMetrics/cm_orders_per_visit -> metrics/orders
calculatedMetrics/cm_orders_per_visit -> metrics/visits
calculatedMetrics/cm_avg_order_value -> metrics/revenue
calculatedMetrics/cm_avg_order_value -> metrics/orders
calculatedMetrics/cm_bounce_rate_filtered -> metrics/bounces
calculatedMetrics/cm_bounce_rate_filtered -> metrics/visits
calculatedMetrics/cm_engagement_rate -> metrics/engaged
calculatedMetrics/cm_engagement_rate -> metrics/visits
calculatedMetrics/cm_signup_conversion -> metrics/signups
calculatedMetrics/cm_signup_conversion -> metrics/visits
calculatedMetrics/cm_paid_ratio -> metrics/paid_visits

Update the formula to reference the current canonical component or retire the calculated metric.

CALC-015 Medium calc metric maint

1 repeated formula text

1 formula text appears verbatim on more than one calculated metric. Identical formulas are a red flag for accidental copy-paste rather than intentional duplication.

'revenue / visits': calculatedMetrics/cm_rev_per_visit_final, calculatedMetrics/cm_rev_per_visit_v2, calculatedMetrics/cm_rev_visit_linear, calculatedMetrics/cm_revenue_per_visit, calculatedMetrics/cm_revenue_visit_corrected, calculatedMetrics/cm_revpv_lasttouch, calculatedMetrics/cm_rpv_marketing

Pick one canonical metric, redirect consumers, and archive the duplicates.

GOV-007 Medium governance posture

1 shared-but-unapproved calculated metric

1 calculated metric is shared with at least 5 users or projects but carry no approval signoff. Approval normally precedes wide distribution in a mature governance posture — the reverse pattern usually means the metric got around the SDR review process and is now powering stakeholder decisions without a vetted owner.

calculatedMetrics/cm_revenue_per_visit  name='Cm Revenue Per Visit'  shared_to=6  complexity=42

Trigger the approval workflow on the offending metrics, or revoke the wider sharing until the definition has been reviewed by the owner and the analytics governance lead. Document the approved definition in the SDR so future reviewers can audit the chain.

SCH-003 Medium schema hygiene

170 components lack descriptions

70 metrics and 100 dimensions in this data view have empty description fields. Descriptions are the primary way new analysts and AI agents understand what a component measures; missing descriptions force readers to infer intent from names alone, which is frequently wrong.

Metrics: 70 of 142 missing (49%). Dimensions: 100 of 203 missing (49%). The rubric threshold is 35%.

Generate the list of components missing descriptions via the data view API and populate them. Establish a CI check that fails new components added without descriptions.

SEG-002 Medium segment complexity

3 segments mix container types

3 segments mix container types (event / session / person) within a single definition. Mixed containers obscure intent and produce surprising population differences from segments that look superficially similar.

segments/seg_qualified_lead_v3  containers: event, session, person  depth: 8
segments/seg_high_intent_returning  containers: session, event  depth: 6
segments/seg_b2b_account_engaged  containers: event, session  depth: 6

Decompose the segment into composed sub-segments, one per container level, and combine them at the surface segment.

SEG-007 Medium segment complexity

3 segments exceed nesting threshold

3 segments exceed the nesting depth threshold of 5. Deep nesting makes intent illegible — small definitional changes have unpredictable population effects.

segments/seg_qualified_lead_v3  depth: 8  containers: event, session, person
segments/seg_high_intent_returning  depth: 6  containers: session, event
segments/seg_b2b_account_engaged  depth: 6  containers: event, session

Decompose the segment into composed sub-segments named for what each layer captures.

CALC-003 Low calc metric maint

30 high-complexity calc metrics

30 calculated metrics have a complexity score above the rubric threshold of 22. High-complexity formulas are hard to review and easy to miscalibrate.

calculatedMetrics/cm_amer_share  complexity=47
calculatedMetrics/cm_emea_share  complexity=46
calculatedMetrics/cm_apac_share  complexity=45
calculatedMetrics/cm_loyalty_revenue  complexity=44
calculatedMetrics/cm_revenue_b2b  complexity=43
calculatedMetrics/cm_revenue_per_visit  complexity=42
calculatedMetrics/cm_rev_per_visit_v2  complexity=42
calculatedMetrics/cm_revpv_lasttouch  complexity=42
calculatedMetrics/cm_rev_visit_linear  complexity=42
calculatedMetrics/cm_revenue_visit_corrected  complexity=42
calculatedMetrics/cm_rpv_marketing  complexity=42
calculatedMetrics/cm_rev_per_visit_final  complexity=42
calculatedMetrics/cm_returning_share  complexity=42
calculatedMetrics/cm_active_user_rate  complexity=41
calculatedMetrics/cm_pages_per_visit  complexity=40
calculatedMetrics/cm_help_resolution  complexity=39
calculatedMetrics/cm_promo_redemption  complexity=38
calculatedMetrics/cm_form_submit_rate  complexity=37
calculatedMetrics/cm_email_ctr  complexity=36
calculatedMetrics/cm_video_completion  complexity=35
calculatedMetrics/cm_search_to_view  complexity=34
calculatedMetrics/cm_cart_to_order  complexity=33
calculatedMetrics/cm_repeat_buyer_rate  complexity=32
calculatedMetrics/cm_organic_share  complexity=31
calculatedMetrics/cm_paid_ratio  complexity=30

Decompose the metric into intermediate calculated metrics whose names describe what each step measures.

GOV-005 Low governance posture

402 components lack tags

402 of 487 components (83%) have no tags. Tags are the cheapest way to keep schema browsing tractable as the data view grows; without them, finding components by purpose requires reading every name.

Adopt a small, documented tag taxonomy and apply it via the platform API.

GOV-008 Low governance posture

1 shared-but-unapproved segment

1 segment is shared with at least 3 users or projects but carry no approval signoff. Widely-shared audiences without a vetted definition fragment reporting consistency — different teams pull subtly different cohorts under the same name.

segments/seg_qualified_lead_v3  name='Seg Qualified Lead V3'  shared_to=4

Approve the segment, narrow its sharing scope, or — if the definition is intentionally provisional — rename it to make the draft status explicit so consumers know the audience may shift.

Methodology

This grade was produced by sdr-grader using the strict@1.0 rubric pack. The rubric encodes 30 rules across 6 active categories; 13 fired against this snapshot. Each rule contributes to a category subtotal weighted by severity (critical: 4, high: 3, medium: 2, low: 1). Category subtotals roll up to the overall score using the category weights defined in the rubric pack.

The grader is rule-based and deterministic — the same input always produces the same grade. Findings are auditable: every rule's source YAML is linked from its finding, and rules can be suppressed or reweighted via a project-level .sdr-grader.yaml.

Generated by sdr-grader v1.1.3 against snapshot 2026-04-25T09:14:00Z. Rubric: strict@1.0. Adapter: cja_auto_sdr@3.5.17. 487 components evaluated; 0 components skipped . Source: github.com/brian-a-au/sdr-grader.