EU AI Act · GDPR · DSA

Decision Lineage for AI Agents

From "Systems of Action" to "Systems of Governance" — extracted decisions, options, outcomes, and rationale queryable in BigQuery.

BigQuery Agent Analytics SDK · examples/decision_lineage_demo
Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo
Part 1 · Market context

The regulatory landscape just changed

Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

Why now — three regulations converging

EU AI Act

Regulation (EU) 2024/1689, in force 1 Aug 2024.
Most operator obligations apply from 2 Aug 2026; full rollout by 2 Aug 2027.

High-risk-system rules cover transparency, record-keeping, human oversight, post-market monitoring.

GDPR

Article 22 — protections around solely automated decisions with legal or similarly significant effects.

Access / transparency rights create the "meaningful information about logic" audit expectation.

Digital Services Act

Article 26 — online platforms presenting ads must disclose that an item is an ad, who paid for it, and the main targeting parameters.

The demo's ad-planning lineage gives teams the upstream evidence behind those disclosures.
Sources: EU AI Act Service Desk timeline · GDPR — Reg. 2016/679 · DSA — Reg. 2022/2065
Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

The threat — Article 99 penalty tiers

The AI Act sets fixed-amount maximums and turnover thresholds. The fine is the higher of the two for non-SMEs.

Article 99 paragraphWhat violates itCap (non-SME)
Art 99(3)Article 5 — prohibited AI practices (e.g. manipulation that exploits vulnerabilities, social scoring, biometric categorisation by protected attributes)€35M or 7% of worldwide annual turnover, whichever is higher
Art 99(4)Most other AI Act obligations — operators of high-risk systems, transparency, record-keeping, post-market monitoring (Arts 16, 22, 23, 24, 26, 31, 33, 34, 50)€15M or 3% of worldwide annual turnover, whichever is higher
Art 99(5)Supplying incorrect, incomplete, or misleading information to authorities€7.5M or 1% of worldwide annual turnover, whichever is higher
For an ad-tech buyer with €20B worldwide annual turnover, the practical maximum under Art 99(4) is €600M per finding (3%, not the €15M floor). For SMEs and start-ups, Art 99(6) caps at the lower of the two values.
Source: Reg. 2024/1689 official journal, Article 99
Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

The narrative shift

Yesterday — Systems of Action

AI agents that do tasks end-to-end:

  • Pick the audience for a campaign
  • Allocate the media budget
  • Choose the creative theme
  • Set the launch window

The agent does. We trust the output.

Today — Systems of Governance

The same agents, plus a queryable evidence layer:

  • The decisions extracted from the agent trace
  • The options and scores the trace exposes
  • The rationale attached to dropped options
  • Linked to the trace span that produced it

The agent acts. The graph proves.

The mandate isn't don't use agents — it's be able to show your work, on demand, in audit format.

Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo
Part 2 · Business value

What "Decision Lineage" gives you

Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

Decision Lineage, defined

Working definition
For any agent decision: the ability to retrieve what was chosen, what alternatives were considered, what scores or criteria were applied, why each alternative was rejected, and which trace span produced the decision — as a single BigQuery query.

Trust

The data exists at the moment the regulator asks. No reconstruction.

Transparency

Same answer for product, legal, and engineering — one source of truth.

Reproducibility

Re-running the audit query a year later returns the same evidence.
Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

What the demo lets you prove

Right to explanationGDPR Art 22 · AI Act Art 86
Bias monitoringAI Act Art 10 / 71
Human oversightAI Act Art 14
ReproducibilityAI Act Art 12 / 13

What this is

  • A queryable record of agent decisions plus alternatives plus reasoning
  • The audit substrate regulators ask for under each article above
  • Built from real agent traces by an open-source SDK

What this is not

  • Not a compliance certification — talk to your legal counsel for that
  • Not a replacement for a Data Protection Impact Assessment
  • Not a model-quality scorecard — it audits what the agent did, not whether it was right
Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo
Part 3 · Customer proof

A concrete pattern from real ad-tech buyers

Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

Real-world pattern — programmatic media-buyer

Example pattern (anonymised)

The pain

A major brand-side media-buyer running a multi-agent media-planning stack:

  • Internal review board needed evidence for campaign decisions (audience, placement, creative, schedule)
  • Compliance team owed regulators a quarterly bias-audit report on demographic targeting
  • An adjudicator asked "why was this audience excluded from this campaign?" — answer required digging through Slack threads + run logs

The shape of the fix

Decision Lineage on BigQuery:

  • Every agent invocation captured by the BQ AA Plugin
  • Decisions + alternatives + rationale extracted by AI.GENERATE
  • Property graph queryable by compliance + product without writing Python
  • Same query reused for the quarterly bias-audit and for one-off subpoena responses
Replace this slide with your customer's pain point and timeline once a reference customer is named — the demo plugs into any agent the BQ AA Plugin already covers.
Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

Customer voice (placeholder)

"Before this, every audit request was a fire drill across three teams. Now we hand the regulator a five-line GQL query and the answer is the same on every run. The compliance posture moved from defensible to queryable."
— Director, Audience Strategy at a major DSP [placeholder — swap with a real attributable quote before external use]
~3 weeksAudit response (before)
One queryAudit response (after)
5 articlesRegulatory hooks mapped
Low costBQ query over existing graph
Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo
Part 4 · Practical demo

What an auditor actually sees

Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

The auditor persona — five questions, live

A compliance reviewer, equipped with the BigQuery Conversational Analytics panel, asks five questions of one dataset:

  • Q1 — "Why did the agent pick this audience?"
  • Q2 — "Did demographic criteria ever cause a candidate to be dropped?"
  • Q3 — "Were any decisions committed below 0.7 confidence?"
  • Q4 — "Show me the full audit trail for the Adidas creative-theme decision."
  • Q5 — "Which decision categories reject candidates least decisively?"

Why this format

Ground the demo in the auditor experience, not the engineering pipeline. The reviewer starts with a business question; the system returns a reproducible evidence path that legal and engineering can inspect together.

The next slides show the natural-language prompt, the GQL pattern, and the live answer from the demo dataset.

Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

Q1 — Right to explanation

EU AI Act Art 86
GDPR Art 22

"For Nike Summer Run, what audience did the agent pick, and why did it reject the alternatives?"

GRAPH `<P>.<D>.rich_agent_context_graph`
MATCH (cr:CampaignRun)-[:CampaignDecision]->(dp:PlanningDecision)
      -[:WeighedOption]->(opt:DecisionOption)
WHERE cr.session_id = '<SESSION>'
  AND LOWER(dp.decision_type) LIKE '%audience%'
RETURN DISTINCT opt.status, opt.name, opt.score, opt.rejection_rationale
ORDER BY opt.status DESC, opt.score DESC;

Three rows. Selected: Serious Runners 18-35 @ 0.99. Dropped: Casual Runners 25-45 (lower purchase intent on high-performance footwear), Fitness Enthusiasts 18-35 (group too broad for running conversion). Each rationale extracted by AI.GENERATE from the LLM_RESPONSE trace text.

Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

Q2 — Bias / fairness audit

EU AI Act Art 10
EU AI Act Art 71

"Across our 2026 portfolio, did the agent ever reject a candidate based on age or demographic criteria?"

GRAPH `<P>.<D>.rich_agent_context_graph`
MATCH (dp:PlanningDecision)-[:WeighedOption]->(opt:DecisionOption)
WHERE opt.status = 'DROPPED'
  AND (LOWER(opt.rejection_rationale) LIKE '%age %'
       OR LOWER(opt.rejection_rationale) LIKE '%demographic%'
       OR LOWER(opt.rejection_rationale) LIKE '%youth%')
RETURN DISTINCT dp.decision_type, opt.name, opt.rejection_rationale;

Multiple matches. Examples (verbatim from the live extraction):

  • "Youth Track & Field (13-15) — outside specified 16-22 range, less focused on in-season purchase"
  • "Affluent Hikers (35-55) — significant age-range mismatch with target demo"

The graph surfaces specific rationales for human review — proxy or legitimate ad-targeting? The reviewer judges from data, not trust.

Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

Q3 — Human-oversight trigger

EU AI Act Art 14

"Did the agent ever commit a decision below 0.7 confidence? That should have triggered human review."

GRAPH `<P>.<D>.rich_agent_context_graph`
MATCH (dp:PlanningDecision)-[:WeighedOption]->(opt:DecisionOption)
WHERE opt.status = 'SELECTED' AND opt.score < 0.7
RETURN DISTINCT dp.session_id, dp.decision_type, opt.name, opt.score
ORDER BY opt.score ASC;
0
rows returned

The empty result is the audit artifact. "We ran the human-oversight predicate against the entire portfolio for this period. The trigger never fired." Tighten to 0.85 → instant new at-risk list, same query.

Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

Q4 + Q5 — Reproducibility and pattern audit

Q4 — Subpoena reproducibility

Art 12
Art 13
MATCH (step:AgentStep)-[:DecidedAt]->(dp:PlanningDecision)
      -[:WeighedOption]->(opt:DecisionOption)
WHERE dp.session_id='<S>'
  AND LOWER(dp.decision_type) LIKE '%creative%'
  AND step.event_type='LLM_RESPONSE'
RETURN dp.span_id, opt.status, opt.name, opt.score,
       opt.rejection_rationale;

3 rows. All point to one evidence_span_id → that span lives in agent_events with full content + timestamp + latency.

Q5 — Systemic pattern

Art 17
Art 60
MATCH (dp:PlanningDecision)-[:WeighedOption]->(opt:DecisionOption)
WHERE opt.status='DROPPED'
RETURN dp.decision_type,
       COUNT(opt) AS rejections,
       AVG(opt.score) AS avg_dropped_score
GROUP BY dp.decision_type
ORDER BY rejections DESC;

Audience Selection rejects with the lowest avg confidence (0.66) — the category most worth a fairness loop-back to Q2.

Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo
Part 5 · Technical architecture

How the evidence is built — every step concrete

Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

End-to-end pipeline

1
ADK agent
Gemini 2.5 Pro · 5 tools · system prompt requires 3-candidate enumeration
▶
2
BQ AA Plugin
InMemoryRunner(plugins=[bq_logging_plugin]) → spans land in agent_events
▶
3
SDK extraction
Two AI.GENERATE calls — biz nodes (MERGE) + decisions (load job)
▶
4
Property graph
Canonical 4-pillar + ads-domain rich layer queried via GQL

What runs locally (one-time setup, ~5–10 min)

./setup.sh does steps 1-4 end-to-end on a fresh GCP project: enables APIs, creates the dataset, runs the live agent, extracts decisions, emits the property-graph DDL.

What runs at audit time (seconds)

Every regulator question is a single GQL query against the property graph. No re-extraction. No agent rerun. The graph is the audit substrate.

Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

Step 1 — The ADK media-planner agent

agent/agent.py

  • google.adk.agents.Agent instance
  • Model: Gemini 2.5 Pro (regional, Vertex AI)
  • 5 decision-commit tools (one per category)
  • BQ AA Plugin attached to InMemoryRunner

agent/tools.py — five tools

Tool Decision category
select_audience audience selection
allocate_budget budget allocation
select_creative creative theme
define_channel_strategy channel strategy
schedule_launch launch scheduling

agent/prompts.py — system prompt

The prompt instructs the agent to, for each structured decision:

1. Name three candidate options
2. Score each on 0.0–1.0 (two decimals)
3. Mark exactly one SELECTED, the other two DROPPED
4. Give an explicit, specific rejection rationale for each dropped option
5. End with Decision: … then call the corresponding tool
The prompt structure is the contract that makes downstream extraction reliable. The LLM_RESPONSE text is what AI.GENERATE later parses.
Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

Step 2 — The 6 campaigns × 27 spans

campaigns.py — 6 briefs

Brand Campaign Budget
Nike Summer Run 2026 $360K
Nike Winter Trail 2026 $500K
Adidas Track Season 2026 $420K
Puma Soccer Cup 2026 $280K
Reebok CrossFit Open 2026 $340K
Lululemon Yoga Flow 2026 $250K

run_agent.py

Iterates briefs; one InMemoryRunner invocation per brief; awaits flush() + shutdown() on the plugin so all spans land before extraction starts; writes campaign_runs mapping (deterministic).

Per session — 27 plugin-recorded spans

Event type Count
INVOCATION_STARTING 1
AGENT_STARTING 1
USER_MESSAGE_RECEIVED 1
LLM_REQUEST / LLM_RESPONSE 5 + 5
TOOL_STARTING / TOOL_COMPLETED 5 + 5
HITL_CONFIRMATION_REQUEST / _COMPLETED 1 + 1
AGENT_COMPLETED 1
INVOCATION_COMPLETED 1

6 sessions × 27 spans = 162 TechNode rows. Each span carries span_id, parent_span_id, session_id, event_type, agent, timestamp, JSON content, latency_ms.

Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

Step 3a — AI.GENERATE extraction (BizNodes)

build_graph.py calls mgr.extract_biz_nodes(session_ids) which runs one MERGE statement against agent_events. The MERGE's USING clause invokes AI.GENERATE per row:

MERGE `<P>.<D>.extracted_biz_nodes` AS target
USING (
  SELECT base.span_id, base.session_id,
    JSON_EXTRACT_SCALAR(entity, '$.entity_type')  AS node_type,
    JSON_EXTRACT_SCALAR(entity, '$.entity_value') AS node_value,
    CAST(JSON_EXTRACT_SCALAR(entity, '$.confidence') AS FLOAT64) AS confidence
  FROM `<P>.<D>.agent_events` AS base,
  UNNEST(JSON_EXTRACT_ARRAY(REGEXP_REPLACE(REGEXP_REPLACE(
    AI.GENERATE('Extract business entities (Product, Audience, Channel, …) from this payload. Return JSON array of {entity_type, entity_value, confidence}.\n\nPayload:\n' || payload_text,
                endpoint => 'gemini-2.5-flash').result,
    r'^```(?:json)?\s*',''), r'\s*```$',''))) AS entity
  WHERE base.session_id IN UNNEST(@session_ids)
    AND base.event_type IN ('USER_MESSAGE_RECEIVED','LLM_RESPONSE','TOOL_COMPLETED','AGENT_COMPLETED')
) AS source
ON target.biz_node_id = source.biz_node_id
WHEN MATCHED THEN UPDATE SET …
WHEN NOT MATCHED BY TARGET THEN INSERT …
WHEN NOT MATCHED BY SOURCE AND target.session_id IN UNNEST(@session_ids) THEN DELETE

Per-session idempotent in one statement. No streaming-buffer pitfall. MERGE is the SDK's chosen pattern for the BizNode write path.

Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

Step 3b — AI.GENERATE extraction (Decisions + Candidates)

mgr.extract_decision_points(session_ids) runs a separate AI.GENERATE query whose prompt asks for structured decision data:

Identify decision points in this agent payload. A decision point is where
the agent evaluated multiple candidates and selected or rejected them.
For each decision, return decision_type, description, and all candidates
with name, score (0-1), status (SELECTED or DROPPED), and rejection_rationale
(null if selected, required reason if dropped).

The Python side parses each row's JSON, builds DecisionPoint + Candidate records, then store_decision_points(...):

1
Dedupe in Python
_dedupe_rows_by_key last-wins on decision_id / candidate_id
2
DELETE FROM ... WHERE session_id IN (...)
Per-session reseat — guards against re-running
3
load_table_from_json
Load job to managed storage; visible to the just-issued DELETE (no streaming-buffer trap)
Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

Step 3c — SQL-only edge derivation

After the node tables exist, three pure-SQL INSERT INTO statements build the edges (no AI.GENERATE):

-- Evaluated edge (BizNode ↔ TechNode lineage)
INSERT INTO context_cross_links (link_id, span_id, biz_node_id, link_type, …)
SELECT b.biz_node_id, b.span_id, b.biz_node_id, 'EVALUATED', …
FROM extracted_biz_nodes b WHERE b.session_id IN UNNEST(@session_ids);

-- MadeDecision edge (TechNode → DecisionPoint)
INSERT INTO made_decision_edges (edge_id, span_id, decision_id, …)
SELECT CONCAT(span_id, ':MADE_DECISION:', decision_id), span_id, decision_id, …
FROM decision_points WHERE session_id IN UNNEST(@session_ids);

-- CandidateEdge (DecisionPoint → CandidateNode, with edge_type)
INSERT INTO candidate_edges (edge_id, decision_id, candidate_id, edge_type, …)
SELECT …, CASE c.status WHEN 'SELECTED' THEN 'SELECTED_CANDIDATE'
                        ELSE 'DROPPED_CANDIDATE' END, …
FROM candidates c WHERE c.session_id IN UNNEST(@session_ids);

Three tables, one statement each, all per-session-scoped. The edge_type on candidate_edges is what powers Block 4's WHERE ce.edge_type = 'DROPPED_CANDIDATE' filter.

Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

Step 4 — The 7 SDK backing tables

Table Key Written by What it holds
agent_events span_id BQ AA Plugin Plugin-recorded spans (162 rows = 6 sessions × 27 spans)
extracted_biz_nodes biz_node_id SDK MERGE Business entities from trace text
context_cross_links link_id SDK DML Span ↔ BizNode references
decision_points decision_id SDK load job Extracted decisions from LLM_RESPONSE text
candidates candidate_id SDK load job Extracted options per decision: selected or dropped
made_decision_edges edge_id SDK DML Span → Decision lineage
candidate_edges edge_id SDK DML Decision → Candidate, with selected / dropped edge type

Every backing table has row_count == distinct_keys after the SDK fix landed in PR #99 — the property-graph KEY contract holds end-to-end.

Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

Step 5 — The rich-graph projection layer

build_rich_graph.py adds demo-only SQL projections so the BigQuery Studio Explorer reads in business language. No new AI calls in this step:

Derived table Built from Purpose
campaign_runs run_agent.py writes directly One row per agent invocation, joined to campaign metadata in campaigns.py
rich_agent_steps agent_events (DISTINCT) Deduped span projection — one row per span_id (TechNode is multi-event per span by design)
rich_decision_types decision_points Normalised decision categories (audience-selection, budget-allocation, …)
rich_candidate_statuses candidates Distinct OptionOutcome values (SELECTED, DROPPED)
rich_rejection_reasons candidates Distinct rejection-rationale strings as first-class nodes

Plus five edge projections wiring the new labels back to SDK-owned facts. Schema lives in rich_property_graph.gql.tpl and is deterministic across reruns.

Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

Step 6 — The 8 node labels (what each node means)

LabelSource tableKEYWhat it represents
CampaignRuncampaign_runssession_idOne agent invocation against one brief — the unit of audit
AgentSteprich_agent_stepsspan_idOne step the agent took (LLM call, tool invocation, HITL check)
MediaEntityextracted_biz_nodesbiz_node_idAn audience, channel, creative, budget unit, or campaign — extracted from trace text
PlanningDecisiondecision_pointsdecision_idA moment the agent committed to a choice between options
DecisionOptioncandidatescandidate_idOne option weighed at a planning decision (selected or dropped)
DecisionCategoryrich_decision_typesdecision_type_idNormalised decision category (audience selection, budget allocation, …)
OptionOutcomerich_candidate_statusesstatus_idSELECTED or DROPPED — the outcome of weighing
DropReasonrich_rejection_reasonsreason_idA distinct rejection rationale (deduplicated across the portfolio)
Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

Step 7 — The 9 edge labels (how the graph connects)

Edge labelSource → DestinationReads as
CampaignActivityCampaignRun → AgentStep"this run produced this step"
NextStepAgentStep → AgentStep (parent_span_id → span_id)"this step caused that step" (causal chain)
ConsideredEntityAgentStep → MediaEntity"this step touched this entity"
DecidedAtAgentStep → PlanningDecision"this step is where the decision committed"
CampaignDecisionCampaignRun → PlanningDecision"this run made this decision"
InCategoryPlanningDecision → DecisionCategory"this decision is an audience-selection / budget-allocation / …"
WeighedOptionPlanningDecision → DecisionOption"this decision considered this option"
HasOutcomeDecisionOption → OptionOutcome"this option was selected / dropped"
RejectedBecauseDecisionOption → DropReason"this option was dropped for this reason"
Read top to bottom in plain English: "this run produced this step → which decided at this planning decision → which weighed this option → which has this outcome → which was rejected because of this reason." Five edges, one query, full audit trail.
Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

Step 8 — How a GQL query actually traverses

The visualization GQL (the demo's Block 2):

GRAPH `<P>.<D>.rich_agent_context_graph`
MATCH p = (cr:CampaignRun)-[:CampaignDecision]->(dp:PlanningDecision)
          -[:WeighedOption]->(opt:DecisionOption)-[:HasOutcome]->(st:OptionOutcome)
WHERE cr.session_id = '<SESSION>'
RETURN p;

What BigQuery does, table by table:

  1. campaign_runs → bind cr, filtered by session_id (1 row).
  2. rich_campaign_decision_edges → join to decision_points on session_id (~5 decisions per session).
  3. candidate_edges → join to candidates on decision_id (~3 options per decision).
  4. rich_candidate_status_edges → join to rich_candidate_statuses on status (1 row per option).
  5. Return paths bound to p. BigQuery Studio renders these as one fan-out per decision in the Graph tab.

5 fan-outs of 3 options each = 15 paths visualized for one session, ~97 paths across all 6 campaigns. The traversal is deterministic, the rendering is interactive.

Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

Step 9 — The temporal dimension

What's timestamped

  • agent_events.timestamp — when each plugin-recorded span happened (microsecond precision)
  • agent_events.latency_ms — {total_ms, time_to_first_token_ms}
  • context_cross_links.created_at — when cross-links were derived
  • candidate_edges.created_at — when decision edges were derived
  • decision_points.span_id → traces back to the source span's timestamp

What you can ask over time

  • "Across the last 30 days, which decision categories saw rising rejection rates?"
  • "Show me the per-day count of decisions made with confidence < 0.7." (oversight trend)
  • "Compare this quarter's rejection-rationale distribution vs last quarter's." (drift)
  • "Latency p50/p95 for LLM_RESPONSE spans on the audience-selection decision over the past week."

Each query is a join of made_decision_edges ⋈ agent_events plus a WHERE timestamp BETWEEN ... filter — no schema changes needed.

Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo
Part 6 · The four-guarantee roadmap

After #58 / #75 / #76 / #104 / #105 ship — own, validate, extract cheaply, resolve

Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

What changes when the four guarantees ship

Today — one pipeline, one cost curve

The SDK does everything from the binding outward:

  • Issues CREATE OR REPLACE PROPERTY GRAPH on every build
  • Runs AI.GENERATE over every relevant span — one call per session, prompt grows with span count
  • No structured pre-flight; binding drift surfaces as an extraction error mid-build, after BQ has already been billed
  • User-typed query inputs match by literal label, not canonical concept

Tomorrow — four checkpoints, four contracts

Same demo, four explicit failure modes:

  1. Own — your DDL; SDK populates tables only (#104)
  2. Validate — binding pre-flight (#105) + post-extract validator (#76)
  3. Extract cheaply — compiled extractors (#75) prune the AI prompt; AI handles semantic gaps only
  4. Resolve — concept-index reader (#58) translates user inputs to canonical names

Same audit answers. Tighter blast radius at every gate. Real cost reduction on the dominant call path.

Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

What the user authors after the four-guarantee roadmap ships

Minimum hand-authored input

One file

Either ontology.yaml (hand-authored), or a .ttl (OWL/SKOS) source compiled via gm import-owl. That's the entire authored surface.

Generated by tooling

binding.yaml + table_ddl.sql come from gm scaffold --ontology X --dataset Y --project Z --out outdir/. Hand-author the binding instead when pointing at pre-existing tables.

Generated by the plugin

agent_events rows arrive as the BQ AA Plugin records spans. The SDK never asks the user to author trace data.

"Provide one ontology file. Everything else — binding, table DDL, property graph — can be generated, owned by you, or both."
Future option (not on the current roadmap): a packaged @builtin:adk-events ontology covering the standard plugin event types — would drop the floor to zero authored YAML for users who don't need domain extraction. Filed as an input-ergonomics follow-up.
Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

The four guarantees — one card each

1 · Own — #104

Command: bq-agent-sdk ontology-build --skip-property-graph …
User does: authors CREATE PROPERTY GRAPH once, applies it.
SDK does: populates base tables; never issues PG DDL.
Evidence: zero rows in INFORMATION_SCHEMA.JOBS_BY_PROJECT WHERE query LIKE '%CREATE OR REPLACE PROPERTY GRAPH%' after the build window; same GQL traversal returns the same row count.

2 · Validate — #105 + #76

Pre-flight: bq-agent-sdk binding-validate runs in <1 s; structured failures carry binding_path and bq_ref.
Post-extract: every build emits a ValidationReport with FallbackScope ∈ {FIELD, NODE, EDGE} from #76 (EVENT scope arrives with #75 C2's runtime wrapper).
Per-field AI fallback re-extracts FIELD-scope failures; NODE / EDGE bubble up to the operator.

3 · Extract cheaply — #75 (C1 then C2)

C1: gm compile-extractors --event-schemas Z produces a deterministic bundle with compile_fingerprint (64 hex). Re-compile is byte-identical.
C2 runtime: spans whose compiled output validates cleanly leave the AI.GENERATE prompt entirely (fully_handled_span_ids); partial coverage stays in prompt with a focused hint; validation failures fall back to AI.
Result: for sessions composed of structured + validating events, AI.GENERATE cost approaches zero.

4 · Resolve — #58 (reader follow-on)

Already shipped (#92): gm compile --emit-concept-index writes a SKOS-aware lookup table plus a __meta provenance sibling.
New (#58 reader): OntologyRuntime resolves user-typed inputs ("Consumer Banking") to canonical entity names (skos_RetailBanking) via skos:altLabel matching before the GQL traversal runs.
Evidence: match score + path returned in the runtime API output.

Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

End-to-end after all four guarantees ship

# 1. One file you author. Either form works.
cat ontology.yaml                                   # or: gm import-owl my.ttl --out ontology.yaml

# 2. Scaffold the binding + base-table DDL. (Emits gen/binding.yaml + gen/table_ddl.sql.)
gm scaffold --ontology ontology.yaml --dataset campaigns --project my-proj --out gen/

# 3. Author the property-graph DDL once (separate from scaffold's outputs). Apply it.
bq query --nouse_legacy_sql < ddl/property_graph.sql

# 4. Compile deterministic extractors for structured event types.
gm compile-extractors --ontology ontology.yaml --binding gen/binding.yaml \
                      --event-schemas event_schemas/ --out compiled/

# 5. Pre-flight: catches binding drift before any extraction job runs.
bq-agent-sdk binding-validate --ontology ontology.yaml --binding gen/binding.yaml --project my-proj

# 6. Build: SDK populates tables, runtime loads the compiled bundle, AI.GENERATE
#          fires only for semantic gaps, post-extract validator emits ValidationReport.
bq-agent-sdk ontology-build --ontology ontology.yaml --binding gen/binding.yaml \
                            --skip-property-graph --validate-binding \
                            --compiled-bundle compiled/ \
                            --session-ids @latest

# 7. Query the audit graph. User-typed inputs resolve via the concept index.
#    "Consumer Banking" resolves to skos_RetailBanking before the GQL fires.
GRAPH `<P>.<D>.rich_agent_context_graph`
MATCH (dp:PlanningDecision)-[:WeighedOption]->(opt:DecisionOption)
WHERE dp.decision_type = @resolved_concept
RETURN opt.status, opt.score, opt.rejection_rationale;
Steps 5 and 6 each produce a typed report; a failed gate stops the next step from spending budget on a doomed run. Step 7's audit query is identical to today's demo — only the input resolution layer is new.
Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

Cost story — where the savings actually land

Today's cost driver

  • One AI.GENERATE per session, not per event
  • Prompt size grows linearly with structured-event span count
  • For the demo's 6 sessions × 27 spans, the BkaDecisionEvent-shaped payloads dominate the prompt budget
  • BQ AI billing is job-level; per-row token cost is not exposed

After #75 C2's validation-gated prune

  • Spans with compiled output that validates cleanly drop out of the prompt entirely
  • Spans with partial coverage stay, with a focused hint instead of full payload
  • Validator failures (NODE / EDGE / EVENT) fall back to AI; FIELD failures re-extract via per-field AI
  • For sessions whose events are entirely structured + validating, AI cost approaches zero
≈ 0Prompt tokens for fully-compiled sessions
F1 measuredvs hand-written + AI baselines
Byte-identicalrecompile reproducibility
Per-fieldfallback granularity
Per-session token columns in the savings table are prompt-size estimates (the transcript actually sent to AI.GENERATE after pruning). Real billing is job-level via INFORMATION_SCHEMA.JOBS_BY_PROJECT.total_bytes_processed. Per-row usage capture is a future enhancement once AI.GENERATE exposes usage_metadata.
Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

Issue gating — what blocks what

Beat in #107 storyboardGating issue(s)Status
Beat 1 — own the graph#104 (--skip-property-graph)Open
Beat 2a — pre-flight#105 (binding-validate)Open
Beat 2b — post-extract validator#76 (ValidationReport, FallbackScope)Open
Beat 3a — compile harness + measurement (C1)#75 PR 4b / 4cOpen
Beat 3b — runtime bundle-loading (C2)#75 C2 (Option A — prune compiled-extractable payloads, validation-gated)Open
Beat 4 — concept-index reader#58 reader follow-onEmission shipped (#92); reader open
Storyboard / migration notebook#107This roadmap
The notebook ships with conditional cells per #107 — each beat is gated by a feature flag and falls back to a "skipped: requires #X" placeholder until its underlying issue lands. The storyboard merges before the last issue ships; each new beat unlocks as it lands.
Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo
Close

Where to start

Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

Open source — try it on your project

Repository

GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK

The demo bundle

examples/decision_lineage_demo/

One-shot setup (5–10 min)

cd examples/decision_lineage_demo
./setup.sh

What ships

  • setup.sh / reset.sh — bootstrap + tear-down
  • agent/ + campaigns.py — real ADK agent + 6 briefs
  • run_agent.py + build_graph.py + build_rich_graph.py
  • bq_studio_queries.gql — six paste-and-run GQL blocks
  • property_graph.gql + rich_property_graph.gql — DDL templates

Documentation that ships with the bundle

License

Apache 2.0

Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

Q&A

Decision Lineage with BigQuery Context Graphs

Open source · Apache 2.0 · github.com/GoogleCloudPlatform/BigQuery-Agent-Analytics-SDK
examples/decision_lineage_demo
Decision Lineage with BigQuery Context Graphs · examples/decision_lineage_demo

Copyright 2026 Google LLC Licensed under the Apache License, Version 2.0. Decision Lineage with BigQuery Context Graphs — leadership deck. Render with Marp (https://marp.app): marp SLIDES.md --html --pdf # SLIDES.pdf marp SLIDES.md --html --pptx # SLIDES.pptx marp SLIDES.md --html # SLIDES.html marp SLIDES.md --watch --html # live preview `SLIDES.html` and `SLIDES.pptx` are checked in alongside this source so reviewers can open the deck without installing Marp. After editing this file, regenerate both with: npx -y @marp-team/marp-cli@latest SLIDES.md --html npx -y @marp-team/marp-cli@latest SLIDES.md --html --pptx --no-stdin (or `marp SLIDES.md --html --pptx` if you have Marp installed globally via `npm install -g @marp-team/marp-cli`).

SPEAKER NOTE — 30s Open with business pressure, not fear. Frame: AI agents are no longer just doing tasks (Systems of Action) — regulators now require we can prove what they did and why (Systems of Governance). The deck walks that line, top to bottom.

SPEAKER NOTE — 45s Frame as evolution, not rewrite. The five issues #58 / #75 / #76 / #104 / #105 ship under the storyboard tracked in #107. Every existing audit question still works the same; the platform contract gets stricter. The cost story sits in beat 3 — that's the slide finance teams care about.

SPEAKER NOTE — close "Three takeaways: (1) AI Act fines are real and the trigger date is near. (2) Decision Lineage is the audit substrate the regulator asks for, not a compliance certification. (3) The technical pipeline is open-source and runs on infrastructure you already pay for. Thanks."