Fictional AI Ops System Review Graph

Code-review graph shows the files; this system-review graph shows how evidence becomes recommended action, outcome, and lesson.

Depth: deep Systems: 6 Artifacts: 7 Gates: 4 Workflows: 6

Bigger Picture

This example shows how a team can explain a complex system without exposing private databases. The system collects safe source summaries, stores evidence contracts, recommends actions, blocks risky operations behind human review, captures outcomes, and turns lessons into better rules.

Source Links

Report Registers

Coverage Register

AreaCountWhat It MeansReviewer Use
Systems6Bounded contexts, services, subsystems, or product surfaces.Use this to see whether the report maps the main operating areas.
Artifacts7Inspectable files, APIs, tables, dashboards, reports, or outputs.Use this to trace where system claims can be inspected.
Schemas/contracts6Public or sanitized contracts for artifacts and handoffs.Use this to rebuild examples without touching private data.
Decision gates4Rules that advance, wait, block, or require human review.Use this to find where the system controls action.
Workflows6Lifecycle steps from input to output.Use this to follow what happens end to end.
Graph edges38Explicit and derived relationships between manifest nodes.Use this to audit connectivity and missing relationships.
Child maps0Linked subsystem maps for large repositories.Use this to drill into a map-of-maps instead of one flat report.
Blueprint sections0Source-evidence-backed operating flows.Use this to review deep behavior claims with proof anchors.
Blueprint evidence rows0Source paths, symbols, roles, and proof levels.Use this to verify whether blueprint claims are source-backed.
Source links0External or public references used by the report.Use this to confirm the report's public evidence base.
Known boundaries4Open limits, unproven claims, redactions, or scope exclusions.Use this to avoid treating the report as stronger than it is.
Review questions6Questions a maintainer, auditor, or agent should answer next.Use this as the human follow-up queue.
Rebuild phases3Documented commands or phases for reproducing the report.Use this to regenerate or verify the report locally.

Evidence Register

EvidenceKindCoverageProofReviewer Use
GET /internal/source-rollupsapidata-platformpublic_summary_onlyReturns sanitized source summaries.
private://evidence_graphprivate_databasedata-platformschema_onlyCanonical internal evidence store.
reports/recommendations.jsonjson_reportintelligencesafe_to_shareReviewable recommendation cards.
queue://action-intentsqueueoperationscounts_onlyDownstream actions waiting for execution or review.
reports/outcomes.jsonljsonl_ledgeroperationssafe_to_shareReality feedback from executed or skipped actions.
reports/lessons.jsonljsonl_ledgerresearchsafe_to_shareValidated lessons and parked rule changes.
app://operatoruioperationspublic_summary_onlyShows recommendations, blockers, approvals, outcomes, and lessons.
SourceEnvelopesanitized_eventsource_id, observed_at, source_type, summary, rights_statuscontract declaredDescribes a source observation without exposing raw vendor payloads.
EvidenceFactgraph_factfact_id, subject, kind, payload, valid_at, source_refcontract declaredStores traceable evidence used by recommendations.
RecommendationCarddecision_inputrecommendation_id, subject, confidence, reason, evidence_refs, risk_levelcontract declaredExplains why the system recommends an action.
ActionIntentdownstream_actionaction_id, recommendation_id, action_type, human_gate_required, statuscontract declaredRepresents a proposed action before execution.
OutcomeRecordreality_feedbackoutcome_id, action_id, capture_status, result, evidence_refscontract declaredCaptures whether action created the intended result.
LessonRecordlearning_looplesson_id, outcome_id, lesson, rule_change_statuscontract declaredTurns outcomes into validated or parked rule changes.

Gap Register

GapAreaStatusBoundaryNext Step
Known boundarywhole reportopenThis report explains architecture and system behavior; it does not prove production correctness.Accept the boundary or add evidence that closes it.
Known boundarywhole reportopenSanitized examples must not be treated as real data.Accept the boundary or add evidence that closes it.
Known boundarywhole reportopenA passing gate in a report still needs implementation tests in the actual system.Accept the boundary or add evidence that closes it.
Known boundarywhole reportopenHuman/legal/security gates should be implemented in production code, not only documented.Accept the boundary or add evidence that closes it.
System truth boundarySource MeshreviewSource summaries are not product claims.Inspect this boundary before making stronger behavior claims.
System truth boundaryEvidence CorereviewDatabase remains private; public report exposes only schema and examples.Inspect this boundary before making stronger behavior claims.
System truth boundaryRecommendation EnginereviewA recommendation is not an executed action.Inspect this boundary before making stronger behavior claims.
System truth boundaryAction EnginereviewRestricted actions require approval before execution.Inspect this boundary before making stronger behavior claims.
System truth boundaryOutcome Learning LoopreviewA lesson does not change future behavior until promotion is validated.Inspect this boundary before making stronger behavior claims.
System truth boundaryOperator UIreviewUI reads canonical reports and requests actions; it does not invent truth.Inspect this boundary before making stronger behavior claims.
Source links missingwhole reportopenNo external source links were declared.Add public repo, docs, issue, or design references.
Blueprint not declaredwhole reportoptionalNo source-backed blueprint sections were declared.Add blueprint sections when the report needs source-level proof.

Action Register

ActionOwnerStatusTriggerExpected Output
Review questionmaintainer / auditoropenWhich source and evidence artifacts prove each recommendation?Answer from source, tests, docs, logs, or maintainer knowledge.
Review questionmaintainer / auditoropenWhich decision gate blocks unsafe downstream action?Answer from source, tests, docs, logs, or maintainer knowledge.
Review questionmaintainer / auditoropenWhich actions require human approval and why?Answer from source, tests, docs, logs, or maintainer knowledge.
Review questionmaintainer / auditoropenWhere is the outcome captured after action or no-action?Answer from source, tests, docs, logs, or maintainer knowledge.
Review questionmaintainer / auditoropenWhich lessons are promoted, parked, or killed based on reality feedback?Answer from source, tests, docs, logs, or maintainer knowledge.
Review questionmaintainer / auditoropenWhat can be safely reviewed if the production database remains private?Answer from source, tests, docs, logs, or maintainer knowledge.
Resolve boundarymaintainer / auditoropenThis report explains architecture and system behavior; it does not prove production correctness.Accept as scope or add proof that closes it.
Resolve boundarymaintainer / auditoropenSanitized examples must not be treated as real data.Accept as scope or add proof that closes it.
Resolve boundarymaintainer / auditoropenA passing gate in a report still needs implementation tests in the actual system.Accept as scope or add proof that closes it.
Resolve boundarymaintainer / auditoropenHuman/legal/security gates should be implemented in production code, not only documented.Accept as scope or add proof that closes it.
Rebuild phasemaintainer / agentrepeatablevalidateCheck the manifest shape.
Rebuild phasemaintainer / agentrepeatablebuildGenerate JSON and Markdown reports.
Rebuild phasemaintainer / agentrepeatablereviewRead the system as an operating map.

Lifecycle Map

flowchart LR
  load_sources["Load Sources"]
  write_evidence["Write Evidence"]
  score_recommendations["Score Recommendations"]
  route_action["Route Action"]
  capture_outcome["Capture Outcome"]
  learn_from_outcome["Learn From Outcome"]
  load_sources --> write_evidence["Write Evidence"]
  write_evidence --> score_recommendations["Score Recommendations"]
  score_recommendations --> route_action["Route Action"]
  route_action --> capture_outcome["Capture Outcome"]
  capture_outcome --> learn_from_outcome["Learn From Outcome"]
  learn_from_outcome --> score_recommendations["Score Recommendations"]

Artifact And Schema Map

flowchart LR
  system_source_mesh["Source Mesh"]
  system_source_mesh --> artifact_source_rollup_api["Source Rollup API"]
  artifact_source_rollup_api --> schema_SourceEnvelope["SourceEnvelope"]
  system_evidence_core["Evidence Core"]
  system_evidence_core --> artifact_evidence_graph["Evidence Graph"]
  artifact_evidence_graph --> schema_EvidenceFact["EvidenceFact"]
  system_recommendation_engine["Recommendation Engine"]
  system_recommendation_engine --> artifact_recommendation_report["Recommendation Report"]
  artifact_recommendation_report --> schema_RecommendationCard["RecommendationCard"]
  system_action_engine["Action Engine"]
  system_action_engine --> artifact_action_queue["Action Queue"]
  artifact_action_queue --> schema_ActionIntent["ActionIntent"]
  system_outcome_learning_loop["Outcome Learning Loop"]
  system_outcome_learning_loop --> artifact_outcome_ledger["Outcome Ledger"]
  artifact_outcome_ledger --> schema_OutcomeRecord["OutcomeRecord"]
  system_outcome_learning_loop --> artifact_lesson_registry["Lesson Registry"]
  artifact_lesson_registry --> schema_LessonRecord["LessonRecord"]
  system_operator_ui["Operator UI"]
  system_operator_ui --> artifact_operator_console["Operator Console"]
  artifact_operator_console --> schema_ActionIntent["ActionIntent"]

Gate Map

flowchart LR
  gate_source_rights_gate{"Source Rights Gate"}
  gate_source_rights_gate --> out_source_rights_gate_internal_only["internal_only"]
  gate_source_rights_gate --> out_source_rights_gate_safe_to_use["safe_to_use"]
  gate_source_rights_gate --> out_source_rights_gate_blocked["blocked"]
  gate_recommendation_confidence_gate{"Recommendation Confidence Gate"}
  gate_recommendation_confidence_gate --> out_recommendation_confidence_gate_advance["advance"]
  gate_recommendation_confidence_gate --> out_recommendation_confidence_gate_wait["wait"]
  gate_recommendation_confidence_gate --> out_recommendation_confidence_gate_reject["reject"]
  gate_human_action_gate{"Human Action Gate"}
  gate_human_action_gate --> out_human_action_gate_approved["approved"]
  gate_human_action_gate --> out_human_action_gate_blocked["blocked"]
  gate_human_action_gate --> out_human_action_gate_needs_review["needs_review"]
  gate_lesson_promotion_gate{"Lesson Promotion Gate"}
  gate_lesson_promotion_gate --> out_lesson_promotion_gate_promote["promote"]
  gate_lesson_promotion_gate --> out_lesson_promotion_gate_park["park"]
  gate_lesson_promotion_gate --> out_lesson_promotion_gate_kill["kill"]
  gate_source_rights_gate{"Source Rights Gate"} --> step_load_sources["Load Sources"]
  gate_source_rights_gate{"Source Rights Gate"} --> step_write_evidence["Write Evidence"]
  gate_recommendation_confidence_gate{"Recommendation Confidence Gate"} --> step_score_recommendations["Score Recommendations"]
  gate_human_action_gate{"Human Action Gate"} --> step_route_action["Route Action"]
  gate_human_action_gate{"Human Action Gate"} --> step_capture_outcome["Capture Outcome"]
  gate_lesson_promotion_gate{"Lesson Promotion Gate"} --> step_learn_from_outcome["Learn From Outcome"]

Relationship Graph

flowchart TD
  operator_console["Operator Console"] -- "requests approval from" --> human_action_gate["Human Action Gate"]
  source_mesh["Source Mesh"] -- "owns or uses" --> source_rollup_api["Source Rollup API"]
  source_mesh["Source Mesh"] -- "is gated by" --> source_rights_gate["Source Rights Gate"]
  evidence_core["Evidence Core"] -- "owns or uses" --> evidence_graph["Evidence Graph"]
  recommendation_engine["Recommendation Engine"] -- "owns or uses" --> recommendation_report["Recommendation Report"]
  recommendation_engine["Recommendation Engine"] -- "is gated by" --> recommendation_confidence_gate["Recommendation Confidence Gate"]
  action_engine["Action Engine"] -- "owns or uses" --> action_queue["Action Queue"]
  action_engine["Action Engine"] -- "is gated by" --> human_action_gate["Human Action Gate"]
  outcome_learning_loop["Outcome Learning Loop"] -- "owns or uses" --> outcome_ledger["Outcome Ledger"]
  outcome_learning_loop["Outcome Learning Loop"] -- "owns or uses" --> lesson_registry["Lesson Registry"]
  outcome_learning_loop["Outcome Learning Loop"] -- "is gated by" --> lesson_promotion_gate["Lesson Promotion Gate"]
  operator_ui["Operator UI"] -- "owns or uses" --> operator_console["Operator Console"]
  operator_ui["Operator UI"] -- "is gated by" --> human_action_gate["Human Action Gate"]
  external_public_sources["external_public_sources"] -- "feeds" --> load_sources["Load Sources"]
  private_internal_sources["private_internal_sources"] -- "feeds" --> load_sources["Load Sources"]
  load_sources["Load Sources"] -- "produces" --> source_rollup_api["Source Rollup API"]
  source_rights_gate["Source Rights Gate"] -- "gates" --> load_sources["Load Sources"]
  load_sources["Load Sources"] -- "routes to" --> write_evidence["Write Evidence"]
  source_rollup_api["Source Rollup API"] -- "feeds" --> write_evidence["Write Evidence"]
  write_evidence["Write Evidence"] -- "produces" --> evidence_graph["Evidence Graph"]
  source_rights_gate["Source Rights Gate"] -- "gates" --> write_evidence["Write Evidence"]
  write_evidence["Write Evidence"] -- "routes to" --> score_recommendations["Score Recommendations"]
  evidence_graph["Evidence Graph"] -- "feeds" --> score_recommendations["Score Recommendations"]
  score_recommendations["Score Recommendations"] -- "produces" --> recommendation_report["Recommendation Report"]
  recommendation_confidence_gate["Recommendation Confidence Gate"] -- "gates" --> score_recommendations["Score Recommendations"]
  score_recommendations["Score Recommendations"] -- "routes to" --> route_action["Route Action"]
  recommendation_report["Recommendation Report"] -- "feeds" --> route_action["Route Action"]
  route_action["Route Action"] -- "produces" --> action_queue["Action Queue"]
  human_action_gate["Human Action Gate"] -- "gates" --> route_action["Route Action"]
  route_action["Route Action"] -- "routes to" --> capture_outcome["Capture Outcome"]
  action_queue["Action Queue"] -- "feeds" --> capture_outcome["Capture Outcome"]
  capture_outcome["Capture Outcome"] -- "produces" --> outcome_ledger["Outcome Ledger"]
  human_action_gate["Human Action Gate"] -- "gates" --> capture_outcome["Capture Outcome"]
  capture_outcome["Capture Outcome"] -- "routes to" --> learn_from_outcome["Learn From Outcome"]
  outcome_ledger["Outcome Ledger"] -- "feeds" --> learn_from_outcome["Learn From Outcome"]
  learn_from_outcome["Learn From Outcome"] -- "produces" --> lesson_registry["Lesson Registry"]
  lesson_promotion_gate["Lesson Promotion Gate"] -- "gates" --> learn_from_outcome["Learn From Outcome"]
  learn_from_outcome["Learn From Outcome"] -- "routes to" --> score_recommendations["Score Recommendations"]

Systems

Source Mesh

Collects sanitized source summaries from internal and public systems.

Architecture
loader mesh
Lifecycle
source -> SourceEnvelope -> source rights gate
Boundary
Source summaries are not product claims.

Evidence Core

Stores facts, relationships, and reasoning traces used by recommendations.

Architecture
private evidence graph
Lifecycle
SourceEnvelope -> EvidenceFact -> graph neighborhood
Boundary
Database remains private; public report exposes only schema and examples.

Recommendation Engine

Turns evidence into reviewable recommendation cards.

Architecture
batch plus API scoring
Lifecycle
EvidenceFact -> RecommendationCard -> confidence gate
Boundary
A recommendation is not an executed action.

Action Engine

Routes approved recommendations into bounded action intents.

Architecture
event-driven workflow
Lifecycle
RecommendationCard -> ActionIntent -> human gate
Boundary
Restricted actions require approval before execution.

Outcome Learning Loop

Captures what happened and converts outcomes into lessons.

Architecture
ledger and validation loop
Lifecycle
ActionIntent -> OutcomeRecord -> LessonRecord -> promotion gate
Boundary
A lesson does not change future behavior until promotion is validated.

Operator UI

Shows recommendations, approvals, outcomes, and lessons without owning truth.

Architecture
report-backed control room
Lifecycle
reports -> UI -> operator decision -> action intent
Boundary
UI reads canonical reports and requests actions; it does not invent truth.

Review Questions