silly-kicks — C4 Architecture

Generated from architecture.dsl using the Structurizr → PlantUML rendering pipeline.

Soccer AnalyticsPractitioner Data scientist or analyst whoclassifies and values footballactionsDownstream Pipeline Production data pipeline thatcalls silly-kicks inside SparkUDFskloppy PySport event datanormalization libraryML Libraries XGBoost, CatBoost, LightGBMgradient boosting frameworkssilly-kicks Classifies football actions intoSPADL representation andvalues them via VAEPConverts event data andvalues actions using[Python API]Calls inside SparkapplyInPandas UDFs via[Python import]Accepts EventDatasetfrom[kloppy bridge]Trains and predicts with[Python API]Legend  person  system 
silly-kicks[system]silly_kicks.xthreat[Python] Expected Threat model: pitchgrid value surface via dynamicprogrammingsilly_kicks.spadl[Python] SPADL conversion +post-conversion enrichments:23 action types, 6 dedicatedDataFrame converters withpreserve_native +goalkeeper_ids passthrough(StatsBomb, Opta, Wyscout,Sportec, Metrica, PFF FC) plusa kloppy gateway coveringStatsBomb / Sportec / Metrica,output coords clamped to [0,105] x [0, 68] AND unified tocanonical 'all-actions-LTR'SPADL orientation across allpaths, ConversionReport audit;public enrichment helpers(add_names, add_possessions,GK analytics suite  gk_role /distribution_metrics /pre_shot_gk_context,use_tackle_winner_as_actor);boundary_metrics +coverage_metrics utilities forvalidating add_possessionsoutput and per-action-typecoverage; ADR-001caller-conventions contract Sportec output usesSPORTEC_SPADL_COLUMNS(KLOPPY_SPADL_COLUMNS+ 4 object tackle qualifierpassthrough columns); PFFoutput usesPFF_SPADL_COLUMNS(SPADL_COLUMNS + 4nullable Int64 tacklepassthrough columns)silly_kicks.vaep[Python] VAEP framework: featureextraction, label generation(binary + xG), model training,action valuation. IncludesHybridVAEP(result-leakage-free).compute_features / rate acceptoptional frames= kwargdispatching frame-aware xfns(ADR-005)silly_kicks.tracking[Python] Tracking namespace(ADR-004): 19-columnlong-form per-frame schema,native Sportec + PFF adapters,kloppy gateway for Metrica +SkillCorner,link_actions_to_frames +slice_around_event linkageprimitives. PR-S20 (ADR-005)shipped the first tracking-awarefeatures:nearest_defender_distance,actor_speed,receiver_zone_density,defenders_in_triangle_to_goal+ add_action_contextaggregator +tracking_default_xfns. PR-S21added pre-shot GK position:pre_shot_gk_x / _y /_distance_to_goal /_distance_to_shot +add_pre_shot_gk_positionaggregator +pre_shot_gk_default_xfns.Schema-agnostic kernels in_kernels shared with atomicSPADL surfacesilly_kicks.atomic[Python] Atomic SPADL/VAEP:continuous actionrepresentation with 33 extendedaction types, deferredsingle-sort conversion, full parityfor the post-conversionenrichment helper family(preserve_native,add_possessions, GK analyticssuite, validate_atomic_spadl).atomic.tracking.features mirrorstracking.features foratomic-shaped column reads (x,y, dx, dy)Soccer AnalyticsPractitioner Data scientist or analyst whoclassifies and values footballactionsDownstream Pipeline Production data pipeline thatcalls silly-kicks inside SparkUDFskloppy PySport event datanormalization libraryML Libraries XGBoost, CatBoost, LightGBMgradient boosting frameworksConverts raw events toSPADL actions andenriches via[convert_to_actions() +add_*()helper family]Converts raw trackingdata to long-form frames+ enriches via[convert_to_frames() +add_action_context()]Values actions via[VAEP.fit() / VAEP.rate() /HybridVAEP (with optionalframes=)]Computes pitch valuesurface via[ExpectedThreat.fit()]Passes per-gameDataFrames to[lazy import inside UDF]Passes per-matchtracking frames to[lazy import inside UDF]Scores actions withpre-trained models via[VAEP.rate()]Accepts kloppyEventDataset in kloppyconverter[kloppy bridge]Accepts kloppyTrackingDataset in kloppygateway (Metrica,SkillCorner)[kloppy bridge]Reads SPADL config,schema constants, andaction names from[Python import]Delegates model trainingto[fit() dispatch via_LEARNER_REGISTRY]Imports frame_awaredecorator + Frames typealias from[vaep.feature_framework]Lazy-importstracking.utils.play_left_to_rightwhen frames= is supplied(no module-import-timecycle, ADR-005)[lazy import]Lazy-importstracking.features.add_pre_shot_gk_positionwhen frames= is suppliedtoadd_pre_shot_gk_context(no module-import-timecycle, ADR-005, PR-S21)[lazy import]Extends SPADL withatomic action types via[Python import]Inherits VAEP pipeline viaAtomicVAEP subclass;auto-inherits frames=extension (ADR-005)[Python import]Reuses _kernels +lift_to_states fromtracking namespace[Python import]Reads SPADL config andschema from[Python import]Legend  person  system  container  system boundary 
Operator workstation + DGX Spark + Media-PC[Hybrid] Local development + trainingWindows 11 (Karsten's PC)[RTX 5070 Ti, 96 GB RAM] Primary dev boxDGX Spark (192.168.68.73)[ARM64, 128 GB unified] Local LLM hostMedia-PC (192.168.68.70)[RTX, SSH-reachable] Secondary GPU hostAWS us-east-1[AWS] Cloud regionDatabricks Workspace[Databricks] Managed analytics platformUnity Catalog (soccer_analytics)[Unity Catalog] Governance layerJobs Serverless[Serverless compute] Workflow executionLakebase (managed Postgres)[Postgres] Read-replica sync targetSQL Warehouse[Serverless SQL] dbt + ad-hoc queriesS3 + KMS[Managed AWS] Terraform state + secretsHuggingFace[HuggingFace] Hugging Face platformDatasets + Models[Hub] Public HF Hub reposJobs (L40Sx1)[L40S 48GB] On-demand GPUSpaces[Spaces] Hosted appsuv + .venv[Local] Python 3.10 venvLlama 3.3 70B Q4[Ollama] Local LLM for evolve cyclesEvolve Engine[Python 3.10, JAX, PyTorch,openevolve, openrouter] Architecture-explorationframework: pre-registered rulespromote/keep/cut variants.Backends: LocalCudaBackend,RemoteSSHBackend(Media-PC + DGX Spark), HFJobs L40Sx1. SandboxedAST-allowlist `exec()` perADR-001.Bronze Delta Tables[Delta Lake on Unity Catalog`soccer_analytics.bronze`] Raw provider data  events,tracking, freeze frames,matches, teams, players.Schema-on-write with`_ingested_at` audit.Per-provider isolation; LiquidClustering on match_id.Silver Views (dbt)[dbt-core view materialization] Provider-specific staging views:stg_<provider>__<table>.Type-cast + dedup +bronze-completenesspassthroughs. Materialized asviews in`soccer_analytics.dev_silver`.Gold Marts (dbt)[Delta Lake on Unity Catalog`soccer_analytics.dev_gold`] 36 fact + 4 dimension marts in`soccer_analytics.dev_gold`.Tagged per ADR-019(PR-Cycle-C): dimension (4:dim_competitions/matches/players/teams),input_mart (3:fct_tracking_frames/shots/discipline_events),intermediate_mart (1:fct_action_values), output_mart(32: fct_passes,fct_match_summary,fct_physical_stats, allxG/PAUSA/DEFCON/embeddings/formations/aggregations).All migrated to Kimballsurrogate FKs(match_key/team_key/player_key/competition_key)per ADR-011. ML inferenceoutput marts(fct_xg_predictions,fct_xg_predictions_v2,fct_pausa_values) followADR-013 with contractenforcement.UC Volume / ModelRegistry[Unity Catalog Volumes + MLflow] Per-method model weights (xGv1+v2, VAEP, ScoutGPT,Football2Vec, PSxG, EPVgrids) under`/Volumes/soccer_analytics/dev_gold/model_weights/`.MLflow `@champion` aliasesper ADR-012 for atomic modelpromotion.Ingestion + ComputePipelines[Python wheel on Databricks JobsServerless] 27 workflow-card-defined jobsrunning on DatabricksServerless. Per-provideringestion(statsbomb/wyscout/idsse/metrica/skillcorner)+ per-method compute (pitchcontrol, OBSO, PAUSA,DEFCON, line-breaking, off-ballxT, space creation, formations,ScoutGPT/Football2Vec/xGinference). `applyInPandas`distribution.Lakebase (PostgreSQL)[Lakebase managed PostgreSQL] 40 synced tables (read replicaof Delta marts) on managedPostgres. 71 custom indexesvia `scripts/create_indexes.py`:65 btree for sub-100ms pointlookups + 6 HNSW (4×192dbehavioral on career/seasonembeddings + 2×144d on360-enriched + 2×13d stat).ADR-005 grants automation;ADR-002 §4 schema-driftguards.dbt build runner[PERFORMANCE_OPTIMIZED] Live-CI runs against thiswarehouse on every PRTF state bucket[S3] Encrypted by KMS,IAM-restrictedluxury-lakehouse org[HF org] 19 datasets + 17 model cards(auto-published via`ingestion.hf_publish.upload_hf_readme`per ADR-014)ML Training Jobs[Python 3.10 + uv + PEP 723 + HFJobs] PEP 723 `uv run` scripts on HFJobs L40Sx1: xG v2 (Deep Sets+ MC dropout), ScoutGPT(Doc2Vec/Word2Vec/transformer),Football2Vec (gensim in-house), VAEP (silly-kicks),OBSO grids. MLflow`@champion` aliases + UCVolume weights.ScoutGPT InferenceSpace[Python 3.10, PyTorch 2.5,transformers] Per-pass counterfactual ranking+ sequence completion.Currently disabled at runtime;weights published as HF modelartifact.Taipy Web Application[Python 3.10, Taipy 4.1, ll_ext, plotly] 16-page interactive dashboard:Pass Map, Heat Map, xG,VAEP, Line Breaking,Formations, Pass Timing, etc.Reads Lakebase synced tablesfor sub-100ms queries.Reads synced marts viapsycopg + connectionpool[PostgreSQL/TCP]Loads ScoutGPT weightsat startup[huggingface_hub mirror]Reads training inputs(events, tracking, freezeframes)[Spark]Uploads weights to UCVolume + sets MLflowchampion[MLflow + Spark]Dispatches variantevaluations[Backend abstraction]Reads bronze sources[view definition]Reads silver views[view + mart joins]Writes raw provider data[Delta MERGE/append]Writes ML inferenceoutputs to dev_gold(legacy direct-write beingretired per ADR-013)[Delta replaceWhere]Reads ML model weightsfor inference[Spark]Synced via DatabricksLakebase Sync[managed sync]Legend  container  node 
workspace "silly-kicks" "Football action classification (SPADL) and valuation (VAEP) library" {

    model {
        // --- Actors ---
        analyst = person "Soccer Analytics Practitioner" "Data scientist or analyst who classifies and values football actions"
        pipeline = person "Downstream Pipeline" "Production data pipeline that calls silly-kicks inside Spark UDFs"

        // --- External Systems ---
        kloppy = softwareSystem "kloppy" "PySport event data normalization library" "External"
        mlLibs = softwareSystem "ML Libraries" "XGBoost, CatBoost, LightGBM gradient boosting frameworks" "External"

        // --- The System ---
        sillyKicks = softwareSystem "silly-kicks" "Classifies football actions into SPADL representation and values them via VAEP" {

            spadl = container "silly_kicks.spadl" "SPADL conversion + post-conversion enrichments: 23 action types, 6 dedicated DataFrame converters with preserve_native + goalkeeper_ids passthrough (StatsBomb, Opta, Wyscout, Sportec, Metrica, PFF FC) plus a kloppy gateway covering StatsBomb / Sportec / Metrica, output coords clamped to [0, 105] x [0, 68] AND unified to canonical 'all-actions-LTR' SPADL orientation across all paths, ConversionReport audit; public enrichment helpers (add_names, add_possessions, GK analytics suite — gk_role / distribution_metrics / pre_shot_gk_context, use_tackle_winner_as_actor); boundary_metrics + coverage_metrics utilities for validating add_possessions output and per-action-type coverage; ADR-001 caller-conventions contract — Sportec output uses SPORTEC_SPADL_COLUMNS (KLOPPY_SPADL_COLUMNS + 4 object tackle qualifier passthrough columns); PFF output uses PFF_SPADL_COLUMNS (SPADL_COLUMNS + 4 nullable Int64 tackle passthrough columns)" "Python" "Library"
            vaep = container "silly_kicks.vaep" "VAEP framework: feature extraction, label generation (binary + xG), model training, action valuation. Includes HybridVAEP (result-leakage-free). compute_features / rate accept optional frames= kwarg dispatching frame-aware xfns (ADR-005)" "Python" "Library"
            tracking = container "silly_kicks.tracking" "Tracking namespace (ADR-004): 19-column long-form per-frame schema, native Sportec + PFF adapters, kloppy gateway for Metrica + SkillCorner, link_actions_to_frames + slice_around_event linkage primitives. PR-S20 (ADR-005) shipped the first tracking-aware features: nearest_defender_distance, actor_speed, receiver_zone_density, defenders_in_triangle_to_goal + add_action_context aggregator + tracking_default_xfns. PR-S21 added pre-shot GK position: pre_shot_gk_x / _y / _distance_to_goal / _distance_to_shot + add_pre_shot_gk_position aggregator + pre_shot_gk_default_xfns. Schema-agnostic kernels in _kernels shared with atomic SPADL surface" "Python" "Library"
            atomic = container "silly_kicks.atomic" "Atomic SPADL/VAEP: continuous action representation with 33 extended action types, deferred single-sort conversion, full parity for the post-conversion enrichment helper family (preserve_native, add_possessions, GK analytics suite, validate_atomic_spadl). atomic.tracking.features mirrors tracking.features for atomic-shaped column reads (x, y, dx, dy)" "Python" "Library"
            xthreat = container "silly_kicks.xthreat" "Expected Threat model: pitch grid value surface via dynamic programming" "Python" "Library"
        }

        // --- Relationships: Context level ---
        analyst -> sillyKicks "Converts event data and values actions using" "Python API"
        pipeline -> sillyKicks "Calls inside Spark applyInPandas UDFs via" "Python import"
        sillyKicks -> kloppy "Accepts EventDataset from" "kloppy bridge"
        sillyKicks -> mlLibs "Trains and predicts with" "Python API"

        // --- Relationships: Container level ---
        analyst -> spadl "Converts raw events to SPADL actions and enriches via" "convert_to_actions() + add_*() helper family"
        analyst -> tracking "Converts raw tracking data to long-form frames + enriches via" "convert_to_frames() + add_action_context()"
        analyst -> vaep "Values actions via" "VAEP.fit() / VAEP.rate() / HybridVAEP (with optional frames=)"
        analyst -> xthreat "Computes pitch value surface via" "ExpectedThreat.fit()"

        pipeline -> spadl "Passes per-game DataFrames to" "lazy import inside UDF"
        pipeline -> tracking "Passes per-match tracking frames to" "lazy import inside UDF"
        pipeline -> vaep "Scores actions with pre-trained models via" "VAEP.rate()"

        spadl -> kloppy "Accepts kloppy EventDataset in kloppy converter" "kloppy bridge"
        tracking -> kloppy "Accepts kloppy TrackingDataset in kloppy gateway (Metrica, SkillCorner)" "kloppy bridge"

        vaep -> spadl "Reads SPADL config, schema constants, and action names from" "Python import"
        vaep -> mlLibs "Delegates model training to" "fit() dispatch via _LEARNER_REGISTRY"
        tracking -> vaep "Imports frame_aware decorator + Frames type alias from" "vaep.feature_framework"
        vaep -> tracking "Lazy-imports tracking.utils.play_left_to_right when frames= is supplied (no module-import-time cycle, ADR-005)" "lazy import"
        spadl -> tracking "Lazy-imports tracking.features.add_pre_shot_gk_position when frames= is supplied to add_pre_shot_gk_context (no module-import-time cycle, ADR-005, PR-S21)" "lazy import"
        atomic -> spadl "Extends SPADL with atomic action types via" "Python import"
        atomic -> vaep "Inherits VAEP pipeline via AtomicVAEP subclass; auto-inherits frames= extension (ADR-005)" "Python import"
        atomic -> tracking "Reuses _kernels + lift_to_states from tracking namespace" "Python import"
        xthreat -> spadl "Reads SPADL config and schema from" "Python import"
    }

    views {
        systemContext sillyKicks "SystemContext" {
            include *
            autoLayout
        }

        container sillyKicks "Containers" {
            include *
            autoLayout
        }

        styles {
            element "Person" {
                shape Person
                background #08427B
                color #ffffff
            }
            element "Software System" {
                background #1168BD
                color #ffffff
            }
            element "External" {
                background #999999
                color #ffffff
            }
            element "Container" {
                background #438DD5
                color #ffffff
            }
            element "Library" {
                shape RoundedBox
            }
            element "Database" {
                shape Cylinder
            }
            element "Component" {
                background #85BBF0
                color #000000
            }
            relationship "Relationship" {
                color #707070
            }
        }
    }

}