# Turbine

Turbine helps teams define dataset expectations and run quality checks against data products.

## Language

**Data Contract**:
A data product owner's enforceable promise about a dataset's schema, ownership, and core quality expectations. Serialised as a conformant ODCS document so it round-trips through the wider ecosystem (see [ADR-0002](docs/adr/0002-turbine-quality-checks-serialise-as-odcs-custom-checks.md)).
_Avoid_: contract file, YAML contract when discussing the domain concept

**Turbine Engine**:
The named quality engine that owns the execution semantics of a Check Definition inside a published contract. Other tools see that a check exists; only the Turbine Engine interprets how it runs.
_Avoid_: turbine runner, custom check engine — `Turbine Engine` is the canonical term

**Turbine Release**:
A versioned publication of the Turbine package (`turbine-data` on PyPI), identified by a PEP 440 string (e.g. `0.5.11`). The installed Release surfaces via `turbine --version`; the full set of published Releases surfaces via `turbine self versions`. Distinct from a **Data Contract**'s ODCS `version:` field — that records a change to a producer's authored promise (managed by `turbine bump`); a Turbine Release upgrades the engine that runs every Check Definition. The CLI also nudges the user when a newer Release is available on PyPI, with an upgrade hint tailored to how Turbine was installed.
_Avoid_: turbine version (overloaded with contract version), turbine-data version — `Turbine Release` is the canonical term for the package's own semver

**Turbine Skill**:
A unit of agent guidance that teaches an AI coding agent how to perform one Turbine task correctly — which commands to run, in what order, and how to read the result. Distributed with a freshly scaffolded Turbine project so any SKILL.md-aware agent (Claude Code, opencode, and others that read the format) can drive Turbine, and frozen into that project at scaffold time. A Turbine Skill describes existing Turbine behaviour; it never adds new engine capability.
_Avoid_: plugin, command, macro — `Turbine Skill` is the canonical term; reserve "Agent Skill" for the cross-tool format in general

**Quality Spec**:
A team's observability-focused set of checks against a dataset covered by a Data Contract.
_Avoid_: secondary contract, monitoring file

**Check Definition**:
A single configured quality check with its own identity, threshold, dimension, and execution settings.
_Avoid_: check template, SQL file

**Check Name**:
The stable identity of a Check Definition in results, dashboards, flags, and history.
_Avoid_: filename, path

**SQL Check Template**:
A reusable SQL query template that one or more Check Definitions can run with their own parameters.
_Avoid_: SQL check, check name

**SLA Property**:
A producer's promise about _service_ the data delivers — when it's fresh, how long it's retained, when the dataset reaches end-of-life. Drawn from the Data QoS periodic table (see [ADR-0001](docs/adr/0001-sla-properties-adhere-to-data-qos.md)). Distinct from a Data Quality Dimension, which is about the _content_ of the data.
_Avoid_: SLA, service level, contract promise — `SLA Property` is the canonical term

**Data Quality Dimension**:
The aspect of data _content_ that a Check Definition measures. Turbine uses five: **Accuracy**, **Completeness**, **Consistency**, **Timeliness**, **Validity**. Carried by a Check Definition's `dimension:` field. Distinct from an SLA Property, which is about _service_, not content.
_Avoid_: data quality, DQ dimension when the audience won't recognise the term; "uniqueness" and "conformity" are not Turbine dimensions — uniqueness rolls up under Consistency, conformity under Validity

**Check Outcome**:
The verdict a single executed Check Definition produced. Closed set: **PASSED**, **FAILED**, **WARNED**, **SKIPPED**, **ERRORED**. Carries explicit semantics — FAILED and ERRORED block downstream consumers; WARNED surfaces a concern but does not block; SKIPPED is benign (empty incremental window, filter excluded all rows).
_Avoid_: result, status — `Check Outcome` is the canonical term for the verdict; `RunStatus` covers the run-level lifecycle separately

**Check Run**:
A single execution of every Check Definition for one Data Contract against one of its **Servers**, on the Turbine Engine. Identified by a `run_id` returned by the management API. Carries a lifecycle **RunStatus** (RUNNING → terminal: COMPLETED, NO*DATA, FAILED); produces a Run Result on reaching terminal status.
A Check Run may be scoped to a **Check Window** via explicit `since`/`until` bounds — this is how partition-aware surfaces (Dagster's partition time-window, Airflow's data interval) request "check exactly this slice." The window is set per Check Run and is independent of incremental mode's watermark; the watermark exists for surfaces that have no partition concept of their own.
\_Avoid*: job, batch — Turbine has no jobs

**Check Window**:
The `(since, until)` half-open time interval that bounds the rows a Check Run evaluates. Either bound may be omitted; omitting both means "every row in the dataset." Distinct from the incremental watermark — the window is supplied by the caller, the watermark is inferred from the audit trail.
_Avoid_: interval, slice — `Check Window` is the canonical term

**Run Result**:
The terminal artifact of a Check Run: its final RunStatus together with one Check Outcome per Check Definition executed. Returned by `RunSession.wait()` in the client. Carries `raise_for_failure()` to escalate a FAILED RunStatus to a `TurbineRunFailedError`; callers opt in.
_Avoid_: result, outcome, summary — `Check Outcome` is one check's verdict, `Run Result` is the per-run roll-up

**Metadata Store**:
Turbine's persistence layer for the operational state the engine itself owns: the audit trail of Check Runs, Check Outcomes, flagged rows, and incremental watermarks. Modelled as a regular **Datasource** — any SQL Datasource (DuckDB, Postgres, Snowflake) can serve as the Metadata Store; the "metadata-store-ness" is purely _which datasource is pointed at_ via config, not a property of the Datasource itself (see [ADR-0003](docs/adr/0003-run-store-is-a-datasource.md)). Scoped to a single schema (default `turbine_metadata`) so it can share a backend with the data being checked. Distinct from the metadata _inside_ a Data Contract (owners, schema, tags) — that lives in the ODCS document, never in the Metadata Store; when the overload bites, say "Metadata Store" for the engine's DB and "contract metadata" for the ODCS fields.
_Avoid_: run store, audit DB — `Metadata Store` is the canonical term for the engine's persistence layer (matching the code's `metadata_store_*` / `TURBINE_METADATA_*` naming); reserve "metadata" unqualified for ODCS contract fields

**Datasource**:
A named, self-contained connection to one SQL backend (DuckDB, Postgres, Snowflake): a YAML file under `datasources/` with `${env.X}` interpolation for secrets, complete enough to connect on its own — so it can introspect or scaffold a dataset before any contract exists. A **Server** binds to a Datasource by matching name to obtain the live connection; the **Metadata Store** is itself a Datasource (see [ADR-0003](docs/adr/0003-run-store-is-a-datasource.md)).
_Avoid_: connection, database, DSN, source — `Datasource` is the canonical term.

**Server**:
The physical place a **Data Contract**'s logical tables are served — an ODCS `servers:` entry carrying `type` plus topology (host, database, schema), bound to a **Datasource** by matching name. A contract declares one or more, so the same contract can list a Postgres Server and a Snowflake Server for the same logical tables; the Server supplies each table's schema/database and SQL dialect at run time. The contract's topology is the published face; the Datasource holds the private connection, and a lint flags drift between them (see [ADR-0005](docs/adr/0005-contracts-bind-to-datasources-via-odcs-servers.md)).
_Avoid_: environment, connection, datasource — a Server is the published _where_; the Datasource is the private _how_.

**Contract Change**:
One structural delta detected between two snapshots of a Data Contract. Closed set covering column add/remove/type/nullable/required/PK/description changes, check add/remove/threshold changes, schema removal, and SLA element changes. Distinct from a **Check Outcome** — a Contract Change is detected statically by comparing two Data Contract documents; a Check Outcome is observed at runtime by executing a Check Definition against data.
_Avoid_: change, delta, mutation — `Contract Change` is the canonical term

**Contract Diff**:
The aggregated set of Contract Changes between two Data Contract snapshots, together with the resulting semver bump classification (NONE / PATCH / MINOR / MAJOR). Lives as a single deep module in `core/contract/diff.py` that owns the computation, the `ContractChange` domain value, and the opt-in Pydantic wire view used by the LSP and CLI. The domain value flows through the engine; the wire view is produced only when crossing a serialisation boundary.
_Avoid_: contract delta, version diff, diff result — `Contract Diff` is the canonical term

**Analysis Result**:
A domain value the Turbine Engine produces by inspecting a Data Contract — Check Run rollups, contract health, drift status, scaffold variants, onboarding status. Owned by `core/analysis/`, consumed across `application/analysis/` and every presentation surface (LSP, CLI, dashboard). Distinct from a _wire shape_: when the LSP wire happens to round-trip an Analysis Result unchanged, the result is still the owner — the wire is a consumer. A wire-only shape (an LSP request's `*Params`, or an ack-style `*Result` built solely in an LSP handler) is not an Analysis Result and lives next to its handler.
_Avoid_: wire DTO, protocol type — `Analysis Result` is the canonical term for engine-produced analysis values

**Document Validation**:
Static validation of a Data Contract's YAML document against the ODCS JSON Schema and Turbine's schema extensions — no database connection involved. Produces diagnostics about the document itself: unknown fields, wrong types, missing required entries. Feeds editor diagnostics and CLI checks. Distinct from **Schema Validation**, which compares a contract against a live database, and from contract linting, which applies semantic rules to a structurally valid document.
_Avoid_: schema validation (overloaded), YAML validation — `Document Validation` is the canonical term for the static check

**Schema Validation**:
Runtime comparison of a Data Contract's declared columns against the live database schema of one of its **Servers** — requires a connection. Detects drift: missing columns, type mismatches, nullability and primary-key disagreements. Routes per-Server through the **Management API** like a Check Run. Distinct from **Document Validation**, which checks the YAML document statically.
_Avoid_: validation (unqualified), drift check — `Schema Validation` is the canonical term for the live comparison

**Orchestrator Integration**:
A packaged adapter that surfaces a Turbine server's checks inside a data orchestrator, so a team's existing pipelines run Turbine checks without calling the HTTP API by hand. Two forms exist, each shaped to its host's grain rather than mirrored mechanically: the **Dagster Component** (declarative, discovery-driven) and the **Airflow Operator** (explicit, task-driven). The asymmetry is deliberate — Dagster discovers at definition-load, which is occasional and tolerant of an unreachable server; Airflow re-parses DAGs constantly, so a discovery network call on the parse path would be an anti-pattern. They share _concepts_ (a Check Run bound to a unit of orchestration), not _mechanism_.
_Avoid_: plugin, connector, "the component" (Component is Dagster-specific, not an umbrella)

**Dagster Component**:
The declarative form of an Orchestrator Integration (`TurbineChecksComponent`). At definition-load it walks the server's contracts, tables, and registered Check Definitions and generates one native Dagster asset check per Check Definition, each running its Check Run in-process via the client. A `TurbineServerResource` is published alongside for assets and checks that call the API directly.
_Avoid_: just "component" without "Dagster"; Dagster plugin

**Airflow Operator**:
The explicit form of an Orchestrator Integration (`TurbineCheckOperator`). A team declares one task in a DAG; the operator starts a Check Run on the server, defers while it executes on the triggerer, and fails the task on any failing Check Outcome. No discovery — the task is the unit of binding the team writes.
_Avoid_: Airflow component, Airflow plugin

**Contract Selection**:
The set of Data Contracts a single check invocation targets. Defaults to _all_ registered contracts; may be narrowed to an explicit list. Because a Check Run is exactly one Data Contract, a selection of N contracts is realised as N independent Check Runs — a fan-out, never one multi-contract run. The same concept surfaces across every entry point: CLI paths (`turbine check a.yml b.yml` / `--all`), the management API (`POST /checks/runs` with `contract_ids`, omitted means all), and both Orchestrator Integrations (`contracts`).
_Avoid_: batch run, multi-contract run — a selection is a fan-out of single-contract runs

**Check Severity**:
A producer's authored declaration, per Check Definition, of whether breaching it is a hard failure or merely advisory. Authored as `severity: error | warn` on the check body (default `error`); `error` is _gating_ — a breach produces a FAILED Check Outcome and blocks downstream — while `warn` is _advisory_ — a breach produces WARNED, surfaced but not blocking. Severity is what decides whether a breach becomes FAILED or WARNED. The vocabulary (`error`/`warn`) matches dbt; "gating"/"advisory" are the semantic descriptions. Per-check, no warn/fail bands. Lives on the Data Contract and round-trips through ODCS like any other Check Definition field. Distinct from Blocking — Severity is the producer's intent about the check itself.
_Avoid_: blocking, gate flag — Severity is the producer-authored intent (`error`/`warn`), not the consumer's halt decision

**Blocking**:
An Orchestrator Integration's decision about whether a failed Check Outcome halts the consumer's downstream pipeline — Dagster's `AssetCheckSpec.blocking` (stops dependent assets), Airflow's task failure (subject to DAG trigger rules). A consumer/orchestration concern, not a property of the check: the same Check Definition may block one team's pipeline and only warn another's. Defaults to the check's Check Severity (gating blocks, advisory never blocks); an integration may override.
_Avoid_: severity, gate — Blocking is consumer-side pipeline topology, separate from the producer's Check Severity

**Dashboard Component**:
A declarative building block of a **Dashboard Page** — layout (Page, Tabs, KPI strips), data (tables, pickers), display, and the chart catalog. The component catalog is the dashboard's extension interface for teams: every component is public authoring surface for team-authored pages, independent of whether the bundled starter pages happen to use it. Removing a component is an API break, not dead-code cleanup.
_Avoid_: widget, chart helper, internal component — the catalog is interface, not implementation

**Chart Renderer**:
The per-chart choice of rendering engine for a chart **Dashboard Component** — ECharts by default, Plotly as the deliberate alternative. Part of the extension interface: a team may select the renderer per chart, so each renderer's chart builders are parallel implementations of one chart contract, not duplication to delete. Not every component supports every renderer; the support matrix is explicit, and an unsupported combination is a loud error, not a silent fallback.
_Avoid_: backend, engine, "the duplicate builders" — renderer choice is a capability, not an accident

**Dashboard Page**:
A team-authored function, marked with `@page`, that composes **Dashboard Components** and declares its data needs; discovered from the project's dashboard folder at startup. The scaffolded starter pages are ordinary Dashboard Pages — a template-derived page and a hand-written one flow through the same rendering path.
_Avoid_: view, screen, built-in page — there is no privileged page kind

**Management API**:
The hand-written ops surface of `turbine serve`, mounted under `/api/v1/manage` with its own Swagger UI. Owns contracts, validation, Check Runs, flagged rows, status, and the dashboard. Binds to a **Datasource** _dynamically, per request_: a request never carries a datasource — the targeted contract's chosen **Server** (an optional `?server=`/`server` field, else its first `servers:` entry) name-matches a Datasource, opened lazily and cached per (Datasource, Server). Both Check Runs and schema validation route this way. `turbine serve --datasource <name>` pins every run onto one Datasource.
_Avoid_: ops API, admin API — `Management API` is the canonical term for the `/api/v1/manage` surface.

**Data Product API**:
The contract-first REST surface generated by `turbine generate api`, served from the project's `api/` dir at `/{domain}/v{major}/{contract_id}/{resource}` with its own Swagger UI (the unprefixed `/docs`). Binds to _exactly one_ connection for the whole sub-app at mount time — its `mount(app, *, datasource=… | engine=…)` takes one or the other, never both. Under `turbine serve` it receives the already-built engine of the _primary_ Datasource; it has no per-Server routing. Serving contracts backed by different Datasources from one server is deferred (issue #6, milestone v0.10) — its remaining open bullets are the data-product endpoints themselves; per-contract schema validation already routes per-Server through the **Management API**.
_Avoid_: data API, generated API — `Data Product API` is the canonical term; "Data API" is only the Swagger UI's display title.

## Relationships

- A **Data Contract** can contain zero or more **Check Definitions**.
- A **Quality Spec** targets exactly one **Data Contract** and contains zero or more **Check Definitions**.
- A **Check Definition** has exactly one **Check Name**.
- A **Check Definition** carries exactly one **Data Quality Dimension**.
- A **Check Definition** is carried in a **Data Contract** as an ODCS custom check executed by the **Turbine Engine**.
- A **Check Definition** can reference at most one **SQL Check Template**.
- A **SQL Check Template** can be reused by many **Check Definitions**.
- A **Data Contract** can carry zero or more **SLA Properties**.
- An **SLA Property** may derive a **Check Definition** (e.g. `freshness`, `latency`, `retention`) or stay metadata-only (e.g. `endOfLife`).
- A **Data Contract** declares one or more **Servers**; a contract with no Server is invalid.
- A **Server** resolves to exactly one **Datasource** by matching name, which supplies the live connection and secrets.
- Table names in a **Data Contract**'s schema are _logical_ (unqualified); the targeted **Server** supplies the schema/database qualifier and SQL dialect.
- A **Check Run** executes the **Check Definitions** of exactly one **Data Contract** against exactly one of its **Servers**; verifying a contract across all its Servers is a fan-out of one Check Run per Server (the **Contract Selection** fan-out, applied to Servers).
- A **Check Run** produces exactly one **Run Result** on reaching terminal status.
- A **Run Result** carries zero or more **Check Outcomes** — one per **Check Definition** executed.
- The **Metadata Store** persists every **Check Run**, its **Run Result**, and any flagged rows the run produced. One Turbine Engine instance reads and writes exactly one Metadata Store at a time.
- The **Metadata Store** is itself a **Datasource** — pointing it at a specific Datasource is a config decision (pyproject / env / CLI), not a Datasource-shape decision. When no Metadata Store is configured, the engine defaults to a project-local DuckDB file and surfaces that default visibly; it does not silently borrow the contract's data Datasource.
- A **Contract Diff** is computed between two snapshots of one **Data Contract** and contains zero or more **Contract Changes**, each typed by its `ChangeType`.
- One `turbine serve` exposes both the **Management API** and the **Data Product API**. The **Management API** binds a **Datasource** per request (both Check Runs and schema validation) via each contract's **Server**; the **Data Product API** binds one **Datasource** (or engine) for its whole sub-app at mount time. The remaining asymmetry — the data-product endpoints themselves — is tracked for convergence in issue #6.

## Example dialogue

> **Dev:** "If two teams use the same SQL rule, do they share the same Check Name?"
> **Domain expert:** "No — they share the SQL Check Template, but each use is a separate Check Definition with its own Check Name, threshold, and parameters."

> **Dev:** "Where does the Snowflake password go — in the contract's `servers:` block?"
> **Domain expert:** "No. The **Server** in the contract names _where_ the data lives — type, host, schema — and links by name to a **Datasource**; the password lives only in that Datasource. The same contract can list a Postgres Server and a Snowflake Server for the same logical tables."

**Marketplace**:
The Enexis data-product catalog a **Data Contract** is published to via `turbine publish` (POST raw ODCS YAML to the PAX public API, `x-api-key` auth). It ingests the contract, maps `domain` to its `primaryBusinessUnit`, fills the author from the API key, and stores the contract under its own UUID while preserving the Data Contract's `id` as the catalog lookup identity. Distinct from a **Datasource** (where the data physically lives) — the Marketplace holds the published _description_, not the data.

**Classification**:
The four governance ratings the **Marketplace** requires on every published **Data Contract**, carried as ODCS `customProperties` entries: `availabilityClassification` (`B0`–`B3`), `integrityClassification` (`I0`–`I3`), `confidentialityClassification` (`V0`–`V3`), `privacyClassification` (`P0`–`P3`). The Dutch BIV+P model (Beschikbaarheid, Integriteit, Vertrouwelijkheid, Privacy). Turbine requires all four, with a valid code, on every contract.
_Avoid_: tag, label — a Classification is one of these four fixed axes with a coded value, not a free-form `tags:` entry.

## Flagged ambiguities

- "SQL check" was used to mean both the YAML check entry and the `.sql` file — resolved: the YAML entry is a **Check Definition**; the reusable `.sql` file is a **SQL Check Template**.
- ODCS offers both a native `sql` check type and a `custom` check type — resolved: Turbine always serialises a **Check Definition** as an ODCS `custom` check run by the **Turbine Engine**, never as native ODCS `sql`/`library` ([ADR-0002](docs/adr/0002-turbine-quality-checks-serialise-as-odcs-custom-checks.md)).
- "Metadata store" collides with contract-level metadata (owners, tags, schema) in ODCS and across the ecosystem. Briefly renamed to **Run Store** to dodge the clash, then reverted: the engine's persistence layer is the **Metadata Store** again, aligning the canonical term with the code's `metadata_store_*` / `TURBINE_METADATA_*` naming. The overload is handled by qualifying — **Metadata Store** for the engine's DB, **contract metadata** for ODCS fields — not by renaming the concept.
- The **Metadata Store**'s default schema name is `turbine_metadata`; concept and schema name now agree (the brief mismatch existed only while the concept was called `Run Store`).
- "Blocking" conflated two concerns inside the `blocking_dimensions` integration knob: the producer's _severity_ (is a breach a failure or advisory) and the consumer's _blocking_ (does a failure halt my pipeline). Resolved by splitting them — **Check Severity** is producer-authored on the contract; **Blocking** is consumer-side orchestration policy that defaults to severity. Current state: the engine does not yet produce WARNED (`CheckResult.warned()` exists but has no call site), so authored Check Severity is a pending core feature; `blocking_dimensions` is the interim consumer policy until it lands.
- "validation" was used for both checking a contract's YAML document against the ODCS JSON Schema and comparing a contract's columns to a live database — resolved: the static check is **Document Validation**, the live comparison is **Schema Validation**. A 2026-06 cleanup review initially misread the two code paths as duplicate validation systems; the split is intentional.
- "table name" was used to mean both a schema-qualified name (`main.orders`) and a logical table — resolved: a **Data Contract**'s schema names the _logical_ table; the targeted **Server** supplies the schema/database, and that schema is **required** on the Server (per ODCS). A schema-qualified form is a hard error (`schema-on-name`, with a quick-fix that strips the prefix) — the schema lives on the Server alone, never on the name (see [ADR-0005](docs/adr/0005-contracts-bind-to-datasources-via-odcs-servers.md)).
