# Changelog

All notable changes to Turbine are documented here. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and the project adheres to [Semantic Versioning](https://semver.org/).

## [Unreleased]

### Breaking Changes

- `generate api` no longer accepts `--include-drafts` / `--include-proposed`. Every contract now generates an API regardless of lifecycle status, so the opt-in flags have no purpose and are removed; passing them now errors as unknown flags.

### Added

- `turbine sync api` reconciles every already-generated API with its current contract: it regenerates each in place to pick up contract changes and removes the ones whose contract no longer exists. It never adds an API for a contract that was never generated (that stays `turbine generate api`), aborts on a contract that fails to validate rather than pruning it, and prints a hint to run `generate` when nothing is generated yet.
- `validate` now prints a "validated" summary below the drift trace: a tree of every column it checked, naming the facets compared (type, nullability) and marking each green, yellow, or red, with a footer tallying verified, warned, and drifted columns. Clean validations now show what was checked instead of nothing.
- Snowflake key-pair auth now supports passphrase-protected private keys via `SF_PRIVATE_KEY_PASSPHRASE`. An encrypted key with a missing or wrong passphrase now raises a clear datasource-config diagnostic instead of a cryptic library error.
- `generate api` now advises, on stderr, when a contract tags no column `filterable`. Because the API then falls back to exposing every column as a filter, the advisory points at the contract file and shows how to add a `filterable` tag. It is informational only and never fails the command.
- `validate`, `lint`, and editor diagnostics now require the governance metadata a data product needs before it is published: a `team` with at least one member whose role is exactly `Data Product Owner` (every member needs a username, name, and role), a non-empty `support` channel (channel, tool, scope, and url), and all four classifications as `customProperties` — availability `B0`–`B3`, integrity `I0`–`I3`, confidentiality `V0`–`V3`, and privacy `P0`–`P3`. A contract that omits an owner, ships an incomplete support entry, or carries a mistyped classification code now goes red everywhere, with the message pointing at the field. Extra team members, support channels, and custom properties stay valid.
- A quick-fix adds each missing governance field as an editable snippet. Choosing it inserts a `team` block with a Data Product Owner, a `support` channel, or the four classifications with their allowed values as tab-through choices. Scaffolded and generated contracts deliberately leave these fields out so the gaps stay visible, and the quick-fixes fill them in.
- `turbine publish` publishes local Data Contracts to the Marketplace by POSTing their exact YAML. Selection matches `lint`/`validate` (a picker with no arguments, `--all`, a single path or id, or several ids). A picker or `--all` run publishes every lint-clean contract and skips the rest, reporting which were skipped; naming contracts explicitly is strict, so a typo or a lint failure in a named contract publishes nothing. Publishing is best-effort across the batch — one API failure does not stop the others — and the command exits success only when every selected contract linted clean and every upload succeeded. `--format json` emits `published`, `failed`, and `skipped` entries. Credentials come from `TURBINE_MARKETPLACE_API_KEY`, with an optional `TURBINE_MARKETPLACE_BASE_URL`.

### Changed

- `generate api` and `sync api` now skip contracts that fail lint instead of aborting the whole run. One broken contract no longer blocks every valid one from generating; the broken contracts' errors are still shown, a warning names which were skipped, and the command exits success after building the rest. When anything is skipped the run also stops pruning, so a skipped contract's already-generated API is preserved rather than swept away. Naming a broken contract explicitly with `--contract` still fails.
- The `generate api` / `sync api` summary now reports removed files. Generated files no longer backed by a contract are counted on a `removed:` line and flip the run to a warning, so a contract change that deletes an API is visible instead of silent.
- Starting the API server with a missing or broken generated API now prints a plain "run `turbine generate api`" message instead of serving no data endpoints silently (missing case) or a bare error (broken case). The hint also rides along on the broken-mount diagnostics.
- `generate api` now warns when the contract selection is empty — for example when nothing is picked interactively — instead of reporting success with no output. Previously only the `--all` path warned.
- The scaffolded starter contracts now tag a representative subset of columns `filterable` (the bare and demo templates, and the cookiecutter starter), so a freshly generated API shows a curated filter set and teaches the tag by example.
- `generate api` is now status-blind: a contract's lifecycle status (draft, proposed, active, deprecated, retired) no longer decides whether it gets an API — every contract generates. Status now affects only the runtime response headers: a served draft, proposed, or retired endpoint advertises itself with `X-Contract-Status`, and a deprecated one still emits `Deprecation` / `Sunset`. This lets a draft contract be turned into a valid, fully-served API without any opt-in.
- `check`'s full output is now a result tree grouped by data quality dimension — each check a green or red leaf with its counts — replacing the per-dimension tables. The score panel is unchanged.
- CLI errors now print to stderr instead of stdout. A configuration error (such as a missing or wrong `--datasource`, e.g. an unfilled CI/CD variable) no longer corrupts the machine report file on stdout and instead shows up in the CI job log.

### Removed

- Removed generated API rate-limit middleware and its `TURBINE_RATE_LIMIT_*` environment configuration.
- Moved dashboard scaffolding from `turbine new dashboard` to `turbine generate dashboard`.
- Removed `turbine init --demo`; editor onboarding can still scaffold its tour workspace through the LSP onboarding flow.
- Removed LSP empty-file completion scaffolds for contracts, quality specs, and datasources; CodeLens scaffold actions remain available.
- Removed the bundled dashboard demo backend and `--backend demo`; `turbine dashboard` now requires a Turbine project with a scaffolded dashboard and uses the API/custom backend path.
- Removed the `junit` and `codequality` output formats. `lint`, `check`, `validate`, and `ci diff` now offer `full`, `concise`, and `json`. Generated GitLab CI pipelines run `turbine lint`/`check` with default output (findings shown directly in the job log) instead of redirecting a machine artifact into a report file.
- Removed the unused `ci publish` placeholder, which never published anything. Use `turbine publish` instead; `ci diff` is unchanged.

### Fixed

- A type error inside a turbine custom check's `implementation` (for example `mustBe` given text instead of a number) no longer triggers spurious "unknown field `engine`/`implementation` — did you mean `dimension`?" diagnostics. The one real error is now reported on its own instead of cascading into misleading unknown-field suggestions.
- `generate api` now emits separate routes for the same contract/table across different major versions instead of skipping v2+ as duplicate tables.
- Installing `dbt-core` / `dbt-snowflake` alongside `turbine-data` no longer silently downgrades Turbine to an old release. Turbine pinned `pathspec>=1.1.1` while dbt requires `pathspec<0.13`, an unsatisfiable conflict that forced the resolver back to turbine-data 0.5.2. The pin is relaxed to `pathspec>=0.12` (Turbine only uses `GitIgnoreSpec.from_lines`, available since 0.10), so dbt and the current Turbine resolve together.
- The `pct_change` and `spike` anomaly checks now run on Snowflake. They build on `LAG`, which Snowflake rejects when a window frame is attached; the frame is now omitted (it never affected the result, since `LAG` always reads the previous row).
- An `slaProperties` `element` pointing at a JSON column whose name contains dots (a flattened path such as `measurement.data.lastTime`) no longer reports a spurious "must not carry a schema" error, and its freshness/latency/retention/frequency checks are no longer silently dropped. Both the contract lint and the ODCS check-derivation path now resolve SLA elements against the contract's declared table and column names instead of counting dots, so a dotted column name is matched verbatim (and derives a check against the full column, not a truncated last segment) while a genuine `schema.table.column` is still flagged.
- `pct_change` now measures the right thing. The percentage was rendered as `ABS(value - prev / prev)` — i.e. `value - 1` — instead of `ABS((value - prev) / prev)`, so it flagged the wrong rows on every engine.
- `check` now shows the actual diagnostics when a contract is skipped for lint errors, not just `failed lint (N error(s))`. A multi-contract run previously hid which column or rule failed.
- A Python check whose function raises now reports as `errored` with the exception message, instead of being silently `skipped` (which also dropped it from the quality score).
- Double-quoted strings in contracts and quality specs now decode their escapes. A regex such as `"^[A-Z]+\\.[0-9]+$"` was read with the backslash doubled, so it matched nothing and every row failed; double-quoted escapes (`\\`, `\n`, `\"`, ...) are now applied as YAML requires.
- Regex checks (`validRegex`, `validFormat`, `invalidRegex`) now flag the same rows on Snowflake as on Postgres and DuckDB. Snowflake's `REGEXP_LIKE` demands a full-string match, so an unanchored pattern behaved differently there; it is now rendered as a partial search to match the other engines.
- A `threshold` supplied to a Python check through the spec's `params` is no longer overwritten by the check's own threshold value — explicit params win.
- Row flagging on a Python check now works on Snowflake. PK extraction assumed lowercase DataFrame columns, so Snowflake's uppercase columns raised `KeyError` and errored the check; the PK is now resolved case-insensitively.
- A scoped check run (`--incremental` / `--since` / `--until`) with `--flag` no longer clears row flags for data outside its window. The run only scanned part of the table, so it now only adds flags for the rows it tested; a full run still reconciles and clears fixed rows.
