# Release 0.8.398

!!!tip inline end "Need Help?"
    The SuperCowPowers team is happy to give any assistance needed when setting up AWS and Workbench. So please contact us at [workbench@supercowpowers.com](mailto:workbench@supercowpowers.com) or on chat us up on [Discord](https://discord.gg/WHAJuz8sw8)

The headline of this release is **first-class validation-set support** for model training. Along the way, `sample_weight` is **de-overloaded** into three explicit, orthogonal row roles — so designating a held-out set, dropping an outlier, and weighting a row are no longer the same knob.

## First-class validation sets

A model can now designate a **held-out validation set** that the training container sees *during* training: the rows are kept in the training view, routed out of the train/CV path, and scored as an honest held-out read (`holdout_mae`). This works across all three model frameworks (XGBoost, PyTorch, ChemProp) and unblocks in-training validation and HPO.

```python
fs.to_model(
    name="my-model",
    model_type=ModelType.REGRESSOR,
    model_framework=ModelFramework.XGBOOST,
    target_column="activity",
    validation_ids=holdout_ids,   # scored, never trained
)
```

Held-out predictions land in `validation_predictions.csv` with a `validation` column (`True` for held-out rows), and the primary-target held-out MAE is logged as `holdout_mae`.

## Row roles: `sample_weight`, `validation`, `exclude`

Previously, `sample_weight == 0` did triple duty — a framework weight, an "exclude this row" flag, *and* a holdout marker. These are now separate:

| role | how to set | in training data? | trains? | scored held-out? |
|---|---|---|---|---|
| **weight** | `sample_weights={id: w}` | yes | yes, at weight `w` | via CV |
| **exclude** (outlier/anomaly) | `exclude_ids=[...]` | **no** (dropped) | no | no |
| **validation** (holdout) | `validation_ids=[...]` | yes, marked | no | yes |

`sample_weight` is now a **pure framework weight**, forwarded as-is to the model script. `exclude` wins if an id is in both `exclude_ids` and `validation_ids`.

!!!warning "Behavior change: `sample_weight == 0` no longer drops rows"
    Zero-weight rows used to be silently excluded from the training view. That is **no longer true** — a weight of `0` is now just a zero framework weight. Use `exclude_ids` to drop rows and `validation_ids` to hold them out. See the porting guide below. Existing models are unaffected until they retrain (a model's script and training view are co-created and co-owned, so migration is per-model).

## Porting guide: `sample_weights={id: 0.0}` → roles

If you designated rows by zero-weighting them, pick the role that matches your intent:

**Excluding outliers / anomalies** (rows that should never be seen):

```python
# before
sample_weights = {row_id: 0.0 for row_id in bad_ids}
fs.to_model(..., sample_weights=sample_weights)

# after
fs.to_model(..., exclude_ids=bad_ids)
```

**Holding rows out for a scored validation read** (the PXR / temporal pattern):

```python
# before — zero-weight the holdout, score it later via endpoint.inference
sample_weights = {row_id: 0.0 for row_id in holdout_ids}
fs.to_model(..., sample_weights=sample_weights)

# after — held out AND scored in-training (holdout_mae)
fs.to_model(..., validation_ids=holdout_ids)
```

**Actual fractional weighting is unchanged** — non-zero weights still work exactly as before:

```python
fs.to_model(..., sample_weights={row_id: 0.5 for row_id in down_weighted})
```

### `temporal_split` now returns a list

`FeatureSet.temporal_split()` returns a **list of holdout ids** (was a `{id: 0.0}` dict), to feed `validation_ids` directly:

```python
# before
sample_weights = fs.temporal_split("date_col", end_date="2025-10-17")   # {id: 0.0}
fs.to_model(..., sample_weights=sample_weights)

# after
validation_ids = fs.temporal_split("date_col", end_date="2025-10-17")   # [id, id, ...]
fs.to_model(..., validation_ids=validation_ids)
```

### Combined pattern (temporal holdout + anomaly exclusion)

The matched pair — designate the holdout at `to_model`, confirm on the endpoint with `ts_inference` — still holds; the roles just become explicit:

```python
exclude_ids = list(compute_sample_weights(...))          # anomalies → dropped
validation_ids = fs.temporal_split("date_col", end_date="2025-10-17")
model = fs.to_model(..., exclude_ids=exclude_ids, validation_ids=validation_ids)
...
end.ts_inference("date_col", after_date="2025-10-17", exclude_ids=exclude_ids)  # unchanged
```

Anomalies that fall in the post-date window get `exclude` precedence — dropped from both training and the held-out read.

## Ray / awswrangler coexistence

Workbench now pins **awswrangler to its single-node Python engine** at import. With `ray` present in an environment (e.g. HPO's `ray[tune]`), awswrangler would otherwise auto-switch to its distributed engine and break the single-node boto3 calls Workbench relies on for Athena/S3. Ray Tune is unaffected — it's an independent consumer of Ray.

## Upgrade notes

* **Training images must be rebuilt** on this workbench version — the templates import the new `workbench.training.validation` util (`base` and `pytorch_chem` training images).
* No client code changes are required unless you relied on `sample_weight == 0` to drop rows — see the porting guide.

## Questions?
<img align="right" src="../../images/scp.png" width="180">

The SuperCowPowers team is happy to answer any questions you may have about AWS and Workbench. Please contact us at [workbench@supercowpowers.com](mailto:workbench@supercowpowers.com) or on chat us up on [Discord](https://discord.gg/WHAJuz8sw8)
</content>
</invoke>
