Metadata-Version: 2.4
Name: sparpartner
Version: 1.5.7
Summary: Deterministic, benchmark-driven stratified sampler for train/test prep.
Author: Henry
Author-email: Henry <osas2henry@gmail.com>
License: All Rights Reserved
Classifier: License :: Other/Proprietary License
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pandas>=1.3
Dynamic: author
Dynamic: license-file
Dynamic: requires-python

# sparpartner

A deterministic stratified sampler for train/test prep. It doesn't
split your data randomly, it deliberately finds the rows that most
resemble a benchmark case, so you can hold those out as a genuine
test set and train on everything else.

## Why this exists

A random train/test split assumes the test set should look
statistically like the train set. That's the wrong question if
what you actually want to know is: **does the model generalize past
one specific profile, or did it just memorize the neighborhood
around it?**

`sparpartner` answers that by ranking every row in your data by how
closely it resembles a benchmark ("bench_marks") you define, then
sorting closest-to-benchmark first. You then slice the sorted frame
yourself:

- **Lookalikes as test, the rest as train**: the rows most like the
  benchmark are always at the top of the sorted result, so `head()`
  gives you the test set and `tail()` (or everything past your cut)
  gives you the train set. This is the harder, more honest check: it
  tells you whether the model actually learned something general, or
  only performs well near cases it's already seen a lot of.
- **Just the lookalikes, nothing else**: pass `sparring_n` and skip
  the manual slice entirely. `sample()` hands you back only the top
  `sparring_n` rows, already ranked, plus a report on exactly that
  set (see the sparring report section below).

`sparpartner` only produces the ranking (and, optionally, the
top-N slice). The actual train/test cut beyond that is a plain
slice on your side (see the usage examples below).

## Where the idea comes from

A fighter in camp doesn't spar with whoever's free in the gym, they
specifically look for a sparring partner who moves, reaches, and
hits like the opponent they're about to face. Training against a
random partner tells you nothing about how you'll actually do;
training against someone who resembles the real threat does.
`sparpartner` applies that same logic to a model: instead of a
random holdout, it finds the rows that resemble the toughest, most
relevant "opponent" profile and holds those back as the real test,
so what's left to train on is everything *unlike* that opponent,
and the test genuinely checks whether the model can handle the
match it's actually walking into.

## How the scoring works

You give it:

- `df`: your data. Any column names are fine, including ones
  starting with `spar`; `sample()` never touches or overwrites
  your own columns (see [Validation](#validation)).
- `bench_marks`: a dict of `{column_name: benchmark_value}`, one
  entry per signal you care about
- `custom_weights`: a **dict** of `{column_name: weight}`, saying
  which columns matter and how much. Insertion order is preserved
  and drives both the `show_progress` readout order and, combined
  with descending weight, the tie-break cascade order (see below).

For each weighted column, `sparpartner` auto-detects the column's
type and scores every row's distance to the benchmark on a 0-1
scale (1.0 = exact match, 0.0 = as far as possible):

| Detected type | How distance is measured |
|---|---|
| **numeric** | `abs(value - bench)`, capped by the column's own max observed distance from bench |
| **date** | both sides converted to "age in days" relative to the benchmark date, capped by the column's own max observed age |
| **string** | exact match = 1, anything else = 0 |

Date detection is automatic. A column is only treated as a date if
its values look date-shaped (contain a separator like `-`, `/`, `.`
or a recognizable month name) **and** parse successfully at least
98% of the time. Bare numeric-looking strings (e.g. `"12345"`) never
even reach the date-parsing attempt, and object-dtype columns of
digit strings fall through to the exact-match string path instead of
being mistaken for numbers. Only a real numeric dtype gets the
numeric path.

Each column's 0-1 score is multiplied by its weight and summed into
one raw score per row. That raw sum is divided by the total weight
to get each row's overall match score, **but only if the total
weight is > 0**. In that normal case the match score always lands in
the 0-1 range, however many signals or weights you used. If the
weights sum to `<= 0`, normalization is skipped entirely and the raw,
unnormalized weighted sum is used instead (not guaranteed to fall in
0-1).

All of this (the per-signal 0-1 scores and the per-row overall
match score) is working state `sample()` uses internally to sort,
tie-break, and slice. **None of it is added as columns to the `df`
you get back.** The df you receive always contains only your
original columns, reordered (and sliced, if `sparring_n` is set).
Score info comes back separately, as aggregates in `sparring_report`
, see [The sparring report](#the-sparring-report).

The result is always sorted by match score descending, the row
closest to the benchmark is always first, this is also exactly what
`best_first=True` means, see
[Row order (`best_first`)](#row-order-best_first) below.

### Tie-breaking

Rows that land on the exact same match score aren't left to random or
arbitrary order. Ties are broken by the per-signal score of the
**highest-weight** signal first (higher wins), then the next-highest,
cascading down the weight-sorted signal list until the tie resolves.
Signals that share the same weight are compared in the order they
appear in `custom_weights` (dict insertion order). Only if every
signal is exhausted and rows are still tied does it fall back to
pandas' stable sort (original row order).

## Row order (`best_first`)

By default (`best_first=True`), the returned `df` is sorted with the
closest match to the benchmark first. Pass `best_first=False` and, as
the very last step before returning, `sample()` flips that same set
of rows so the worst-of-selection is first and the best-of-selection
is last.

This only changes **presentation order**. It never changes which
rows get selected (e.g. via `sparring_n`), never touches scoring or
tie-breaking, and `sparring_report` is identical either way, since
it's an order-independent aggregate.

It exists to replace a manual post-hoc flip like:

```python
result = result.sort_index(ascending=False).reset_index(drop=True)
```

which is fragile against how `sample()`'s own indexing/reset works.
Use `best_first=False` instead when you want the worst-of-selection
row first:

```python
result, report = sample(
    df,
    bench_marks=bench_marks,
    custom_weights=custom_weights,
    best_first=False,
)
```

## The sparring report

`sample()` doesn't just return the ranked frame, it returns a
`(df, sparring_report)` tuple. `sparring_report` is a flat dict
summarizing match quality, with one `spar_<name>` key per signal
plus `spar_min_score` / `spar_max_score` / `spar_mean_score`, e.g.:

```python
{
    "spar_age": 55.0, "spar_income": 57.5,   # one key per signal, avg score
    "spar_min_score": 0.0,
    "spar_max_score": 91.93,
    "spar_mean_score": 56.45,
}
```

Every value is on a 0-100 scale (100 = perfect match to benchmark)
and rounded to 2 decimal places. This is the *only* place score
information comes back to you: individual per-row scores are never
returned, only these aggregates.

A signal with `weight=0` is excluded from ranking/sorting entirely
(it can never move the match score or break a tie), but it still
gets its own `spar_<name>` entry in `sparring_report`, so you can
track how close rows are on a signal without letting that signal
influence which rows are considered "closest".

By default (`sparring_n=None`) both the returned `df` and the report
cover **all** rows. Pass an int and `sample()` slices the sorted
result down to just the top `sparring_n` rows (the ones closest to
the benchmark) **before** anything else happens. That slice is what
you get back as `df`, and it's also exactly what `sparring_report`
and the score distribution are computed on. There's no separate
"full set" kept around once `sparring_n` is set; if you need the
rest of the rows too (e.g. to build the train set), take them from
your original `df` yourself, or call `sample()` again with
`sparring_n=None`.

### Progress readout (`show_progress`)

When `show_progress=True`, the sparring report section of the
printed readout marks each signal's average score, and the min /
max / mean of the score distribution, with a traffic-light emoji:

- 🔴 avg score in the bottom third (0-33.3)
- 🟡 avg score in the middle third (33.3-66.7)
- 🟢 avg score in the top third (66.7-100)

```
    age                  avg score= 64.33  🟡
    signup_date          avg score= 38.00  🟡
    country              avg score= 50.00  🟡

  SCORE DISTRIBUTION
  ------------------
    spar_min_score       =   6.67  🔴
    spar_max_score       =  82.00  🟢
    spar_mean_score      =  50.78  🟡
```

This is purely a print-time visual, it doesn't change anything about
`sparring_report`'s actual values.

## Usage

### Sample usage

`df` is the only argument you can pass positionally. Every other
argument, including `bench_marks`, `custom_weights`, and
`best_first`, must be passed by keyword (see
[Keyword-only arguments](#keyword-only-arguments) below).

```python
import pandas as pd
from sparpartner import sample

df = pd.DataFrame({
    "id": [1, 2, 3, 4, 5],
    "age": [25, 30, 47, 52, 33],
    "signup_date": ["2023-01-15", "2023-03-02", "2022-11-20", "2023-01-10", "2023-06-01"],
    "country": ["US", "US", "CA", "US", "MX"],
})

bench_marks = {
    "age": 30,
    "signup_date": "2023-01-01",
    "country": "US",
}

custom_weights = {
    "age": 2,
    "signup_date": 1,
    "country": 1,
}

result, sparring_report = sample(
    df,
    bench_marks=bench_marks,
    custom_weights=custom_weights,
    best_first=True,      # required, no default, True = best match first; False = worst-of-selection first
    sparring_n=None,      # None = every row scored, sorted, and returned
    drop_nan=True,
    show_progress=True,   # prints the full scoring breakdown
)

print(result)
print(sparring_report)
```

`age=30`, `signup_date="2023-01-01"`, and `country="US"` closely
match row `id=1` (age 25, close date, US), so that row lands at or
near the top of the sorted output (or the bottom, if
`best_first=False`).

```python
# --- post-sample: turn the ranking into an actual train/test split ---
# Result is always sorted closest-to-benchmark first (default best_first=True).
ranked, _ = sample(
    df,
    bench_marks=bench_marks,
    custom_weights=custom_weights,
    best_first=True,
)

# Slice however large you want the test set to be, e.g. the top 30%:
cut = int(len(ranked) * 0.3)
test = ranked.iloc[:cut]    # lookalikes, the harder, honest test set
train = ranked.iloc[cut:]   # everything unlike the benchmark

# Or skip the manual slice and get the test set directly:
test_only, report = sample(
    df,
    bench_marks=bench_marks,
    custom_weights=custom_weights,
    best_first=True,
    sparring_n=cut,
)
```

### Parameters

| Name | Type | Default | What it does |
|---|---|---|---|
| `df` | DataFrame | required, positional | Must contain a column for every name (key) in `custom_weights`. No restriction on your own column names, `spar`-prefixed columns are fine |
| `bench_marks` | dict | `None`, but required (raises if left `None`) | `{column_name: benchmark_value}`. Keyword-only |
| `custom_weights` | dict of `{name: weight}` | `None`, but required (raises if left `None`) | Which columns to score and how much each contributes. Keyword-only |
| `best_first` | bool | `None`, but required (raises if left `None`) | Applied last, after everything else (including `sparring_n` slicing). `True` = best match first. `False` = flips that same set of rows to worst-of-selection first, best last. Never changes which rows are selected; see [Row order](#row-order-best_first). Keyword-only |
| `sparring_n` | int or `None` | `None` | `None` = every row scored, sorted, and returned, report covers all of them. An int slices the sorted result down to the top `sparring_n` rows **before** anything else. That slice is what's returned as `df`, and what the report/distribution are computed on. Keyword-only |
| `drop_nan` | bool | `True` | If `True`, drops any row with a NaN in a per-signal working score **or** in the overall match score, after printing (if `show_progress`) a sanity check of what was dropped and why. Keyword-only |
| `show_progress` | bool | `False` | Prints a full readout, in run order: header (including a `signals used` count), per-column type/cap/sample scores, drop_nan check (if `drop_nan=True`), normalize check, sort-apply readout, sparring_n slice readout (if `sparring_n` is set), top N ranked rows with weighted contributions (`N` in the header always matches the number of rows actually shown), and the sparring report itself, with 🔴/🟡/🟢 markers next to each score, flagging any signal contributing zero separation. Keyword-only |

### Returns

`sample()` returns a `(df, sparring_report)` tuple, not just a
DataFrame. The `df` always contains only your original columns
(reordered/sliced), no score columns are ever attached to it. See
[The sparring report](#the-sparring-report) for how score
information comes back to you instead.

### Validation

Input validation runs upfront, before any scoring starts, in two
passes: general parameter checks, then `custom_weights` checks.

Raises `TypeError` if:
- `df` isn't a pandas DataFrame
- `bench_marks` isn't a dict
- `custom_weights` isn't a dict
- `drop_nan`, `show_progress`, or `best_first` isn't a bool
- `sparring_n` isn't an int or `None` (bools are rejected too)

Raises `ValueError` if:
- `bench_marks` is left as `None` (its default)
- `custom_weights` is left as `None` (its default)
- `best_first` is left as `None` (its default)
- `df` has no rows
- `custom_weights` is an empty dict
- `sparring_n` isn't a positive integer
- a `custom_weights` key isn't a string, or doesn't match a column
  in `df`
- a `custom_weights` key has no matching entry in `bench_marks`
- a `custom_weights` key is literally named `"score"`. `sample()`
  keeps its own overall-total working score internally, and a
  signal named `"score"` would generate the exact same internal name,
  corrupting that total instead of just shadowing a per-signal value.
  Rename that column in `df` (and its entries in
  `bench_marks`/`custom_weights`) before calling `sample()`. Names
  that merely *contain* "score", like `test_score` or `score_pct`,
  are unaffected, only an exact match on `"score"` collides
- a weight isn't numeric (bools are rejected too, a `bool` is
  technically an `int` in Python but was never meant as a weight)
- a weight is `NaN`, `inf`, or `-inf`

**Note on duplicate signal names:** since `custom_weights` is now a
dict, keys are inherently unique, so a repeated column name can no
longer be passed in the first place, Python itself resolves a
repeated key in a dict literal (keeping only the last value) before
`sample()` ever sees it. There's nothing left for validation to
catch here.

## A couple of things worth knowing

- **Your own column names are unrestricted**: `sample()` computes
  its working scores under internally-generated names that can't
  collide with anything you'd realistically name a column, and those
  working columns are always dropped before the df is returned. You
  can freely have your own columns named `spar_score`, `spar_age`,
  or anything else, `sample()` won't touch, rename, or overwrite
  them.
- **A signal literally named `"score"`**: still rejected, the one
  case where a *signal name itself* (not a pre-existing df column)
  would collide with `sample()`'s own internal total. See
  [Validation](#validation) above.
- **Object-dtype numeric strings**: a column of strings like
  `"100"`, `"200"` (object dtype, no separator) is scored as an
  exact-match string column, *not* auto-converted to numeric. Only
  genuine numeric dtypes (`int`, `float`) get the numeric distance
  path.
- **Keyword-only arguments**: `df` is the only argument `sample()`
  accepts positionally. Every other argument, `bench_marks`,
  `custom_weights`, `best_first`, `sparring_n`, `drop_nan`, and
  `show_progress`, must be passed by name. This is enforced by
  Python itself: a positional call like `sample(df, weights,
  benchmarks)` fails immediately with a `TypeError`, before any of
  `sample()`'s own code runs. It exists specifically to rule out
  accidentally swapping `bench_marks` and `custom_weights`, which are
  both dicts and can't be told apart by type alone. `bench_marks`,
  `custom_weights`, and `best_first` additionally default to `None`
  but are not actually optional, leaving any of them out (or passing
  `None` explicitly) raises a `ValueError` naming exactly which one
  is missing.
