Metadata-Version: 2.4
Name: sparpartner
Version: 1.5.4
Summary: Deterministic, benchmark-driven stratified sampler for train/test prep.
Author: Henry
Author-email: Henry <osas2henry@gmail.com>
License: All Rights Reserved
Classifier: License :: Other/Proprietary License
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pandas>=1.3
Dynamic: author
Dynamic: license-file
Dynamic: requires-python

# sparpartner

A deterministic stratified sampler for train/test prep. It doesn't
split your data randomly, it deliberately finds the rows that most
resemble a benchmark case, so you can hold those out as a genuine
test set and train on everything else.

## Why this exists

A random train/test split assumes the test set should look
statistically like the train set. That's the wrong question if
what you actually want to know is: **does the model generalize past
one specific profile, or did it just memorize the neighborhood
around it?**

`sparpartner` answers that by ranking every row in your data by how
closely it resembles a benchmark ("bench_marks") you define, then
sorting closest-to-benchmark first. You then slice the sorted frame
yourself:

- **Lookalikes as test, the rest as train**: the rows most like the
  benchmark are always at the top of the sorted result, so `head()`
  gives you the test set and `tail()` (or everything past your cut)
  gives you the train set. This is the harder, more honest check: it
  tells you whether the model actually learned something general, or
  only performs well near cases it's already seen a lot of.
- **Just the lookalikes, nothing else**: pass `sparring_n` and skip
  the manual slice entirely. `sample()` hands you back only the top
  `sparring_n` rows, already ranked, plus a report on exactly that
  set (see the sparring report section below).

`sparpartner` only produces the ranking (and, optionally, the
top-N slice). The actual train/test cut beyond that is a plain
slice on your side (see the usage examples below).

## Where the idea comes from

A fighter in camp doesn't spar with whoever's free in the gym, they
specifically look for a sparring partner who moves, reaches, and
hits like the opponent they're about to face. Training against a
random partner tells you nothing about how you'll actually do;
training against someone who resembles the real threat does.
`sparpartner` applies that same logic to a model: instead of a
random holdout, it finds the rows that resemble the toughest, most
relevant "opponent" profile and holds those back as the real test,
so what's left to train on is everything *unlike* that opponent,
and the test genuinely checks whether the model can handle the
match it's actually walking into.

## How the scoring works

You give it:

- `df`: your data. Must not already contain any column starting
  with `spar` (see [Validation](#validation)).
- `bench_marks`: a dict of `{column_name: benchmark_value}`, one
  entry per signal you care about
- `custom_weights`: a **dict** of `{column_name: weight}`, saying
  which columns matter and how much. Insertion order is preserved
  and drives both the `show_progress` readout order and, combined
  with descending weight, the tie-break cascade order (see below).

For each weighted column, `sparpartner` auto-detects the column's
type and scores every row's distance to the benchmark on a 0–1
scale (1.0 = exact match, 0.0 = as far as possible):

| Detected type | How distance is measured |
|---|---|
| **numeric** | `abs(value - bench)`, capped by the column's own max observed distance from bench |
| **date** | both sides converted to "age in days" relative to the benchmark date, capped by the column's own max observed age |
| **string** | exact match = 1, anything else = 0 |

Date detection is automatic. A column is only treated as a date if
its values look date-shaped (contain a separator like `-`, `/`, `.`
or a recognizable month name) **and** parse successfully at least
98% of the time. Bare numeric-looking strings (e.g. `"12345"`) never
even reach the date-parsing attempt, and object-dtype columns of
digit strings fall through to the exact-match string path instead of
being mistaken for numbers. Only a real numeric dtype gets the
numeric path.

Each column's 0–1 score is stored as its own `spar_<name>` column,
multiplied by its weight, and summed into one raw score per row.
That raw sum is divided by the total weight to get the final
`spar_score`, **but only if the total weight is > 0**. In that
normal case `spar_score` always lands in the 0–1 range, however many
signals or weights you used. If the weights sum to `<= 0`,
normalization is skipped entirely and `spar_score` is left as the
raw, unnormalized weighted sum (not guaranteed to fall in 0–1).

The result is always sorted `spar_score` descending, the row closest
to the benchmark is always first, unless you pass `best_first=True`,
see [Row order (`best_first`)](#row-order-best_first) below.

### Tie-breaking

Rows that land on the exact same `spar_score` aren't left to random or
arbitrary order. Ties are broken by the per-signal score of the
**highest-weight** signal first (higher wins), then the next-highest,
cascading down the weight-sorted signal list until the tie resolves.
Signals that share the same weight are compared in the order they
appear in `custom_weights` (dict insertion order). Only if every
signal is exhausted and rows are still tied does it fall back to
pandas' stable sort (original row order).

## Row order (`best_first`)

By default (`best_first=False`), the returned `df` is sorted with the
closest match to the benchmark first, same as always. Pass
`best_first=True` and, as the very last step before returning,
`sample()` flips that same set of rows so the worst-of-selection is
first and the best-of-selection is last.

This only changes **presentation order**. It never changes which
rows get selected (e.g. via `sparring_n`), never touches scoring or
tie-breaking, and `sparring_report` is identical either way, since
it's an order-independent aggregate.

It exists to replace a manual post-hoc flip like:

```python
result = result.sort_index(ascending=False).reset_index(drop=True)
```

which is fragile against how `sample()`'s own indexing/reset works.
Use `best_first=True` instead:

```python
result, report = sample(df, bench_marks, custom_weights, best_first=True)
```

## The sparring report

`sample()` doesn't just return the ranked frame, it returns a
`(df, sparring_report)` tuple. `sparring_report` is a flat dict
summarizing match quality, with one `spar_<name>` key per signal
plus `spar_min_score` / `spar_max_score` / `spar_mean_score`, e.g.:

```python
{
    "spar_age": 55.0, "spar_income": 57.5,   # one key per signal, avg score
    "spar_min_score": 0.0,
    "spar_max_score": 91.93,
    "spar_mean_score": 56.45,
}
```

Every value is on a 0–100 scale (100 = perfect match to benchmark)
and rounded to 2 decimal places, regardless of `return_score`.

A signal with `weight=0` is excluded from ranking/sorting entirely
(it can never move `spar_score` or break a tie), but it still gets
its own `spar_<name>` column and report entry, so you can track how
close rows are on a signal without letting that signal influence
which rows are considered "closest".

By default (`sparring_n=None`) both the returned `df` and the report
cover **all** rows. Pass an int and `sample()` slices the sorted
result down to just the top `sparring_n` rows (the ones closest to
the benchmark) **before** anything else happens. That slice is what
you get back as `df`, and it's also exactly what `sparring_report`
and the score distribution are computed on. There's no separate
"full set" kept around once `sparring_n` is set; if you need the
rest of the rows too (e.g. to build the train set), take them from
your original `df` yourself, or call `sample()` again with
`sparring_n=None`.

## Usage

### Sample usage

```python
import pandas as pd
from sparpartner import sample

df = pd.DataFrame({
    "id": [1, 2, 3, 4, 5],
    "age": [25, 30, 47, 52, 33],
    "signup_date": ["2023-01-15", "2023-03-02", "2022-11-20", "2023-01-10", "2023-06-01"],
    "country": ["US", "US", "CA", "US", "MX"],
})

bench_marks = {
    "age": 30,
    "signup_date": "2023-01-01",
    "country": "US",
}

custom_weights = {
    "age": 2,
    "signup_date": 1,
    "country": 1,
}

result, sparring_report = sample(
    df,
    bench_marks,
    custom_weights,
    sparring_n=None,      # None = every row scored, sorted, and returned
    drop_nan=True,
    show_progress=True,   # prints the full scoring breakdown
    return_score=True,    # keep spar_score / spar_<name> columns in the output
    best_first=False,     # False (default) = best match first; True = worst-of-selection first
)

print(result)
print(sparring_report)
```

`age=30`, `signup_date="2023-01-01"`, and `country="US"` closely
match row `id=1` (age 25, close date, US), so that row lands at or
near the top of the sorted output (or the bottom, if
`best_first=True`).

```python
# --- post-sample: turn the ranking into an actual train/test split ---
# Result is always sorted closest-to-benchmark first (default best_first=False).
ranked, _ = sample(df, bench_marks, custom_weights)

# Slice however large you want the test set to be, e.g. the top 30%:
cut = int(len(ranked) * 0.3)
test = ranked.iloc[:cut]    # lookalikes, the harder, honest test set
train = ranked.iloc[cut:]   # everything unlike the benchmark

# Or skip the manual slice and get the test set directly:
test_only, report = sample(df, bench_marks, custom_weights, sparring_n=cut)
```

### Parameters

| Name | Type | Default | What it does |
|---|---|---|---|
| `df` | DataFrame | required | Must contain a column for every name (key) in `custom_weights`, and must not already have any column starting with `spar` |
| `bench_marks` | dict | required | `{column_name: benchmark_value}` |
| `custom_weights` | dict of `{name: weight}` | required | Which columns to score and how much each contributes |
| `sparring_n` | int or `None` | `None` | `None` = every row scored, sorted, and returned, report covers all of them. An int slices the sorted result down to the top `sparring_n` rows **before** anything else. That slice is what's returned as `df`, and what the report/distribution are computed on |
| `drop_nan` | bool | `True` | If `True`, drops any row with a NaN in a per-signal `spar_<name>` column **or** in `spar_score` itself, after printing (if `show_progress`) a sanity check of what was dropped and why |
| `show_progress` | bool | `False` | Prints a full readout, in run order: header (including a `signals used` count), per-column type/cap/sample scores, drop_nan check (if `drop_nan=True`), normalize check, sort-apply readout, sparring_n slice readout (if `sparring_n` is set), top N ranked rows with weighted contributions (`N` in the header always matches the number of rows actually shown), and the sparring report itself, flagging any signal contributing zero separation |
| `return_score` | bool | `False` | If `True`, keeps `spar_score` and `spar_<name>` columns in the returned df instead of dropping them |
| `best_first` | bool | `False` | Applied last, after everything else (including `sparring_n` slicing). `False` = best match first (default). `True` = flips that same set of rows to worst-of-selection first, best last. Never changes which rows are selected; see [Row order](#row-order-best_first) |

### Returns

`sample()` returns a `(df, sparring_report)` tuple, not just a
DataFrame. See the sparring report section above.

### Validation

Input validation runs upfront, before any scoring starts, in two
passes: general parameter checks, then `custom_weights` checks.

Raises `TypeError` if:
- `df` isn't a pandas DataFrame
- `bench_marks` isn't a dict
- `custom_weights` isn't a dict
- `drop_nan`, `show_progress`, `return_score`, or `best_first` isn't a bool
- `sparring_n` isn't an int or `None` (bools are rejected too)

Raises `ValueError` if:
- `df` has no rows
- `df` already has one or more columns starting with `spar`; these
  would collide with the `spar_score`/`spar_<name>` columns `sample()`
  creates internally, silently overwriting them and producing
  unexpected output. The error names every offending column; rename
  them in `df` before calling `sample()`
- `custom_weights` is an empty dict
- `sparring_n` isn't a positive integer
- a `custom_weights` key isn't a string, or doesn't match a column
  in `df`
- a `custom_weights` key has no matching entry in `bench_marks`
- a `custom_weights` key is literally named `"score"`. That name
  would generate an internal column named `spar_score`, which
  collides with the `spar_score` total column `sample()` keeps for
  the overall weighted result, corrupting it instead of just
  overwriting a per-signal column. Rename that column in `df` (and
  its entries in `bench_marks`/`custom_weights`) before calling
  `sample()`. Names that merely *contain* "score", like
  `test_score` or `score_pct`, are unaffected, only an exact match
  on `"score"` collides
- a weight isn't numeric (bools are rejected too, a `bool` is
  technically an `int` in Python but was never meant as a weight)
- a weight is `NaN`, `inf`, or `-inf`

**Note on duplicate signal names:** since `custom_weights` is now a
dict, keys are inherently unique, so a repeated column name can no
longer be passed in the first place, Python itself resolves a
repeated key in a dict literal (keeping only the last value) before
`sample()` ever sees it. There's nothing left for validation to
catch here.

## A couple of things worth knowing

- **Reserved column names**: don't bring your own `spar`-prefixed
  columns into `df`; `sample()` uses that prefix internally
  (`spar_score`, `spar_<name>`) and will raise rather than risk
  silently overwriting one of your columns.
- **A signal literally named `"score"`**: also rejected, for the
  same reason, even though it doesn't start with `spar`. See
  [Validation](#validation) above.
- **Object-dtype numeric strings**: a column of strings like
  `"100"`, `"200"` (object dtype, no separator) is scored as an
  exact-match string column, *not* auto-converted to numeric. Only
  genuine numeric dtypes (`int`, `float`) get the numeric distance
  path.
- **Positional calls**: if you call `sample()` positionally beyond
  `custom_weights`, the parameter order is `sparring_n`, `drop_nan`,
  `show_progress`, `return_score`, `best_first`. Use keyword
  arguments for anything past the first three positional params to
  avoid ambiguity.
