Metadata-Version: 2.4
Name: sd-auto-encoder
Version: 0.3.0
Summary: auto-picks onehot/ordinal/binary encoding per categorical column
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pandas>=1.2
Dynamic: license-file

# SDEncoder

Auto-picks an encoding method per categorical column instead of you manually
deciding onehot vs ordinal vs label encoding for each one. Companion to
SDScaler, same idea but for categorical data instead of numeric.

Two things in here:
- `SDEncoder` - for feature columns (X)
- `SDTargetEncoder` - for your target column (y)

They're kept separate on purpose. Label-style integer codes are fine for a
target (a model just needs distinct class ids, order doesn't matter), but
doing that on a *feature* column tricks a lot of models into thinking there's
a numeric relationship between categories that don't actually have one.
`SDEncoder` will refuse to touch a Series for this reason, and `SDTargetEncoder`
will refuse a DataFrame.

## Install

```
cd SDEncoder
pip install -e .
```

## Usage

```python
import pandas as pd
from sdencoder import SDEncoder, SDTargetEncoder

df = pd.DataFrame({
    "color": ["red", "blue", "green", "blue"],
    "size": ["S", "M", "L", "M"],
    "is_member": ["yes", "no", "yes", "yes"],
})

encoder = SDEncoder(ordinal_maps={"size": ["S", "M", "L"]})
X_encoded = encoder.fit_transform(df)
print(X_encoded)
print(encoder.summary())
```

For the target:

```python
y = pd.Series(["approved", "denied", "approved"])
target_encoder = SDTargetEncoder()
y_encoded = target_encoder.fit_transform(y)
```

Same train/test rule as SDScaler - fit on train, just transform on test:

```python
X_train_enc = encoder.fit_transform(X_train)
X_test_enc = encoder.transform(X_test)
```

## How SDEncoder decides

For each object/category column:
- if you gave it an `ordinal_maps` entry for that column -> uses your order
- 2 categories -> binary (0/1)
- more than 2 but under `onehot_cutoff` (default 10) -> onehot
- more than `onehot_cutoff` -> falls back to ordinal, since onehot on
  something like a "city" column with 200 values would blow up your
  feature count for not much benefit

Numeric columns are left completely alone.

`ordinal_maps` is the important one to use correctly - only pass a real
order there for columns that actually have one (like S/M/L, or
low/medium/high). Don't use it just to avoid getting a onehot column.

## Unseen categories

If `.transform()` sees a category that wasn't there during `.fit()`, it
becomes `NaN` for ordinal/binary columns, or an all-zero row for onehot
columns. That's intentional - better to notice missing data than have it
silently mapped to some made-up number.

## ColumnTransformer

If you'd rather use this inside an sklearn `Pipeline`:

```python
encoder = SDEncoder(ordinal_maps={"size": ["S", "M", "L"]}).fit(df)
ct = encoder.to_column_transformer()
```

This just builds an sklearn `ColumnTransformer` using the same decisions
SDEncoder already made. Needs scikit-learn installed, only used if you
actually call this method.

## Files

```
sdencoder/
  encoder.py   - SDEncoder, for X
  target.py    - SDTargetEncoder, for y
examples/
  example_usage.py
tests/
  test_encoder.py
```

## Todo / ideas

- frequency encoding as another fallback option for high-cardinality columns
- inverse_transform for onehot columns (currently raises NotImplementedError)
- handle missing values (NaN) as their own category instead of erroring later
