clover.over_sampling.KMeansSMOTE

class clover.over_sampling.KMeansSMOTE(sampling_strategy='auto', random_state=None, k_neighbors=5, kmeans_estimator=None, imbalance_ratio_threshold='auto', distances_exponent='auto', raise_error=True, n_jobs=None)[source]

Applies KMeans clustering to the input space before applying SMOTE.

This is an implementation of the algorithm described in [R27a4eeeafc27-1].

Read more in the user guide.

Parameters:
sampling_strategy : float, str, dict or callable, default=’auto’

Sampling information to resample the data set.

  • When float, it corresponds to the desired ratio of the number of samples in the minority class over the number of samples in the majority class after resampling. Therefore, the ratio is expressed as \alpha_{os} = N_{rm} / N_{M} where N_{rm} is the number of samples in the minority class after resampling and N_{M} is the number of samples in the majority class.

    Warning

    float is only available for binary classification. An error is raised for multi-class classification.

  • When str, specify the class targeted by the resampling. The number of samples in the different classes will be equalized. Possible choices are:

    'minority': resample only the minority class;

    'not minority': resample all classes but the minority class;

    'not majority': resample all classes but the majority class;

    'all': resample all classes;

    'auto': equivalent to 'not majority'.

  • When dict, the keys correspond to the targeted classes. The values correspond to the desired number of samples for each targeted class.

  • When callable, function taking y and returns a dict. The keys correspond to the targeted classes. The values correspond to the desired number of samples for each class.

random_state : int, RandomState instance, default=None

Control the randomization of the algorithm.

  • If int, random_state is the seed used by the random number generator;
  • If RandomState instance, random_state is the random number generator;
  • If None, the random number generator is the RandomState instance used by np.random.
k_neighbors : int or object, default=5

Defines the number of nearest neighbors to be used by SMOTE.

  • If int, this number is used to construct synthetic samples.
  • If object, an estimator that inherits from sklearn.neighbors.base.KNeighborsMixin that will be used to find the number of nearest neighbors.
kmeans_estimator : None or object or int or float, default=None

Defines the KMeans clusterer applied to the input space.

imbalance_ratio_threshold : ‘auto’ or float, default=’auto’

The threshold of a filtered cluster. It can be any non-negative number or 'auto' to be calculated automatically.

  • If 'auto', the filtering threshold is calculated from the imbalance ratio of the target for the binary case or the maximum of the target’s imbalance ratios for the multiclass case.
  • If float then it is manually set to this number.

Any cluster that has an imbalance ratio smaller than the filtering threshold is identified as a filtered cluster and can be potentially used to generate minority class instances. Higher values increase the number of filtered clusters.

distances_exponent : ‘auto’ or float, default=’auto’

The exponent of the mean distance in the density calculation. It can be any non-negative number or 'auto' to be calculated automatically.

  • If 'auto' then it is set equal to the number of features. Higher values make the calculation of density more sensitive to the cluster’s size i.e. clusters with large mean euclidean distance between samples are penalized.
  • If float then it is manually set to this number.
raise_error : boolean, default=True
n_jobs : int, default=None

Number of CPU cores used during the cross-validation loop. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.

References

[R27a4eeeafc27-1]Georgios Douzas, Fernando Bacao, Felix Last, “Improving imbalanced learning through a heuristic oversampling method based on k-means and SMOTE” https://www.sciencedirect.com/science/article/pii/S0020025518304997

Examples

>>> import numpy as np
>>> from clover.over_sampling import KMeansSMOTE
>>> from sklearn.datasets import make_blobs
>>> blobs = [100, 800, 100]
>>> X, y  = make_blobs(blobs, centers=[(-10, 0), (0,0), (10, 0)])
>>> # Add a single 0 sample in the middle blob
>>> X = np.concatenate([X, [[0, 0]]])
>>> y = np.append(y, 0)
>>> # Make this a binary classification problem
>>> y = y == 1
>>> kmeans_smote = KMeansSMOTE(random_state=42)
>>> X_res, y_res = kmeans_smote.fit_resample(X, y)
>>> # Find the number of new samples in the middle blob
>>> n_res_in_middle = ((X_res[:, 0] > -5) & (X_res[:, 0] < 5)).sum()
>>> print("Samples in the middle blob: %s" % n_res_in_middle)
Samples in the middle blob: 801
>>> print("Middle blob unchanged: %s" % (n_res_in_middle == blobs[1] + 1))
Middle blob unchanged: True
>>> print("More 0 samples: %s" % ((y_res == 0).sum() > (y == 0).sum()))
More 0 samples: True
Attributes:
clusterer_ : object

A fitted sklearn.cluster.KMeans or sklearn.cluster.MiniBatchKMeans instance.

distributor_ : object

A fitted clover.distribution.DensityDistributor instance.

labels_ : array, shape (n_samples,)

Cluster labels of each sample.

oversampler_ : object

A fitted imblearn.over_sampling.SMOTE instance.

random_state_ : object

An instance of RandomState class.

sampling_strategy_ : dict

Actual sampling strategy.

__init__(self, sampling_strategy='auto', random_state=None, k_neighbors=5, kmeans_estimator=None, imbalance_ratio_threshold='auto', distances_exponent='auto', raise_error=True, n_jobs=None)[source]

Initialize self. See help(type(self)) for accurate signature.

fit(self, X, y)

Check inputs and statistics of the sampler.

You should use fit_resample in all cases.

Parameters:
X : {array-like, dataframe, sparse matrix} of shape (n_samples, n_features)

Data array.

y : array-like of shape (n_samples,)

Target array.

Returns:
self : object

Return the instance itself.

fit_resample(self, X, y, **fit_params)

Resample the dataset.

Parameters:
X : {array-like, dataframe, sparse matrix} of shape (n_samples, n_features)

Matrix containing the data which have to be sampled.

y : array-like of shape (n_samples,)

Corresponding label for each sample in X.

Returns:
X_resampled : {array-like, dataframe, sparse matrix} of shape (n_samples_new, n_features)

The array containing the resampled data.

y_resampled : array-like of shape (n_samples_new,)

The corresponding label of X_resampled.

fit_sample(self, X, y)

Resample the dataset.

Parameters:
X : {array-like, dataframe, sparse matrix} of shape (n_samples, n_features)

Matrix containing the data which have to be sampled.

y : array-like of shape (n_samples,)

Corresponding label for each sample in X.

Returns:
X_resampled : {array-like, dataframe, sparse matrix} of shape (n_samples_new, n_features)

The array containing the resampled data.

y_resampled : array-like of shape (n_samples_new,)

The corresponding label of X_resampled.

get_params(self, deep=True)

Get parameters for this estimator.

Parameters:
deep : bool, default=True

If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:
params : mapping of string to any

Parameter names mapped to their values.

set_params(self, **params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as pipelines). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:
**params : dict

Estimator parameters.

Returns:
self : object

Estimator instance.

Examples using clover.over_sampling.KMeansSMOTE