Metadata-Version: 2.2
Name: clustexts
Version: 0.0.1
Author-email: Jordi Carrera Ventura <jordi.carrera.ventura@gmail.com>
Project-URL: GitHub repository, https://github.com/JordiCarreraVentura/clustexts
Project-URL: Similar project (independent), https://github.com/anshulp2912/automeans
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: Unidecode==1.3.8
Requires-Dist: build
Requires-Dist: matplotlib==3.10.1
Requires-Dist: numpy==2.2.3
Requires-Dist: pandas==2.2.3
Requires-Dist: pytest==8.3.5
Requires-Dist: scikit-learn==1.6.1
Requires-Dist: scipy==1.15.2
Requires-Dist: seaborn==0.13.2
Requires-Dist: tqdm==4.67.1
Requires-Dist: twine

# Clustexts

Performs k-means clustering on a collection of texts. It automates the selection of `k` by running the elbow method implicitly. The algorithm only expects the range of minimum and maximum values for `k` (default to 2 and 20, respectively).
    
Texts are encoded using a TFIDF Bag-of-Words representation. Optionally, Truncated Singular Value Decomposition can be used to reduce the dimensionality of the resulting matrix and project the topology onto an embedded space, for improved data compression and schema generalization.
    
The call to returns an iterator containing the cluster identifiers associated with each input document.


## Dependencies

Ensure you have the following packages installed:

```
matplotlib==3.10.1
numpy==2.2.3
pandas==2.2.3
scikit-learn==1.6.1
scipy==1.15.2
seaborn==0.13.2
tqdm==4.67.1
```


## Usage

Example of usage:

```
rows = [
  'one text',
  'another text',
  'this sentence',
  'fourth sentence',
  'fifth sentence',
]
df = pd.DataFrame(rows, columns=['text'])

cls = Clustexts(
  reducer={},
  range = (2, 10),
  min_gain=0.001,
  vectorizer={'min_df': 0.0}
)
df['cluster'] = cls(df['text'])
```

## Parameters

### Clustering

- `range: Tuple[int, int] = (2, 20)`: Specifies the minimum and maximum values of `k` to explore when applying the elbow method.
- `min_size: int = 0`: The minimum cluster size to be accepted. If reached, the clustering stops.
- `min_gain: float = 0.03`: The minimum relative improvement for the clustering to continue running (as a percentage of the inertia).

### Vectorization (required) & Dimensionality reduction (optional)

Refer to the scikit-learn's documentation for the `TfidfVectorizer` and the `TruncatedSVD` classes.

### Reporting (optional)

- `plot_density: bool = False`: If set to `True`, the system will plot cluster densities (number of documents in each cluster).
- `plot_k: bool = False`: If set to `True`, the algorithm will plot the inertia trendline for every `k` that has been explored.
- `show_examples: bool = False`: If set to `True`, the algorithm will display 3 examples of each output cluster after the elbow has been found.
- `verbose: bool = False`: If set to `True`, prints a message on the terminal specifying the clustering termination condition.

## Methods

- `encode(X: Iterable[str]) -> np.ndarray`: Transforms input text X to a numerical vector using TF-IDF Vectorizer, and optionally applies SVD dimensionality reduction.
- `__call__(self, X: Iterable[str]) -> Iterable[int]`: fits model on input data X.

