Metadata-Version: 2.4
Name: scweatwords
Version: 1.0.0
Summary: Calculate SC-WEAT bias from GloVe 200d Gigaword embeddings.
Author-email: Katharina Nemtsev <katharina.nemtsev@uzh.ch>
License-Expression: MIT
Project-URL: Homepage, https://gitlab.uzh.ch/katharina.nemtsev/genderbiaser
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Requires-Python: >=3.13
Description-Content-Type: text/markdown
License-File: LICENSE.md
Requires-Dist: numpy==2.4
Dynamic: license-file

# Gender Bias in Word Embeddings

A command-line tool for measuring gender bias in GloVe word embeddings using the
Single-Category Word Embedding Association Test (SC-WEAT).

Student: Katharina Nemtsev

ID Number: 19-922-137

Repo: [gitlab](https://gitlab.uzh.ch/katharina.nemtsev/genderbiaser)

## Background

Word embeddings are reflective of the texts they were trained on. Much research has
shown that there is a gender bias reflected in word embeddings. With this module,
I want to make it possible to quickly find the gender bias of words based on the
newest GloVe word embeddings.

The GloVe corpus used is the 2024 Wikipedia + Gigaword 5 (11.9B tokens, 1.2M 
vocab, uncased, 200d vectors, 1.1 GB download), downloadable also at:
[GloVe](https://nlp.stanford.edu/projects/glove/). The program already includes
the necessary file, so no need to download anything for you.

The SC-WEAT methodology is based on Caliskan et al. (2022), *Gender Bias in Word Embeddings:
A Comprehensive Analysis of Frequency, Syntax, and Semantics*
[arXiv](https://doi.org/10.48550/arXiv.2206.03390), which used the SC-WEAT score to demonstrate
the widespread prevalence of gender bias across GloVe and fastText embeddings. I also used
their anchor words, which came from a previous paper of Caliskan et al. (2017):
[arXiv](https://doi.org/10.48550/arXiv.1608.07187).
I am not very familiar with the SC-WEAT measurement and only learned about it for this
project, so please excuse any comprehension errors you may find.

## SC-WEAT Formula

For a given input word `w`, the module computes a Cohen's *d* effect size by comparing
how close `w` is to a set of male anchor words versus a set of female anchor words in
vector space:

```
ES(w) = ( mean_cos(w, male_anchors) - mean_cos(w, female_anchors) )
        ─────────────────────────────────────────────────────────────
                    std_dev( all cosine similarities )
```

A **negative** effect size indicates male association; a **positive** effect size
indicates female association. The magnitude follows Cohen's *d* conventions, as
also described by Caliskan et al.:

| |d|       | Interpretation |
|-----------|----------------|
| < 0.20    | Negligible     |
| 0.20–0.49 | Small          |
| 0.50–0.79 | Medium         |
| ≥ 0.80    | Large          |

Each side requires at least 8 anchor words, as specified by the SC-WEAT methodology.
The default anchors can be changed by editing `anchors.json` directly.

## Setup

### Requirements

- Python 3.13
- numpy

## Usage

Installation:
```bash
python3 pip install scweatwords
```


### Score a single word

```bash
python3 embeddings.py --word nurse
python3 embeddings.py --word engineer
```

### Score a list of words from a file

```bash
python3 embeddings.py --list words.txt
```

One word per line. Words not found in the GloVe vocabulary are skipped with an error.

### Find nearest neighbours

Add `--neighbours N` to either the word or file to retrieve the N most similar words in the
embedding space, alongside their similarity to the input word:

```bash
python3 embeddings.py --word doctor --neighbours 10
python3 embeddings.py --list jobs.txt --neighbours 5
```

Be aware: In the list option, this iterates over the entire GloVe file once for each word. 
The size of N will not have a noticeable impact on your runtime, but the amount of words
in your list will.

## Anchor words

The gender anchor words are stored in `anchors.json`. The defaults are:

```json
{
  "male":   ["male", "he", "him", "his", "man", "boy", "son", "brother"],
  "female": ["female", "she", "her", "hers", "woman", "girl", "daughter", "sister"]
}
```

These are taken from the two Caliskan et al. papers mentioned earlier.

## Reference

Caliskan, A. et al. (2022) Gender bias in word embeddings: A comprehensive analysis of frequency, syntax, and semantics, arXiv.org. Available at: https://doi.org/10.48550/arXiv.2206.03390 (Accessed: 07 June 2026).

Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. GloVe: Global Vectors for Word Representation. [pdf] [bib]

Riley Carlson, John Bauer, and Christopher D. Manning. 2025. A New Pair of GloVes. [pdf]
