Metadata-Version: 2.4
Name: wordshift
Version: 0.1.0
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Rust
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Text Processing :: Linguistic
Summary: Fast weighted-average word shift computation with bundled labMT lexicons, powered by Rust
Keywords: word-shift,sentiment,labmt,text-analysis,linguistics
Author-email: Jonathan St-Onge <jonathanstonge7@gmail.com>
License: MIT
Requires-Python: >=3.9
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Repository, https://github.com/Vermont-Complex-Systems/wordshift-core

# wordshift

Fast **weighted-average word shift** computation with bundled labMT lexicons,
powered by Rust ([wordshift-core](https://github.com/Vermont-Complex-Systems/wordshift-core)).

A word shift quantifies which words drive the difference in average sentiment
between two texts, and in which direction. This is a lean Rust port of
[Shifterator](https://github.com/ryanjgallagher/shifterator)'s `WeightedAvgShift`
for the single-lexicon (sentiment) case.

## Install

```bash
pip install wordshift        # once published
# or, from source:
maturin build --release -m crates/wordshift-py/Cargo.toml && pip install target/wheels/wordshift-*.whl
```

## Usage

```python
import wordshift

type2freq_1 = {"happy": 1, "sad": 9, "love": 2}
type2freq_2 = {"happy": 9, "sad": 1, "love": 8}

result = wordshift.weighted_avg_shift(
    type2freq_1,
    type2freq_2,
    lexicon="labMT_English",   # or "English", "French", ...
    reference_value=None,       # None -> system 1's mean score (the baseline)
    top_n=50,                   # cap returned entries (0 = all)
)

for entry in result["entries"][:5]:
    print(entry["type"], round(entry["shift_score"], 4))
```

`result` is a dict with:

- `entries` — per-word `{type, p_diff, s_diff, p_avg, s_ref_diff, shift_score}`,
  sorted by absolute contribution. `shift_score` is normalized so absolute
  scores sum to 1 (the "variation" scheme).
- `component_sums` — the six sign-quadrant totals for the stacked summary bars.
- `total_diff`, `norm`, `s_avg_1`, `s_avg_2`, `reference_value`, `normalization`.

To supply your own scores instead of labMT, use `weighted_avg_shift_custom`:

```python
scores = {"happy": 8.0, "sad": 2.0}
result = wordshift.weighted_avg_shift_custom(type2freq_1, type2freq_2, scores)
```

Words present in the texts but missing from the lexicon are excluded.

## Reference value

By convention, **system 1 is the baseline** and system 2 is read as a shift away
from it. `reference_value=None` (or `"average"`) partitions words around system
1's frequency-weighted mean score. Pass a float for a fixed reference — labMT's
neutral midpoint is `5.0`.

