Metadata-Version: 2.5
Name: glinerel-fast
Version: 0.1.0
Summary: Fast, lightweight inference for glinerel entity, PII and relation extraction models
Project-URL: Homepage, https://github.com/VAGOsolutions/glinerel-fast
Project-URL: Repository, https://github.com/VAGOsolutions/glinerel-fast
Author: Michele Montebovi
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: anonymization,gliner,inference,ner,pii,pytorch,relation-extraction
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Text Processing :: Linguistic
Requires-Python: >=3.10
Requires-Dist: huggingface-hub>=0.20
Requires-Dist: numpy>=1.22
Requires-Dist: safetensors>=0.4
Requires-Dist: tokenizers>=0.15
Requires-Dist: torch>=2.5
Description-Content-Type: text/markdown

# glinerel-fast

Fast, lightweight inference for glinerel entity, PII and relation extraction models.

* **Same predictions** as the models' reference implementation in float32 (checked on 1,288 texts, model sizes
  72m, 128m and 314m, CPU and Apple GPU).
* **Faster**: up to 4x lower single-text latency on CPU, up to 2x the reference's speed on the GPU, and 4-7x faster
  loading (see [Performance](#performance)).
* **Small**: ~1,650 lines of Python on PyTorch, NumPy, `tokenizers` and `safetensors`. No transformers, no model
  conversion: the checkpoint's `model.safetensors` is read directly.
* **Runs where PyTorch runs**: CPU (Linux, Windows, macOS), NVIDIA / AMD GPUs, Apple silicon GPUs. The best
  available device is picked automatically.

## Install

```bash
pip install glinerel-fast
```

For a specific PyTorch build (CPU-only, a given CUDA version, ROCm), install PyTorch first from
[pytorch.org](https://pytorch.org/get-started/locally/). Requires Python 3.10+ and PyTorch 2.5+.

The models are downloaded from the Hugging Face Hub on first use. If a repository is private, log in first
(`hf auth login`) or set `HF_TOKEN`.

## Quick start

```python
from glinerel_fast import GLiNERel

model = GLiNERel.from_pretrained("path/to/glinerel-model")   # local folder or Hugging Face repository id

model.extract(
    "Hi, I'm Laura Bianchi. Reach me at +39 347 555 0192 or laura.bianchi@example.com.",
    entities=["first name", "last name", "phone number", "email"],
)
# {'entities': [{'label': 'first name', 'text': 'Laura', 'start': 8, 'end': 13, 'score': 0.9986},
#               {'label': 'last name', 'text': 'Bianchi', 'start': 14, 'end': 21, 'score': 0.9991},
#               {'label': 'phone number', 'text': '+39 347 555 0192', 'start': 35, 'end': 51, 'score': 0.9986},
#               {'label': 'email', 'text': 'laura.bianchi@example.com', 'start': 55, 'end': 80, 'score': 0.9972}],
#  'relations': []}
```

### Entities and relations

```python
result = model.extract(
    "Marie Curie was born in Warsaw in 1867. She married Pierre Curie.",
    entities=["person", "location", "date"],
    relations=["born in", "spouse"],
)
for r in result["relations"]:
    print(r["head"]["text"], f"--[{r['label']}]-->", r["tail"]["text"], r["score"])
# Marie Curie --[born in]--> Warsaw 0.9686
# Marie Curie --[spouse]--> Pierre Curie 0.9479
```

`extract(texts, entities=None, relations=None, *, entity_threshold=0.5, relation_threshold=0.25, batch_size=8,
trim_punctuation=True, merge_mentions=False)`

* `texts`: one string (returns one result) or a list of strings (returns a list in the same order). Pass many texts
  per call: they are sorted by length and batched internally.
* `entities` / `relations`: a list of labels, or `{label: threshold}` for per-label thresholds.
* Each result is `{"entities": [{"label", "text", "start", "end", "score"}], "relations": [{"label", "head":
  {"text", "start", "end"}, "tail": {...}, "score"}]}`, with character offsets (`end` exclusive). Entities are sorted
  by position, relations by score.
* `trim_punctuation=False` keeps the model's word-level spans (`"Warsaw."`); `merge_mentions=True` keeps one relation
  per pair of entity surfaces.

### PII redaction

```python
PII = ["first name", "last name", "date of birth", "phone number", "email", "city", "passport number"]
text = "Hi, I'm Laura Bianchi, born on 3 May 1988. Call me at +39 347 555 0192, I live in Milano."

model.predict_entities(text, PII)   # GLiNER-style list of entities
model.redact(text, PII)
# "Hi, I'm [FIRST_NAME] [LAST_NAME], born on [DATE_OF_BIRTH]. Call me at [PHONE_NUMBER], I live in [CITY]."
model.redact(text, PII, placeholder="<{label}>")
```

### Devices and precision

```python
GLiNERel.from_pretrained(model, device=None, precision="fp32", threads=None)
```

* `device`: `"cpu"`, `"cuda"`, `"cuda:1"`, `"mps"`; `None` picks CUDA, then MPS, then the CPU.
* `precision`: `"fp32"` (the reference's predictions), `"bf16"` or `"fp16"` (half-precision matrix products, for
  GPUs; 99.6-100% agreement with fp32).
* `threads`: CPU threads. On Linux, prefer setting `OMP_NUM_THREADS` before starting Python.

Tips: pass many texts per call and, on GPUs, use `batch_size=32` or more. Every label adds a few tokens to every
text, so ask only for the labels you need. Long texts are not truncated; split very long documents by paragraph.

### Command line

```bash
glinerel-fast extract --model path/to/glinerel-model --text "Contact Laura at laura@example.com" \
    --entities person email
glinerel-fast extract --model path/to/glinerel-model --input texts.jsonl --output out.jsonl \
    --entities person organization --relations "works for"
glinerel-fast bench --model path/to/glinerel-model      # time per text on this machine
glinerel-fast info                   # versions, GPUs, default device
```

## Performance

Apple M3 Pro, PyTorch 2.14, 64 PII texts with 20 PII labels, time per text. The reference is the models'
`inference.py` on the same PyTorch and device.

| Model size | Device | reference, 1 text | glinerel-fast, 1 text | reference, batch 32 | glinerel-fast, batch 32 | load |
|---|---|---:|---:|---:|---:|---:|
| 72m | CPU | 102.5 ms | 26.7 ms (**3.8x**) | 21.5 ms | 17.3 ms (1.2x) | 5.3 s → 0.8 s |
| 72m | GPU (MPS) | 26.2 ms | 15.2 ms (**1.7x**) | 13.3 ms | 6.6 ms (**2.0x**) | 7.0 s → 0.9 s |
| 128m | CPU | 155.4 ms | 37.9 ms (**4.1x**) | 30.9 ms | 24.3 ms (1.3x) | 5.4 s → 0.7 s |
| 128m | GPU (MPS) | 24.4 ms | 17.1 ms (1.4x) | 14.2 ms | 9.2 ms (**1.5x**) | 5.5 s → 1.1 s |
| 314m | CPU | 231.7 ms | 73.8 ms (**3.1x**) | 48.4 ms | 47.4 ms (1.0x) | 6.8 s → 1.1 s |
| 314m | GPU (MPS) | 32.6 ms | 23.2 ms (1.4x) | 20.3 ms | 15.1 ms (1.3x) | 7.2 s → 2.0 s |

On a Linux CPU (OpenBLAS) the two implementations are bound by the same matrix products and run at about the same
speed (0.9-1.4x), with the faster loading. NVIDIA GPUs and Windows have not been benchmarked yet.

Accuracy against the reference (agreement F1, the reference's predictions as gold; 1,000 PII texts in 5 languages
and the 288 CoNLL04 test sentences):

| Precision | PII entities | CoNLL04 entities | CoNLL04 relations |
|---|---:|---:|---:|
| fp32 (CPU, MPS), all three sizes | 1.0000 | 1.0000 | 1.0000 |
| fp16 (MPS) | 0.9991-0.9998 | 0.9995-1.0000 | 0.9963-1.0000 |
| *the reference's own bf16 mode, for comparison* | *0.9915-0.9926* | *0.9978-0.9990* | *0.9902-0.9967* |

## License

Apache-2.0 for this library. The model weights are subject to their own license on the Hugging Face Hub.
