Metadata-Version: 2.2
Name: explainable-attention
Version: 0.0.1
Summary: Implementation of various tools for multi-head attention explainability from transformers.
Author-email: "David W. Ludwig II" <davidludwigii@gmail.com>
Project-URL: Homepage, https://github.com/DLii-Research/explainable-attention
Project-URL: Bug Tracker, https://github.com/DLii-Research/explainable-attention/issues
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Requires-Dist: attn-gym
Requires-Dist: torch>=2.5.0

# explainable-attention

Implementation of various tools for multi-head attention explainability from transformers.

## Self-Attention Attribution

[Hao, Yaru, et al. "Self-attention attribution: Interpreting information interactions inside transformer." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 35. No. 14. 2021.](https://arxiv.org/abs/2004.11207)

```py
from explainable_attention.self_attention_attribution import compute

...

def objective(batch):
    x, y = batch
    y = model(x)
    loss = loss_fn(x, y)
    return loss

attribution = saa.compute(
    model.transformer_encoder.layers,
    objective,
    batch,
    integration_steps=20)
```
