Metadata-Version: 2.4
Name: llmSHAP
Version: 1.0.0
Summary: Multi-threaded Shapley explainability for LLMs: words, sentences, documents, and tools.
Author: Filip Naudot
License-Expression: MIT
Project-URL: Homepage, https://github.com/filipnaudot/llmSHAP
Project-URL: Repository, https://github.com/filipnaudot/llmSHAP
Project-URL: Documentation, https://filipnaudot.github.io/llmSHAP/
Project-URL: Issues, https://github.com/filipnaudot/llmSHAP/issues
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: tqdm
Requires-Dist: scikit-learn
Provides-Extra: openai
Requires-Dist: openai; extra == "openai"
Requires-Dist: python-dotenv; extra == "openai"
Provides-Extra: embeddings
Requires-Dist: sentence-transformers; extra == "embeddings"
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: matplotlib; extra == "dev"
Requires-Dist: ipywidgets; extra == "dev"
Requires-Dist: sphinx; extra == "dev"
Requires-Dist: myst-parser; extra == "dev"
Requires-Dist: sphinx-book-theme; extra == "dev"
Requires-Dist: sphinx-design; extra == "dev"
Provides-Extra: full
Requires-Dist: openai; extra == "full"
Requires-Dist: python-dotenv; extra == "full"
Requires-Dist: sentence-transformers; extra == "full"
Requires-Dist: pytest; extra == "full"
Requires-Dist: matplotlib; extra == "full"
Requires-Dist: ipywidgets; extra == "full"
Requires-Dist: sphinx; extra == "full"
Requires-Dist: myst-parser; extra == "full"
Requires-Dist: sphinx-book-theme; extra == "full"
Requires-Dist: sphinx-design; extra == "full"
Dynamic: license-file

<div align='center'>
    <picture>
        <source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/filipnaudot/llmSHAP/main/docs/llmSHAP-logo-lightmode.png">
        <img alt="lighbench logo" src="https://raw.githubusercontent.com/filipnaudot/llmSHAP/main/docs/llmSHAP-logo-darkmode.png" width="50%" height="50%">
    </picture>
</div>

# llmSHAP
![Unit Tests](https://github.com/filipnaudot/llmSHAP/actions/workflows/test.yml/badge.svg)
[![Documentation](https://img.shields.io/badge/docs-online-blue.svg)](https://filipnaudot.github.io/llmSHAP/)

A multi-threaded explainability framework using Shapley values for LLM-based outputs.

---

## Getting started
Install (from GitHub) as a package:
```bash
pip install "llmSHAP[full] @ git+https://github.com/filipnaudot/llmSHAP.git"
```

Install in editable mode with all optional dependencies (after cloning the repository):
```bash
pip install -e .[full]  # for bash
```
```bash
pip install -e '.[full]'  # for zsh
```

Documentation is available at [llmSHAP Docs](https://filipnaudot.github.io/llmSHAP/) and a hands-on tutorial can be found [here](https://filipnaudot.github.io/llmSHAP/tutorial.html).

- [Full documentation](https://filipnaudot.github.io/llmSHAP/)  
- [Tutorial](https://filipnaudot.github.io/llmSHAP/tutorial.html)

---

## Example usage

```python
from llmSHAP import DataHandler, BasicPromptCodec, ShapleyAttribution
from llmSHAP.llm import OpenAIInterface

data = "In what city is the Eiffel Tower?"
handler = DataHandler(data, permanent_keys={0,3,4})
prompt_codec = BasicPromptCodec(system="Answer the question briefly.")
llm = OpenAIInterface("gpt-4o-mini")

shap = ShapleyAttribution(model=llm,
                          data_handler=handler,
                          prompt_codec=prompt_codec,
                          use_cache=True,
                          num_threads=7)
result = shap.attribution()

print("\n\n### OUTPUT ###")
print(result.output)

print("\n\n### ATTRIBUTION ###")
print(result.attribution)

print("\n\n### HEATMAP ###")
print(result.render())
```

---

## Example data

You can pass either a string or a dictionary:

```python
from llmSHAP import DataHandler

# String input
data = "The quick brown fox jumps over the lazy dog"
handler = DataHandler(data)

# Dictionary input
data = {"a": "The", "b": "quick", "c": "brown", "d": "fox"}
handler = DataHandler(data)
```

To exclude certain keys from the computations, use `permanent_keys`:
```python
from llmSHAP import DataHandler

data = {"a": "The", "b": "quick", "c": "brown", "d": "fox"}
handler = DataHandler(data, permanent_keys={"a", "d"})

# Get data with index 1 WITHOUT the permanent features.
print(handler.get_data({1}, exclude_permanent_keys=True, mask=False))
# Output: {'b': 'quick'}

# Get data with index 1 AND the permanent features.
print(handler.get_data({1}, exclude_permanent_keys=False, mask=False))
# Output: {'a': 'The', 'b': 'quick', 'd': 'fox'}
```
---


## Comparison with TokenSHAP
| Capability                                                                | **llmSHAP**                                                 | **TokenSHAP**                  |
| ------------------------------------------------------------------------- | ----------------------------------------------------------- | ------------------------------ |
| Threaded                                                                  | ✅ (optional ``num_threads``)                                | ❌                              |
| Modular architecture                                                      | ✅                                                           | ❌                              |
| Exact Shapley option                                                      | ✅ (Full enumeration)                                        | ❌ (Monte Carlo sampling)       |
| Generation caching across coalitions                                      | ✅                                                           | ❌                              |
| Heuristics                                                                | SlidingWindow • Monte Carlo • Counterfactual                 | Monte Carlo                    |
| Sentence-/chunk-level attribution                                         | ✅                                                           | ✅                             |
| Permanent context pinning (always-included features)                      | ✅                                                           | ❌                              |
| Pluggable similarity metric                                               | ✅ TF-IDF, embeddings                                        | ✅ TF-IDF, embeddings          |
| Docs & tutorial                                                           | ✅ Sphinx docs + tutorial                                    | ✅ README only                 |
| Unit tests & CI                                                           | ✅ Pytest + GitHub Actions                                   | ❌                              |
| Vision object attribution                                                 | ❌                                                           | ✅ PixelSHAP                   |
