Metadata-Version: 2.4
Name: lexframe
Version: 0.1.0
Summary: A toolkit for interpreting LM residual streams in their native semantics
Author-email: Jian Gu <jianguda@gmail.com>
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/jianguda/lexframe
Project-URL: Repository, https://github.com/jianguda/lexframe
Project-URL: Issues, https://github.com/jianguda/lexframe/issues
Keywords: interpretability,language models,logit lens,residual stream,steering
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Typing :: Typed
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=2.4.1
Requires-Dist: torch>=2.10.0
Requires-Dist: transformers>=5.0.0
Provides-Extra: test
Requires-Dist: pytest>=9.0.3; extra == "test"
Provides-Extra: demo
Requires-Dist: datasets>=4.5.0; extra == "demo"
Dynamic: license-file

# lexframe

A toolkit for interpreting LM residual streams in their native semantics.

```bash
pip install lexframe
```

```python
import lexframe as lf

frame = lf.LexFrame(model, tokenizer).init(texts)   # a loaded Hugging Face model; texts: any unlabelled text
frame.view("The Eiffel Tower is located in the city of", k=5).tokens
```

[demo.py](https://github.com/jianguda/lexframe/blob/main/demo.py) walks through every command: where SmolLM3 decides that the Eiffel Tower is in Paris, who wrote that decision, and how to change it.

## Commands

| Command | Returns |
|---|---|
| `LexFrame(model, tokenizer, order=1, allow_untested=False)` | a frame with `order` rounds of attention between positions; `"inf"` is the native frame |
| `frame.init(texts, size=256, cache=True, cache_dir=None, tol=0.05)` | the frame, calibrated on `size` sequences of 128 tokens (cached on disk) and checked: `frame.identities` |
| `frame.config(steps=30, ridge=1e-3, warm=False)` | the frame, with these LSMR settings for `basis`: `frame.solver` |
| `frame.view(text, k=1, layers=None)` | `View`: `tokens`, `ids`, `probs` `[layer, position, k]` and `kl` `[layer, position]` |
| `frame.state(text, layer)` | `State`: `vector` `[position, hidden]` and `layer`; `s + d` and `s - d` give a moved state |
| `frame.basis(text, tokens, layers=None)` | `Delta`: vectors that add one logit to each token, and `coef`, the state's coefficients |
| `frame.readout(text, tokens, layers=None)` | `Delta`: covectors that measure each token's logit; `readout @ basis` has ones on the diagonal |
| `frame.output(state)` | the model's log-probabilities from a (moved) state, `[position, vocab]` |
| `frame.attr(text, component="block", target=None)` | each layer's contribution to the target's logit, `[layer, position]`; `"block"`, `"attn"` or `"mlp"` |
| `frame.bound(texts, token, layers=None)` | the best agreement any fixed lens can reach, `[layer]` |

`view` and `attr` take a list of texts of one token length; `view`, `basis` and `readout` also take a `State`. `basis` and `readout` cover layers `0..num_layers`, where the last is the output. `d[token]` selects one token's vectors, and `d @ x` takes inner products with a `Delta`, a `State` or a vector.

Supported models are SmolLM3 and OLMo3; Llama, OLMo2 and Qwen3 load with `allow_untested=True` and should pass the identity check first.

Citation: [*Lexical Reference Frames: Interpreting LM Residual Streams in Native Semantics*](https://arxiv.org/abs/2610.22169) (2027).

Licensed under the [Apache License 2.0](https://github.com/jianguda/lexframe/blob/main/LICENSE).
