Metadata-Version: 2.4
Name: icalens
Version: 0.1.0
Summary: Fit, share, and apply ICA lenses for language-model activations.
Project-URL: Homepage, https://liusida.github.io/ica-lens-paper/
Project-URL: Repository, https://github.com/liusida/icalens
Author: Sida Liu, Feijiang Han
License-Expression: MIT
License-File: LICENSE
License-File: THIRD_PARTY_NOTICES.md
Keywords: ICA,activations,interpretability,language-models
Classifier: Development Status :: 3 - Alpha
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Requires-Python: >=3.10
Requires-Dist: huggingface-hub>=0.25
Requires-Dist: numpy>=1.24
Requires-Dist: safetensors>=0.4
Requires-Dist: torch>=2.1
Requires-Dist: tqdm>=4.66
Description-Content-Type: text/markdown

# ICA Lens

ICA Lens fits, shares, and applies Independent Component Analysis bases for
language-model activations. Version 0.1 operates on activations supplied by the
caller; it does not load language models or capture activations.

```bash
uv add icalens
```

Load a published lens:

```python
from icalens import ICALens

lens = ICALens.from_pretrained("liusida/icalens-gpt2-small")
scores = lens.transform(activations, layer=6)
reconstructed = lens.inverse_transform(scores, layer=6)
```

Fit and publish your own:

```python
from icalens import ICALens

lens = ICALens(
    base_model="openai-community/gpt2",
    base_model_revision="FULL_COMMIT_HASH",
    activation_site="resid_post",
)
lens.fit(activations, layer=6, random_state=0)
lens.save("./my-icalens")
lens.push_to_hub("username/icalens-gpt2-small")
```

Inputs may be NumPy arrays or PyTorch tensors. Leading dimensions are treated
as sample dimensions and the final dimension must be the model hidden size.
Fitting uses ICA Lens's built-in PyTorch FastICA implementation and can run on
the input tensor's device. NumPy inputs are fitted on CPU. ICA Lens does not
depend on scikit-learn or SciPy.

See [`docs/api.md`](docs/api.md) and
[`docs/artifact-format.md`](docs/artifact-format.md) for the initial API and
portable artifact format.

For the 1,000-token GPT-2/Pile-10k fitting demo, run:

```bash
uv sync
uv run python demo/fit.py
```
