Metadata-Version: 2.4
Name: icalens
Version: 0.2.0
Summary: Fit, share, and apply ICA lenses for language-model activations.
Project-URL: Homepage, https://liusida.github.io/ica-lens-paper/
Project-URL: Repository, https://github.com/liusida/icalens
Author: Sida Liu, Feijiang Han
License-Expression: MIT
License-File: LICENSE
License-File: THIRD_PARTY_NOTICES.md
Keywords: ICA,activations,interpretability,language-models
Classifier: Development Status :: 3 - Alpha
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Requires-Python: >=3.10
Requires-Dist: huggingface-hub>=0.25
Requires-Dist: numpy>=1.24
Requires-Dist: safetensors>=0.4
Requires-Dist: torch>=2.1
Requires-Dist: tqdm>=4.66
Provides-Extra: analyze
Requires-Dist: gb10-load-llm>=0.1.2; extra == 'analyze'
Requires-Dist: transformers>=5.0; extra == 'analyze'
Description-Content-Type: text/markdown

# ICA Lens

ICA Lens fits, shares, and applies Independent Component Analysis bases for
language-model activations. The core API operates on activations supplied by
the caller; it does not load language models or capture activations.

```bash
uv add icalens
```

Load a published lens:

```python
from icalens import ICALens

lens = ICALens.from_pretrained("sida/icalens-gpt2-small-pile10k")
scores = lens.transform(activations, layer=6)
reconstructed = lens.inverse_transform(scores, layer=6)
energy = lens.energy(scores)  # per-token component fractions summing to 1
```

Fit and publish your own:

```python
from icalens import ICALens

lens = ICALens(
    model_id="openai-community/gpt2",
    model_type="base",
    activation_site="resid_post",
)
lens.fit(activations, layer=6, random_state=0)
lens.save("./my-icalens")
lens.push_to_hub("username/icalens-gpt2-small")
```

For the standalone publishing demo, create a project-root `.env` file containing
a Hugging Face token with write permission:

```dotenv
HF_TOKEN=hf_...
```

The `.env` file is ignored by Git. Publish a saved lens with:

```bash
uv run python demo/publish.py \
  --lens ./my-icalens \
  username/icalens-model-name
```

Instruction-tuned checkpoints use the same activation-level API and are
identified explicitly in their portable metadata:

```python
lens = ICALens(
    model_id="Qwen/Qwen2.5-0.5B-Instruct",
    model_type="instruct",
)
```

`model_type` describes the checkpoint and accepts `"base"` or `"instruct"`.
Install `icalens[analyze]` to capture and analyze text or completed chat
conversations directly. `result = lens.analyze(text, layer=6)` returns aligned
tokens, activations, signed scores, and per-token component energy shares.

Inputs may be NumPy arrays or PyTorch tensors. Leading dimensions are treated
as sample dimensions and the final dimension must be the model hidden size.
Fitting uses ICA Lens's built-in PyTorch FastICA implementation and can run on
the input tensor's device. NumPy inputs are fitted on CPU. ICA Lens does not
depend on scikit-learn or SciPy.

See [`docs/api.md`](docs/api.md) and
[`docs/artifact-format.md`](docs/artifact-format.md) for the public API and
portable artifact format.

For the 1,000-token GPT-2/Pile-10k fitting demo, run:

```bash
uv sync
uv run python demo/fit.py
```

For the corresponding instruct-model demo using assistant tokens from
UltraChat conversations, run:

```bash
uv run python demo/fit_chat.py --layers 12
```

Then inspect assistant-token component scores with:

```bash
uv run python demo/apply_chat.py
```

Both `apply.py` and `apply_chat.py` also write standalone interactive HTML
explorers under `demo/output/`; pass `--output-file` to choose another path.
