Metadata-Version: 2.4
Name: llm-layer-collector
Version: 1.0.0
Summary: Load and dispatch individual transformer layers from HuggingFace model checkpoints
Project-URL: Homepage, https://github.com/erinclemmer/language-pipes
Project-URL: Issues, https://github.com/erinclemmer/language-pipes/issues
Author-email: Erin Clemmer <erin.c.clemmer@gmail.com>
License-Expression: MIT
License-File: LICENSE
Keywords: Language Model,huggingface,inference,layer,transformers
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3.10
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: safetensors
Requires-Dist: torch
Requires-Dist: transformers
Description-Content-Type: text/markdown

# llm-layer-collector

Load and dispatch **individual transformer layers** from HuggingFace model
checkpoints, rather than instantiating a whole model at once. Given a model
directory (config + safetensors shards), it can materialize just the embedding,
a range of decoder layers, the final norm, or the LM head — each as a
standalone `torch.nn.Module` — and run computation through them.

This powers the layer-sharding used by
[language-pipes](https://github.com/erinclemmer/language-pipes) for distributed
inference, but has no dependency on it and can be used on its own.

## Install

```bash
pip install llm-layer-collector
```

## Usage

```python
from llm_layer_collector import LlmLayerCollector

collector = LlmLayerCollector(model_dir="/path/to/model", cache_file="cache.json")
embedding = collector.load_input_embedding()
layers = collector.load_layer_set(0, 4)   # decoder layers 0..4 (end inclusive)
norm = collector.load_norm()
head = collector.load_head()
```

## Supported architectures

Llama, Phi-3, Qwen3, Qwen3-MoE, Gemma3, Gemma4, Ministral3. See
`src/llm_layer_collector/modeling/` for per-architecture support.

## License

MIT
