Metadata-Version: 2.4
Name: zadapter
Version: 0.1.0
Summary: TPU-native bottleneck FFN adapter for parameter-efficient fine-tuning (LoRA alternative, static graph, XLA-friendly)
Author-email: Riki <alkenzyhaikal@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/RikZD/ZAdapter
Project-URL: Repository, https://github.com/RikZD/ZAdapter
Keywords: pytorch,tpu,xla,fine-tuning,peft,adapter,lora
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: torch>=2.0

# ZAdapter

TPU-native bottleneck FFN adapter for parameter-efficient fine-tuning.

Unlike LoRA (parallel low-rank matrices merged into existing weights),
ZAdapter injects a small bottleneck feed-forward block **serially** after
attention and MLP in each transformer layer. This keeps the computation
graph static — no merge/unmerge step, no dynamic branching — which plays
well with XLA compilation on TPU.

```
input -> down_proj [d_model -> r] -> activation -> up_proj [r -> d_model] -> output
output = input + adapter(input)
```

## Install

```bash
pip install zadapter
```

## Usage

```python
from transformers import AutoModelForCausalLM
from zadapter import inject_adapter, get_trainable_params

model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-Coder-7B")
model = inject_adapter(model, r=64)

trainable_params = get_trainable_params(model)
optimizer = torch.optim.AdamW(trainable_params, lr=1e-4)
```

## Why not LoRA

| | LoRA | ZAdapter |
|---|---|---|
| Injection | Parallel to existing weight | Serial FFN block |
| Mergeable | Yes (zero inference overhead after merge) | No |
| Graph | Dynamic (merge/unmerge) | Static, XLA-friendly |
| Trainable params | ~0.1-1% | ~0.5-3% |

ZAdapter trades a slightly larger parameter count and a small inference
overhead for a simpler, more XLA-friendly training graph — useful when
targeting TPU where static graphs compile more predictably.

## Companion library

For TPU sharding (data-parallel, tensor-parallel) and quantization (int8,
NF4) to use alongside ZAdapter, see [zenbit](https://pypi.org/project/zenbit/).

## License

MIT
