Metadata-Version: 2.5
Name: openternary
Version: 0.1.0a1
Summary: Open-source tooling for researching ternary LLMs (Gemma 4 E2B first target)
Project-URL: Repository, https://github.com/ELRdn/OpenTernary
Author: OpenTernary
License-Expression: Apache-2.0
License-File: LICENSE
License-File: NOTICE
License-File: TRADEMARKS.md
Keywords: gemma,llm,quantization,ternary
Requires-Python: >=3.11
Requires-Dist: filelock>=3
Requires-Dist: huggingface-hub>=0.23
Requires-Dist: packaging>=23
Requires-Dist: psutil>=6
Requires-Dist: pydantic>=2.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: rich>=13
Requires-Dist: typer>=0.12
Provides-Extra: calibration
Requires-Dist: datasets>=2.18; extra == 'calibration'
Provides-Extra: diffusion
Requires-Dist: diffusers==0.40.0; extra == 'diffusion'
Provides-Extra: ml
Requires-Dist: accelerate>=0.30; extra == 'ml'
Requires-Dist: pillow>=12.3.0; extra == 'ml'
Requires-Dist: safetensors>=0.4; extra == 'ml'
Requires-Dist: torch>=2.5; extra == 'ml'
Requires-Dist: torchvision>=0.28.0; extra == 'ml'
Requires-Dist: transformers<6,>=5.6.2; extra == 'ml'
Provides-Extra: torchao
Requires-Dist: torchao==0.18.0; extra == 'torchao'
Description-Content-Type: text/markdown

# OpenTernary

> Open-source tooling for researching, calibrating, benchmarking, and eventually deploying ternary LLMs.

**OpenTernary** is an experimental CLI-first project for exploring how existing open-weight language models can be converted toward ternary weights such as `{-1, 0, +1}` while retaining as much model quality as possible.

The project begins with **Gemma 4 E2B** as the first research target and focuses on building a reproducible quantization pipeline before attempting a GUI or large-model scaling.

> [!WARNING]
> OpenTernary is a research project. Early outputs may be slow, inaccurate, incompatible with common runtimes, or significantly worse than the original model.

Product planning: [CLI Product Roadmap](CLI_ROADMAP.md) covers CLI reliability, backend integration, artifacts, export, and automation. Model-quality research remains tracked in [Research Roadmap](ROADMAP.md) and the [P0–P7 acceptance plan](docs/plans/p0-p7-research-acceptance.md).

**CLI implementation update (2026-09-23):** Offline planning, backend/adapter services, atomic manifests, packed export, bounded search, JSON output, and plugin discovery are implemented. Validation now includes seeded synthetic Llama/Diffusers networks, real TorchAO, Windows/WSL installed wheels, RX 9070 XT FP32/BF16 execution, and pinned llama.cpp CPU generation. Pretrained-model quality, native low-bit kernel certification, and release acceptance remain pending. See the [CLI guide](docs/CLI.md) and [implementation/validation matrix](docs/CLI_IMPLEMENTATION_STATUS.md).

---

## Install

From PyPI (after the first release), as a command available everywhere:

```bash
uv tool install openternary --python 3.12
openternary setup        # detect the GPU and install a matching PyTorch build
openternary doctor
```

`openternary update` checks PyPI, upgrades, and reinstalls the PyTorch build that `openternary setup` recorded. On Windows it prints the commands to run from a new terminal, because a running `openternary` cannot replace itself.

## Quickstart (core CLI)

**推奨: uv で再現可能な環境を構築**

```bash
git clone https://github.com/ELRdn/OpenTernary.git
cd OpenTernary
uv sync
uv run openternary --help
uv run --frozen openternary doctor --json
uv run --frozen openternary backends list --json
uv run --frozen openternary plan --json
```

開発時チェック:

```bash
uv run --frozen ruff check src tests
uv run --frozen ruff format --check src tests
uv run --frozen mypy src/openternary
```

Fallback (pip):

```bash
pip install -e .
openternary --help
openternary doctor --json
```

**`openternary` をどこからでも使う（推奨）**

`uv sync` だけではコマンドはリポジトリ内の `.venv` にしか入らないため、`uv run openternary ...` で呼ぶか、仮想環境を有効化する（Windows: `.venv\Scripts\Activate.ps1`、macOS/Linux: `source .venv/bin/activate`）。どのフォルダからでも `openternary` と打てるようにするには、リポジトリのフォルダで次を実行する。

```bash
uv tool install --editable . --python 3.12
uv tool update-shell
```

`uv tool update-shell` は初回だけ必要で、実行後にターミナルを開き直す。`--python 3.12` を付けないと、uvはPCにある最新のPythonを選ぶことがある（CIで検証しているのは3.11と3.12）。このコマンドで入るのは本体だけなので、量子化・校正には `uv run openternary ...`（`uv sync --extra ml --extra calibration` 済みの `.venv`）を使うか、`uv tool install --editable ".[ml,calibration]" --python 3.12 --reinstall` で機械学習の依存ごと入れる。`--editable` なので `git pull` した変更はそのまま反映される。依存パッケージが増えた更新のあとは `uv tool install --editable . --reinstall` を実行する。Windowsはコマンド名の大文字小文字を区別しないため `OpenTernary` でも動く。

GPU用のPyTorchは `openternary setup` で入れられる。GPU（NVIDIA / AMD / Apple）を検出して合うビルドを提案し、確認のうえ `openternary` を動かしている環境に入れ、実際のデバイスで動作を確認する（`openternary setup --list` で候補の一覧、`--dry-run` で計画だけ表示）。詳細は [CLI guide](docs/CLI.md) の「PyTorchの環境構築」。

`--json` を付けると、結果のJSON 1行だけがstdoutに出て、人間向けの表示はstderrに回る。ターミナルでは両方が見えるので2回出たように見えるが、`> result.json` のようにリダイレクトすると1行だけが保存される。

- Python 3.12 を推奨（`uv python pin 3.12` で固定、`requires-python >=3.11`）
- `uv.lock` + `.python-version` + `config.yaml` + `environment.json` で再現性を担保

詳細は `docs/ENVIRONMENT.md` を参照。

The core install does not include PyTorch. Tensor conversion requires `[ml]`; optional integrations use `[ml,torchao]` or `[ml,diffusion]`. Keep hardware-specific environments separate. Run `python scripts/validate_cli_offline.py` in an existing development environment for the fixture-only checks; this script installs nothing and uses empty Hugging Face caches.

---

## Why OpenTernary?

Current low-bit LLM tooling is powerful, but the path from:

```text
open-weight model
    ↓
weight analysis
    ↓
ternary conversion
    ↓
calibration / reconstruction
    ↓
benchmark
    ↓
packed deployment
```

is still difficult to explore as a single, reproducible workflow.

OpenTernary aims to make that workflow inspectable and repeatable.

The first goal is **not** to immediately create the smallest possible model.

The first goal is:

> **Make a ternary Gemma 4 E2B that can be generated, loaded, benchmarked, and iterated on reproducibly.**

---

## Project Principles

1. **CLI first, GUI later**
   - Research interfaces change quickly.
   - Stabilize the quantization pipeline before building a desktop/web GUI.

2. **Measure before optimizing**
   - Every conversion must be compared against a fixed baseline.

3. **Quality per byte, not bit-count vanity**
   - Hybrid precision is allowed when it preserves substantially more capability.

4. **Reproducibility**
   - Every experiment should record config, model revision, code commit, hardware, dataset split, and metrics.

5. **Model adapters, not model-specific spaghetti**
   - Gemma 4 is the first target, not the only possible target.

6. **Research claims require evidence**
   - Do not claim CAT-Q, ScaleQ, Bonsai, BitNet, or any paper/framework reproduction unless the implementation has actually been verified against its source.

---

## Initial Scope

### First target

- Gemma 4 E2B instruction model
- Text path first
- Transformer linear layers first
- Embeddings / PLE / LM head / multimodal components handled separately

### First quantization target

Ternary weights:

```text
{-1, 0, +1}
```

with group-wise scaling.

### Initial development order

```text
Inspect
  ↓
Baseline benchmark
  ↓
Naive ternary fake quant
  ↓
Compare
  ↓
Calibration / reconstruction
  ↓
Japanese-focused evaluation
  ↓
Packed export
  ↓
Scale to larger Gemma 4 variants
  ↓
GUI
```

---

## CLI

Commands: `inspect`, `plan`, `doctor`, `backends list/info`, `quantize`, `calibrate`, `benchmark`, `quality`, `evaluate`, `compare`, `export`, `artifacts validate`, `optimize`, `plugins`, `cache list/remove`, and the compatible `cache-info` entry point.

Model-free planning (no download, inference, or output directory):

```bash
openternary plan /path/to/local-snapshot --backend ternary --scheme absmean --json
openternary quantize /path/to/local-snapshot --scale-granularity per_group --group-size 128 --dry-run --json
openternary calibrate --config configs/gemma4-e2b-calibration.yaml --dry-run --json
```

The following is a future model-validation sequence, not evidence that a particular model works:

```bash
openternary quantize /path/to/local-snapshot --device cpu --output runs/cli-ternary
openternary export runs/cli-ternary --format ternary-packed --output runs/cli-packed
openternary export runs/cli-packed --format safetensors --output runs/cli-restored
openternary artifacts validate runs/cli-restored --json
openternary benchmark runs/cli-restored --suite smoke --output runs/cli-smoke
```

`artifacts validate` checks the manifest and file integrity; model reload and quality are separate checks. Packed artifacts require unpacking for inference. GGUF uses an explicitly selected local llama.cpp converter, currently restricted to Llama/Mistral/Qwen2 metadata and floating-point output. Synthetic Llama FP32 has passed pinned llama.cpp CPU validation; other architectures and precisions remain unverified. Full options, JSON/exit-code contracts, and migration notes are in [docs/CLI.md](docs/CLI.md).

---

## Research Stages

### Stage A — Naive PTQ

Purpose: prove the pipeline.

- Load model
- Find quantizable tensors
- Group weights
- Compute scale
- Map weights to ternary values
- Dequantize for fake-quant inference
- Run benchmark
- Save experiment metadata

Performance may be poor. That is acceptable.

### Stage B — Calibration / Reconstruction

Purpose: preserve the original model's internal behavior.

Potential research directions include:

- learnable scale
- learnable threshold
- softened ternarization
- layer/window reconstruction
- teacher/student activation matching
- sensitivity-aware mixed precision

These methods must be implemented and validated incrementally.

### Stage C — Deployment

Purpose: turn research weights into something useful.

- packed ternary representation
- runtime integration
- GGUF or other practical export
- RAM / VRAM / throughput measurement

---

## Repository Structure

```text
OpenTernary/
├─ src/openternary/
│  ├─ cli/
│  ├─ config/
│  ├─ services/
│  ├─ backends/
│  ├─ experiment/
│  ├─ utils/
│  ├─ adapters/
│  ├─ quant/
│  ├─ calibration/
│  ├─ benchmark/
│  └─ export/
├─ configs/
├─ docs/
├─ tests/
├─ pyproject.toml
├─ uv.lock
├─ .python-version
├─ README.md
├─ ROADMAP.md
├─ PROJECT_SPEC.md
├─ ARCHITECTURE.md
├─ RESEARCH_PLAN.md
├─ BENCHMARKS.md
├─ AGENTS.md
├─ EXPERIMENT_LOG.md
└─ CONTRIBUTING.md
```

---

## AI Development Workflow

OpenTernary is initially developed with:

- **DSH — lead development/research agent**
- **Codex — implementation/review sub-agent**

The exact division of responsibilities is defined in [`AGENTS.md`](./AGENTS.md).

High-level rule:

```text
DSH
Plan / Research / Integrate
        ↓
Codex
Focused implementation / tests / review
        ↓
DSH
Merge / benchmark / next experiment
```

No agent should silently modify the experimental target, benchmark protocol, or model scope.

---

## Definition of the First MVP

The first MVP is complete when all of the following work:

- [ ] Gemma 4 E2B can be inspected
- [ ] Quantizable tensors are listed correctly
- [ ] Baseline inference works
- [ ] Baseline benchmark can be reproduced
- [ ] Naive group-wise ternary fake quantization works
- [ ] Quantized model can run inference without NaN/crash
- [ ] Baseline vs ternary metrics can be compared
- [ ] Run configuration is saved automatically
- [ ] At least one Japanese evaluation set is held out from calibration
- [ ] Tests cover core ternary math and tensor mapping

The MVP does **not** require:

- competitive quality
- Bonsai-level results
- complete CAT-Q reproduction
- GGUF export
- GUI
- MoE support

---

## Long-Term Vision

OpenTernary should eventually become a model-agnostic research and conversion toolkit:

```text
Gemma
Qwen
Llama
DeepSeek-style models
Other open-weight LLMs
        ↓
OpenTernary
        ↓
Inspect
Quantize
Calibrate
Benchmark
Export
```

A later GUI can make these workflows accessible without hiding the underlying experiment configuration.

---

## Status

The CLI product status is recorded separately in [CLI_IMPLEMENTATION_STATUS.md](docs/CLI_IMPLEMENTATION_STATUS.md). The research milestones below are historical evidence; the new CLI integrations have not yet been tested with real models.

**Phase 3 — Model-Level Ternarization (bounded sharded fake-quant): CLOSED (2026-08-21)**

* Phase 0 (Foundation) ✅, Phase 1a (Inspection) ✅, Phase 1b (BF16 Baseline v1) ✅, Phase 2 (AbsMean + 2bit-v1) ✅, **Phase 3 (LD-RW per_group + bounded sharded fake-quant) ✅**
* Per-tensor `scale=mean(abs(W))` and per-group LD-RW `last-dim-rowwise-v1` vectorized, zero-branch, `G>=D` → `rows` groups, `00=-1/01=0/10=+1` packed `ceil(N/4)` per-tensor sum, `ideal_ternary_bits≈1.585` vs `packed` vs `scale_overhead` (per_tensor `205*32` vs per_group `14.3M*32`) separated
* Bounded sharded writer `512 MiB` flush-before-add, over-size single allowed, HF `blobs/` symlink dereferenced, content fingerprint MUST vs file hash SHOULD, JSON stores `scale_stats` + `scale_fingerprint` (no scale array)
* Whole-model `e2b-naive-pt` (12 shards, 10.2GB) and `e2b-naive-g128-rw` (14.3M groups, 12 shards) both loadable via `AutoModelForMultimodalLM` and smoke `protocol True` / `result observational different` PASS
* See `ROADMAP.md` for M1–M13 and `EXPERIMENT_LOG.md` for whole-model validation

Current focus:

> Phase 4 — Calibration / reconstruction (learnable scales) — `inspect→quantize→benchmark→compare` pipeline now stable

---

## License

The source code is licensed under the [Apache License 2.0](LICENSE). Copyright 2026 The OpenTernary Authors.

- Forks and other redistributions must keep the [NOTICE](NOTICE) file, which names this project as the origin (Apache-2.0, Section 4(d)), and mark files they changed (Section 4(b)).
- The OpenTernary name and logo are **not** licensed under Apache-2.0: see [TRADEMARKS.md](TRADEMARKS.md). Public forks should use their own name and logo.
- Contributions are accepted under the same license (Apache-2.0, Section 5).

The project must not copy code from external implementations until their licenses and attribution requirements have been checked.
