Metadata-Version: 2.4
Name: fermion-research
Version: 0.1.7
Summary: Five-value sub-2-bit LLMs: chat with a ~2 GB 8B container at native-runtime speed, load it as a Transformers model, or serve it on an OpenAI-compatible endpoint
Author: Fermion Research
Maintainer: Fermion Research
License-Expression: Apache-2.0
Project-URL: Homepage, https://fermionresearch.com
Project-URL: Documentation, https://fermionresearch.com/learn
Project-URL: Repository, https://github.com/fermionresearch
Project-URL: Models, https://huggingface.co/fermionresearch/Neutrino-8B
Project-URL: Receipts, https://github.com/fermionresearch/kit
Project-URL: Issue Tracker, https://github.com/fermionresearch/kit/issues
Keywords: llm,quantization,ternary,sub-2-bit,bitnet,inference,transformers,openai-api,cpu-inference,edge-ai
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.3
Requires-Dist: transformers>=5.0
Requires-Dist: numpy>=1.26
Requires-Dist: huggingface_hub>=0.23
Dynamic: license-file

# fermion

Five-value sub-2-bit language models by Fermion Research. One CLI chats with
a ~2 GB 8B container (TRTC v4) or a 2.5 KB QR-code model — and serves either
one on an **OpenAI-compatible HTTP endpoint** (`fermion serve`, new in 0.1.3),
so it drops into Open WebUI, Continue, LangChain, LlamaIndex or any agent
harness that speaks `/v1/chat/completions`.

> NAME FINAL 2026-07-24: `fermion` is the final pip/CLI name (rename chain
> from the pre-launch working slug documented in RELEASE_RUNBOOK.md §5).
> HF org `fermionresearch`, GitHub org `fermionresearch`. Model family =
> **Neutrino** (2026-07-24 launch-shape addendum): the CLI's default model
> is the one published SKU `fermionresearch/Neutrino-8B`, whose repo also
> bundles the prebuilt native `fermion-run` binaries under `bin/`.

## Quickstart (2 commands)

```
pip install fermion-research
fermion chat                # downloads fermionresearch/Neutrino-8B, opens REPL
```

That first run fetches **3.89 GB** — the container, the tokenizer, the config
and the native runner — and nothing else. The model repo also carries a GGUF
build and a coded transport that this CLI never opens; they are filtered out.
(`FERMION_DOWNLOAD_ALL=1` fetches the whole repo if you want them.)

With a local file (no download):

```
fermion chat --model /path/to/neutrino-8b_v4.bin         # 8B container
fermion chat --model /path/to/qr_chatmax_artifact.bin    # the QR-code model
fermion generate "hello" --model ... --max-new 32        # one-shot
fermion info --model ...                                 # header + integrity check
fermion serve --model ...                                # OpenAI-compatible API
```

### Sampling defaults: chat and serve are sampled, generate is deterministic

`fermion chat` and `fermion serve` default to the **graded shipping config** —
temperature 0.01, top-p 1.0, repetition penalty 1.05 over a 256-token window —
because that is the configuration the conversational surfaces were graded at.
`fermion generate` defaults to **deterministic greedy** (temperature 0, no
penalty) because it is the scriptable, pipeable path that our receipts,
`fermion verify` and the token-identity gates depend on reproducing. Every
knob is a flag on all three, so either default is one argument away.

### `fermion info` — did my download actually work?

`info` prints the container header **and proves the file is the whole file**:
it checks the length against the record structure the container itself
describes, and against the `sha256` the model repo's `MANIFEST.json` pins.
It exits non-zero on a truncated or altered container, so it is safe to use
in a script:

```
$ fermion info --model ./Neutrino-8B
./Neutrino-8B/neutrino-8b_v4.bin: TRTC v4 arch=3 layers=36 hidden=4096 vocab=151936 (3.61 GiB)
[fermion] integrity OK: 3,875,404,812 bytes · length matches its own record
          structure · length matches MANIFEST.json · sha256 matches MANIFEST.json

$ fermion info --model ./half-downloaded.bin ; echo $?
./half-downloaded.bin: INTEGRITY CHECK FAILED
  container is truncated: input embedding weights needs bytes up to 622,329,932
  but the file is only 67,108,864 bytes
1
```

`--no-checksum` skips the hash (the length checks always run).

## `fermion serve` — OpenAI-compatible endpoint

```
fermion serve                                  # 127.0.0.1:8000, default model
fermion serve --model /path/to/neutrino-8b_v4.bin --port 8000
```

```
POST /v1/chat/completions    messages, temperature, top_p, max_tokens, stop, stream
POST /v1/completions         plain-text completion for older clients
GET  /v1/models              the loaded model id
GET  /health                 ok + the loaded container's sha256
```

Streaming is real SSE (`data: {...}\n\n` chunks with `choices[].delta`,
terminated by `data: [DONE]`); non-streaming returns `choices[].message`
plus a `usage` block. Errors come back OpenAI-shaped (`{"error": {...}}`).

**Point any OpenAI client at it.** Python:

```python
from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="not-needed")
print(client.chat.completions.create(
    model="neutrino-8b_v4.bin",
    messages=[{"role": "user", "content": "What is 2+2?"}],
).choices[0].message.content)
```

**Open WebUI** — Settings → Connections → OpenAI API:

```
Base URL: http://127.0.0.1:8000/v1
API key:  not-needed          # any non-empty string
```

(Open WebUI in Docker: use `http://host.docker.internal:8000/v1`.)

**Continue** (`~/.continue/config.json`):

```json
{"models": [{"title": "Neutrino 8B", "provider": "openai",
             "model": "neutrino-8b_v4.bin", "apiKey": "not-needed",
             "apiBase": "http://127.0.0.1:8000/v1"}]}
```

**curl:**

```
curl http://127.0.0.1:8000/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"What is 2+2?"}],"max_tokens":32}'
```

Notes:

- **Localhost only by default.** The server is unauthenticated; pass
  `--api-key SECRET` to require `Authorization: Bearer SECRET`, and expect a
  printed warning if you bind a non-loopback `--host`. `--cors` adds
  `Access-Control-Allow-Origin: *` for browser-side clients (off by default).
- Sampler defaults are the **graded shipping config** (same as `fermion chat`,
  and unlike `fermion generate`, which stays greedy — see above): temperature
  0.01, top-p 1.0, repetition penalty 1.05 over a 256-token window. A client that
  asks for `temperature: 0` gets a deterministic argmax **with the repetition
  penalty still applied** on either backend — the C runtime no-ops its sampler
  flags at temp 0, so the native path emulates argmax on top of a live
  sampler rather than dropping the penalty. Send the vLLM-style
  `"repetition_penalty": 1.0` extension to turn the penalty off.
- One request at a time (single resident model, serialised behind a lock);
  `n > 1`, embeddings and function-calling are not implemented and say so.
- Chat templating is the same code path as `fermion chat`, and served output
  is gated token-identical to `fermion generate` at matched settings.
- `--draft PATH` attaches a second container as a speculative-decoding draft
  model (assisted generation; the draft must share the tokenizer).

## What backs it

### Two decode backends, and the CLI tells you which one you got

Since **0.1.4** the CLI runs the prebuilt native runtime by default.

| backend | what it is | when it is used |
|---|---|---|
| `native` | `bin/fermion-run-<platform>`, the C runtime downloaded with the model — the path every published tokens/s number was measured on | automatically, whenever a binary exists for your platform and `--device cpu` |
| `torch` | the `hf_ternary` reference loader + `transformers.generate` | everywhere else, and whenever you ask for it |

```
fermion info --model ...       # prints the active backend and why
fermion generate --backend torch ...   # force the reference path
fermion generate --backend native ...  # fail loudly instead of running slow
FERMION_THREADS=8 fermion chat ...     # override the thread count
```

`fermion serve` reports the same thing in `GET /health` as `"backend"`.

**Native runtimes exist for macOS arm64 and Linux x86-64 only.** On Windows,
Linux arm64, or with `--device cuda/mps`, you get the torch path: correct, and
one to two orders of magnitude slower. The published speed figures are native
figures and do not describe the torch path.

Two known differences on the native path, both deliberate and documented in
`fermion/native.py`:

- The C runtime disables its whole sampler at `--temp 0`, so a greedy request
  that also carries a repetition penalty is mapped to
  `--temp 0.01 --min-p 0.999 --seed 0` — argmax by construction, penalty
  intact, still byte-reproducible.
- Greedy output is token-identical to the torch reference **at float32**, not
  at the CLI's bfloat16 default, and identity is a near-tie property rather
  than a guarantee: two independent implementations pick different tokens when
  the top-2 logits are within measurement noise. Measured agreement and the
  divergence analysis are in `handoffs/results_pip_native/RESULTS.md`.

## What backs it

- **TRTC v4 containers** load through the `hf_ternary` integration (vendored
  verbatim, sha-recorded): native `Qwen3ForCausalLM` etc. with packed
  five-value planes resident (98.9% memory honesty), correctness-gated at
  0 greedy mismatches over 768 tokens vs the expander reference.
- The torch path is the honest **reference**: correct everywhere torch runs,
  fast nowhere. It is what `fermion verify` and `--draft` (speculative
  decoding is a torch-graph feature) use, and what the native path is gated
  against. Measured numbers live in the model card and eval-receipts, each
  with venue+version+date.
- CLI activation dtype defaults to **bfloat16** (`--dtype` to override):
  fp16 NaN-overflows at 8B scale (measured; receipts in
  `packaging/acceptance/`), while `--dtype float16` reproduces the
  fp16-identity receipts on the small twin containers gated that way.
- **Tiny models** (QR 2.5 KB GRU / GIF 170 KB transformer) run through the
  float64 reference decoder the browser demo is bit-exactness-gated against.

```python
import fermion
from transformers import AutoModelForCausalLM
fermion.write_transformers_config("neutrino-8b_v4.bin", "cfg-dir")
model = AutoModelForCausalLM.from_pretrained("cfg-dir")   # native Qwen3
```

## Dev

```
pip wheel --no-deps -w dist .     # build the wheel
python -m venv /tmp/v && /tmp/v/bin/pip install dist/*.whl
/tmp/v/bin/fermion --version
```

License: Apache-2.0 (flagship is a Qwen3-8B derivative, Apache-2.0 upstream).
