Metadata-Version: 2.4
Name: laya-mnn
Version: 0.1.0
Summary: Run the Laya System 1 decision model locally on MNN (CPU, no PyTorch, no API)
Author: yunfengwang
License: Apache-2.0
Project-URL: Homepage, https://huggingface.co/yunfengwang/laya-MNN-fp16
Project-URL: Source, https://huggingface.co/yunfengwang/laya-MNN-fp16
Keywords: mnn,onnx,decision-model,system-one,laya,calibration,edge-inference
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Operating System :: MacOS
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: Microsoft :: Windows
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Requires-Dist: mnn>=3.6.1
Requires-Dist: tokenizers>=0.15
Requires-Dist: numpy>=1.21
Dynamic: license-file

# laya-mnn

Run the [Laya](https://huggingface.co/convaiinnovations/laya) System 1 decision model **locally on MNN** — no PyTorch, no GPU, no API key.

Laya takes a **state** (email, ticket, JSON) plus **typed questions** and returns typed answers with calibrated probabilities in one forward pass. It never generates text, so there is nothing to parse and nothing to hallucinate. This package runs a converted Laya checkpoint on Alibaba's [MNN](https://github.com/alibaba/MNN) runtime.

```bash
uvx laya-mnn decide \
  --state "Hi, we were billed twice for March. Please refund the duplicate or we will cancel our plan." \
  --questions '{"department":{"type":"choice","instructions":"Which department should handle this?","criteria":{"billing":"invoices, payments, refunds","technical":"bugs, outages","other":"everything else"}},
                "churn_risk":{"type":"noul","instructions":"Does the user threaten to cancel?"}}'
```

```json
{
  "model": "laya-mnn",
  "answers": {
    "department": {
      "type": "choice", "choice": "billing",
      "probabilities": { "billing": 0.9631, "technical": 0.0165, "other": 0.0204 },
      "confidence": 0.8067, "action": { "act_probability": 0.9977 }
    },
    "churn_risk": { "type": "noul", "noul": 0.9476, "confidence": 0.9476, "action": { "act_probability": 0.9977 } }
  }
}
```

Weights download automatically on first use (about 850 MB for `fp16`).

## Question types

| type | answer | use for |
|---|---|---|
| `choice` | one of the `criteria` keys, plus a probability per option | routing, intent, classification |
| `score` | expected value over ordered `criteria` levels (0-based) | severity, urgency, sentiment strength |
| `noul` | `P(true)` | yes/no judgements |

`criteria` is defined **at request time** — new schemas do not need retraining. Up to 64 options per question with the shipped checkpoints.

## CLI

```
laya-mnn decide --state S --questions Q [--precision fp16|int8] [--threads N] [--quiet]
laya-mnn download [--precision ...]      # pre-fetch weights
laya-mnn info     [--precision ...]      # cache dir + manifest
```

`--state` accepts literal text, inline JSON, or `@path`. `--questions` accepts inline JSON or `@path`.

## Python

```python
from laya_mnn import LayaMNN, ensure_model

model = LayaMNN(ensure_model("fp16"), threads=4)
result = model.system_one(
    {"subject": "Duplicate charge", "body": "We were billed twice..."},
    {"department": {"type": "choice", "instructions": "Which department?", "criteria": ["billing", "technical", "other"]}},
)
print(result["answers"]["department"]["choice"], result["answers"]["department"]["probabilities"])
```

## Checkpoints

| precision | size | notes |
|---|---|---|
| `fp16` | 846 MB | weights stored in half precision; probability deltas vs. the original PyTorch model are below 1e-3 |
| `int8` | 581 MB | weight-only 8-bit quantization; larger deltas, check it against your own eval |

Both are conversions of `convaiinnovations/laya` (English, ModernBERT-large backbone, 421M parameters). The model was exported to ONNX (`opset 17`, eager attention, static 512-token sequence) and converted with the MNN converter 3.6.1.

Hosted on [Hugging Face](https://huggingface.co/yunfengwang/laya-MNN-fp16) and [ModelScope](https://modelscope.cn/models/yunfeng/laya-MNN-fp16); `modelscope` is tried first, then Hugging Face (override with `HF_ENDPOINT`).
Set `LAYA_MNN_CACHE` to change the cache directory (default `~/.cache/laya-mnn`).

## Notes and limits

- **CPU is the supported backend.** The MNN Metal backend was measured to return all-zero logits for this graph in some builds of the 3.6.1 wheel; `--backend metal` is exposed but not recommended.
- A 512-token forward pass costs roughly 1.3 s on an Apple Silicon CPU. Shorter states are cheaper — the graph is exported at a fixed 512 tokens, so the CLI pads and masks.
- Max 64 options per question, 512 tokens of context, English only. The multilingual checkpoint (`mmBERT-base`) is not converted here.
- Percentile-latency and calibration in the upstream model card are for the original PyTorch model; quantized results will differ slightly.

## License and attribution

Apache-2.0. The model, its prompt format and the calibration scheme come from [NandhaKishorM/laya](https://github.com/NandhaKishorM/laya) by Convai Innovations (Apache-2.0); this package is an independent MNN port and is not affiliated with or endorsed by them. See `NOTICE`.
