Metadata-Version: 2.4
Name: laya-apple
Version: 1.6.0
Summary: Correctness-validated heterogeneous Laya runtime for Apple Silicon (MLX GPU + Apple Neural Engine)
Keywords: laya,apple-silicon,mlx,coreml,apple-neural-engine,ane,inference,heterogeneous-computing,machine-learning
Author: oliver
Author-email: oliver <tc3oliver@gmail.com>
License-Expression: Apache-2.0
License-File: LICENSE
License-File: NOTICE
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: MacOS
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Environment :: GPU
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Typing :: Typed
Requires-Dist: mlx>=0.32.2,<0.33 ; platform_machine == 'arm64' and sys_platform == 'darwin'
Requires-Dist: numpy>=1.26
Requires-Dist: tokenizers>=0.21,<1
Requires-Dist: huggingface-hub>=0.34,<2
Requires-Dist: safetensors>=0.6
Requires-Dist: coremltools==9.0 ; extra == 'ane'
Requires-Dist: numpy>=1.26,<2.2 ; extra == 'ane'
Requires-Dist: pyobjc-framework-coreml==12.2.2 ; sys_platform == 'darwin' and extra == 'ane'
Requires-Dist: coremltools==9.0 ; extra == 'convert'
Requires-Dist: numpy>=1.26,<2.2 ; extra == 'convert'
Requires-Dist: torch==2.7.0 ; extra == 'convert'
Requires-Dist: pytest>=8 ; extra == 'dev'
Requires-Dist: ruff>=0.12 ; extra == 'dev'
Requires-Dist: psutil>=6 ; extra == 'dev'
Requires-Dist: httpx>=0.27,<1 ; extra == 'dev'
Requires-Dist: torch==2.7.0 ; extra == 'reference'
Requires-Dist: transformers==5.17.0 ; extra == 'reference'
Requires-Dist: laya==0.3.20 ; extra == 'reference'
Requires-Dist: fastapi>=0.115,<1 ; extra == 'serve'
Requires-Dist: uvicorn>=0.30,<1 ; extra == 'serve'
Requires-Python: >=3.11, <3.14
Project-URL: Homepage, https://github.com/tc3oliver/laya-apple
Project-URL: Documentation, https://github.com/tc3oliver/laya-apple/blob/main/docs/guide.md
Project-URL: Changelog, https://github.com/tc3oliver/laya-apple/blob/main/CHANGELOG.md
Project-URL: Issues, https://github.com/tc3oliver/laya-apple/issues
Project-URL: Repository, https://github.com/tc3oliver/laya-apple
Project-URL: Upstream, https://github.com/NandhaKishorM/laya
Provides-Extra: ane
Provides-Extra: convert
Provides-Extra: dev
Provides-Extra: reference
Provides-Extra: serve
Description-Content-Type: text/markdown

# laya-apple

**English** | [繁體中文](README.zh-TW.md) | [简体中文](README.zh-CN.md)

[![CI](https://github.com/tc3oliver/laya-apple/actions/workflows/ci.yml/badge.svg)](https://github.com/tc3oliver/laya-apple/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/laya-apple)](https://pypi.org/project/laya-apple/)
![Python 3.11–3.13](https://img.shields.io/badge/python-3.11%E2%80%933.13-blue)
[![License: Apache-2.0](https://img.shields.io/badge/license-Apache--2.0-green)](LICENSE)

**Correctness-validated, adaptive [Laya](https://github.com/NandhaKishorM/laya) inference on
Apple silicon:** the MLX GPU and the Apple Neural Engine, serving at the same time.

## Fetch validated ANE artifacts, then route and predict (1.6)

`laya-apple artifacts fetch` → `from_pretrained("auto")` → `predict` / `predict_shortlist`.

```bash
pip install -U 'laya-apple[ane]'
laya-apple artifacts fetch laya                  # prebuilt ANE artifacts, validated here
laya-apple artifacts fetch laya-multilingual
```

```python
from laya_apple import Laya

with Laya.from_pretrained("auto") as model:       # laya or laya-multilingual, per request
    result = model.predict(context=context, questions=questions)
    rt = result.runtime
    print(rt.model, rt.model_routing, rt.device, rt.routing_reason)
```

- **Language routing.** `Laya.from_pretrained("auto")` returns a `LayaRouter` that loads
  `laya` and `laya-multilingual` and picks one per request by the language of the context,
  with the same function `laya-apple serve --model auto` uses. Why is in
  `RuntimeInfo.model_routing` ([`docs/api.md`](docs/api.md)).
- **Prebuilt ANE artifacts, checked on your Mac.** `laya-apple artifacts fetch MODEL`
  downloads a prebuilt artifact from
  [`tc3oliver/laya-apple-artifacts`](https://huggingface.co/tc3oliver/laya-apple-artifacts)
  and registers it through the validating `artifacts import`: SHA-256 against the index,
  manifest and platform profile, compute plan, the full FP16 parity gate and the placement
  probe, all on the receiving machine. A download is trusted no more than a local build.
  laya-apple 1.6.0 reads that repository at the commit it was validated against, never at
  its mutable `main`. It replaces the local build and its PyTorch dependency: the `convert` extra is not needed.
- **`predict` as before; `predict_shortlist` opt-in.** For `choice` questions with many
  labels, `predict_shortlist(..., embed_fn, k=20)` keeps the `k` labels most similar to the
  request, then runs one `predict`. `predict` itself is unchanged
  ([`examples/auto_fetch_shortlist.py`](examples/auto_fetch_shortlist.py)).

Limits:
- **Prebuilt artifacts exist for exactly one platform profile: Apple M4 Max, macOS 26,
  coremltools 9.0.** Every other Mac gets `ArtifactMissingError` naming the build command,
  and builds locally as before (`laya-apple artifacts build MODEL`, `convert` extra).
- **The published artifacts were checked by a clean-cache download** on that profile: fetch,
  verify, parity and placement passed for all 10 model/bucket pairs
  ([`benchmarks/prebuilt-artifacts-1.6.0.md`](benchmarks/prebuilt-artifacts-1.6.0.md)). That
  check ran on the build machine with an empty cache. Every receiving machine repeats the
  integrity, platform, parity and placement checks before it registers an artifact.
- **The first Core ML load still compiles on the device:** about 4.5 minutes. In one run,
  laya-typed-decisions (buckets 64/96/128) took 273.5 s cold at a new location against 2.7 s
  warm ([`research/coreml-compile-cache/screen.md`](research/coreml-compile-cache/screen.md)).
  Fetching does not remove it. `ane_startup="background"` serves on MLX in the meantime.

Also in 1.6, not headlines: an MLX fast path (a token-id cache on by default; `mx.compile`
and length-bucketed batching opt-in; see [Limitations](#limitations) for what was measured),
and W8 ANE research that ships nothing ([Limitations](#limitations)). Release notes:
[`docs/releases/v1.6.0.md`](docs/releases/v1.6.0.md).

## Adaptive GPU + Neural Engine serving (1.5)

laya-apple sends short, single-question decisions to the Apple Neural Engine (ANE) and keeps
long or multi-question work on the MLX GPU, with both engines serving at once. Since v1.0,
this split has kept short requests from queueing behind long GPU work.

**The GPU was done. Python was still waiting.** With the ANE on a thread in the same process,
the synchronous Core ML call holds the GIL for much of each prediction. A GPU request that had
already finished could not hand its result back until that call returned
([`research/coreml-gil-completion-path/`](research/coreml-gil-completion-path/README.md)).
In 1.5, laya and laya-typed-decisions run eligible ANE requests through asynchronous Core ML
execution, which avoids the synchronous path's long GIL hold. The runtime watches that faster
path, and on a sustained slowdown falls back to the known-safe 1.4 synchronous path. With
`execution="workers"` and `device="auto"` it is on by default and needs no other code change.

![The GPU finishes but its result waits on the GIL held by the synchronous Core ML predict; with asynchronous Core ML it goes straight through, GPU result return 8.60 to 0.043 ms; then a recorded run where the asynchronous path turns slow, and a controlled recovery test where 1.5 detects it, falls back to the 1.4 path and recovers: 12 of 12 within 164–414 ms](https://raw.githubusercontent.com/tc3oliver/laya-apple/main/docs/media/release15-social.gif)

The first 10 s are a schematic. Sources for every number on screen:
[`docs/media/release15-social.md`](docs/media/release15-social.md).

| vs the 1.4 path (one Apple M4 Max) | laya | laya-typed-decisions |
|---|---:|---:|
| GPU return (P50) | 4.28–4.29 → **0.035–0.037 ms** | 8.60 → **0.043 ms** |
| Throughput | **1.042×** | **1.038×** |

*GPU return* is the time from the GPU worker finishing a request to its result reaching the
caller, not GPU compute time. On the 1.4 path, finished work waited 4.3–8.6 ms, almost all
of it for the GIL.

- **Validation.** 154 production validation episodes (76 laya, 78 typed-decisions, including
  bursts and soaks). Every episode stayed on the asynchronous path after the handoff, with no
  mismatches, routing failures, lost requests or crashes. No slow state occurred in these
  runs, so the fallback never triggered in them
  ([`val_tables.md`](research/coreml-adaptive-breaker/val_tables.md)).
- **Recovery, a separate controlled experiment.** In 12 of 12 episodes where the asynchronous
  path was already slow, the runtime detected it within 40 ms and was back to the 1.4 path's
  latency within 164–414 ms
  ([`phase1_tables.md`](research/coreml-adaptive-breaker/phase1_tables.md)).

[Try Switchyard](#see-it-yourself-switchyard) ·
[Install](#install) · [Local Jev-compatible server](#local-jev-compatible-server)

## Install

Apple silicon, Python 3.11–3.13:

```bash
pip install 'laya-apple[ane]'   # MLX GPU + the Neural Engine runtime
```

| Extra | Adds |
|---|---|
| (none) | The MLX GPU runtime only |
| `ane` | The Neural Engine runtime (coremltools 9.0, pyobjc-framework-CoreML) |
| `convert` | Building ANE artifacts on this Mac (torch 2.7.0); not needed for `artifacts fetch` |
| `serve` | `laya-apple serve`, the local Jev-compatible server |

Without the `ane` extra, or without a built ANE artifact, everything runs on the MLX GPU. To
run from source or develop laya-apple, see [`CONTRIBUTING.md`](CONTRIBUTING.md).

## Quickstart (30 seconds)

```python
from laya_apple import Laya

model = Laya.from_pretrained("convaiinnovations/laya-typed-decisions", device="auto")

result = model.predict(
    context="The customer was charged twice for the same invoice and is frustrated.",
    questions={
        "urgency": {
            "type": "choice",
            "instructions": "How urgent is this?",
            "criteria": ["low", "medium", "high"],
        }
    },
)
rt = result.runtime
print(result.answers["urgency"]["choice"])
print(rt.device, rt.routing_reason, f"{rt.latency_ms:.1f} ms")
```

On the tested machine:

```text
high
ane validated_short_single_question_path 11.2 ms
```

- The first call downloads the pinned checkpoint. After that it works offline
  (`local_files_only=True`).
- Without an ANE artifact the same request runs on MLX, and `routing_reason` says why. On
  Apple M4 Max, macOS 26, coremltools 9.0, fetch a prebuilt one, validated on this Mac:
  `laya-apple artifacts fetch laya-typed-decisions`. On any other Mac, build and
  parity-validate one (with the `convert` extra):
  `laya-apple artifacts build laya-typed-decisions`.
- Probabilities, other question types and the full API: [`examples/`](examples/), the
  [user guide](docs/guide.md) and [`docs/api.md`](docs/api.md).

For concurrent GPU + ANE serving, use `execution="workers"` and submit requests from any thread:

```python
with Laya.from_pretrained("convaiinnovations/laya-typed-decisions", execution="workers") as model:
    futures = [model.submit(context=c, questions=q) for c, q in requests]   # thread-safe
```

## How it works

Laya answers typed questions about a context (`choice`, `score`, `noul`) in one forward pass.
A Mac has two engines that can run it, and each is faster for different requests.

![Requests of at most 128 tokens with one question and a validated artifact go to the Apple Neural Engine; longer, multi-question or unvalidated requests go to the MLX GPU; with execution="workers" both engines serve independent requests concurrently](https://raw.githubusercontent.com/tc3oliver/laya-apple/main/docs/readme/architecture.svg)

1. **Correct ANE execution.** A fast Core ML export is not necessarily a correct one. On the
   tested Mac, the ordinary export ran on the Neural Engine without any error and disagreed
   with upstream on up to 85 decisions. laya-apple uses an ANE artifact only after it passes
   a parity gate on the machine that uses it ([Correctness](#correctness)).
2. **Automatic routing.** The router picks a device before a request runs and records why in
   `routing_reason`. Under load it also compares the two queues' backlogs.
3. **Concurrent GPU + ANE serving.** With `execution="workers"`, the GPU runs in a worker
   process and the ANE on its own dispatcher, so short requests stop queueing behind long ones.
4. **Adaptive ANE execution (1.5).** After a conservative handoff at the start of each GPU + ANE
   overlap, eligible ANE requests use asynchronous Core ML, so finished GPU work is not held
   back by the synchronous path's long GIL hold. Per-request timing tells the runtime when
   that path slows down; a sustained slowdown sends the rest of the overlap back to the
   known-safe synchronous path. `ane_handoff=False` turns it off
   ([guide](docs/guide.md#adaptive-ane-execution-in-process-ane-the-default-since-15)).

| Request | Goes to | Why (measured on the tested Mac) |
|---|---|---|
| One question, ≤ 128 tokens, validated artifact present | **ANE** | Faster: laya-typed-decisions L128 takes 9.9 ms on the ANE against 12.2 ms on MLX (forward P50) |
| Longer context | **MLX GPU** | MLX is faster there: 19.2 ms at L256 and 71.0 ms at L1024 |
| Several questions | **MLX GPU** | MLX batches the questions; the ANE runs them one at a time |
| Unvalidated Mac, missing artifact, or no Core ML | **MLX GPU** | Recorded as `platform_not_validated`, `ane_artifact_unavailable` or `ane_runtime_unavailable` |

Full routing thresholds and calibration evidence: [`docs/support-matrix.md`](docs/support-matrix.md).

Completed requests can emit a `RequestTrace` (routing decision, queue, service and response
timing) through `trace=` ([`docs/api.md`](docs/api.md)). Adaptive execution watches the same
per-request timing; you do not need to pass `trace=` for it. Architecture:
[`docs/architecture.md`](docs/architecture.md).

## See it yourself: Switchyard

```bash
uvx laya-apple switchyard
```

![Switchyard: the same recorded timetable replayed GPU-only, where trains queue at red signals during rush hour, and GPU + ANE, where they flow into their platforms; then the result card: 1,407 of 1,422 trains late GPU-only against 0 with GPU + ANE](https://raw.githubusercontent.com/tc3oliver/laya-apple/main/docs/media/switchyard-launch.gif)

**Every train is a real Laya decision** ("which platform is clear?"). A red signal means the
train is waiting for the model's answer; a train is late when it has no answer within 100 ms.
Rush hour adds bursts of load, with long background requests on the GPU.

| Same timetable, same model | MLX GPU only | MLX GPU + Neural Engine |
|---|---:|---:|
| Late trains | 1,407–1,408 / 1,422 | **0 / 1,422** |
| P99 decision latency | 3,107.7–3,170.6 ms | **54.5–54.7 ms** |
| P99 queue wait | 3,095.9–3,158.8 ms | **39.6–42.8 ms** |

- Three standard runs on one Apple M4 Max (macOS 26.6.2, `laya-typed-decisions`). In every
  run both rounds gave the same answer for every train.
- Most of the gap comes from short decisions no longer waiting in the GPU queue while the long
  requests keep running there.
- The benchmark runs headless; the animation is a replay, not the measurement. These numbers
  are not comparable with the v1.0 results below (different rate, boundary and workload).

First-run download, setup, method and raw data: [`docs/switchyard.md`](docs/switchyard.md)
and [`benchmarks/switchyard/README.md`](benchmarks/switchyard/README.md).

## Local Jev-compatible server

`laya-apple serve` is a local stand-in for the Jev API. It serves the same API
(`POST /v1/systemone`) on loopback and answers with upstream Laya, running on your Mac. Point
an existing Jev client at it through its base-URL setting; the client's code does not change.

```bash
pip install 'laya-apple[serve,ane]'
laya-apple serve                                   # http://127.0.0.1:8642
export TYPESAFE_BASE_URL=http://127.0.0.1:8642     # then run your Jev client as usual
```

![A terminal: laya-apple serve starts locally; the released typesafe-sdk 0.7.1 for Python, unmodified, is pointed at it with TYPESAFE_BASE_URL and gets the answer fix_code; the same request with curl shows that laya-apple answered it with the laya checkpoint on the Apple Neural Engine](https://raw.githubusercontent.com/tc3oliver/laya-apple/main/docs/readme/serve-demo.svg)

- **Tested with 7 unmodified Jev clients** at their released versions, including the Python
  and JS SDKs and two Claude Code plugins
  ([`integrations/jev-plugins/README.md`](integrations/jev-plugins/README.md)).
- **The answers are Laya's, not Jev's.** 792 of 792 requests matched unmodified upstream
  `laya.serve` 0.3.20 within the FP16 parity gate
  ([`benchmarks/serve-compat/`](benchmarks/serve-compat/README.md)). No claim is made about
  accuracy relative to Jev.

API keys, models, security and client caveats: [`docs/serve.md`](docs/serve.md).

### Earlier serve benchmark

Measured on the 1.3 path with a 27B local LLM generating at saturation on the same Mac,
short-decision P99 was **47.2 ms** with heterogeneous `auto` serving against **122.3 ms**
GPU-only (one run on one M4 Max, `--model laya`). Run 3, on laya-apple 1.5.0 with its
defaults, measured **41.7 ms** (`auto`) against **79.5 ms** (`--device gpu`), and every
preregistered criterion passed. Adaptive execution was enabled, but its asynchronous path
engaged in 0 of 3,222 Neural Engine forwards, so run 3 measures 1.5 serve as shipped, not the
asynchronous path ([`benchmarks/serve/m4-max-r3/tables.md`](benchmarks/serve/m4-max-r3/tables.md)). The runs are separate campaigns, compared descriptively only.
LLM throughput cost, method and limits:
[`docs/serve.md`](docs/serve.md#beside-a-local-llm).

## Correctness

Parity against upstream Laya on PyTorch CPU FP32, over the shipped golden rows (v1.0).
Each cell gives hard mismatches, then the max probability error.

| Implementation on the tested Mac | laya | laya-multilingual | laya-typed-decisions |
|---|---|---|---|
| Ordinary Core ML export · `CPU_AND_NE` | ❌ 12, 0.56 | ❌ 85, 1.0 | ❌ 19, 0.42 |
| Ordinary Core ML export · `CPU_AND_GPU` | ✅ 0, 0.0066 | ✅ 0, 0.0059 | ✅ 0, 0.0028 |
| **laya-apple MLX FP16** | ✅ 0, 0.0037 | ✅ 0, 0.0045 | ✅ 0, 0.0017 |
| **laya-apple ANE FP16** | ✅ 0 (1 near-tie), 0.012 | ✅ 0, 0.013 | ✅ 0, 0.0077 |

- **The FP16 gate:** probability error ≤ 0.02 and 0 hard mismatches. Near-tie flips are
  listed, not hidden.
- **No silent fallback:** an explicit ANE request runs the validated artifact or raises.
- **The 1.5 asynchronous path** must match coremltools' output exactly on the goldens.

Definitions, every configuration tested and the fallback audit:
[`docs/correctness.md`](docs/correctness.md) and
[`docs/no-silent-fallback.md`](docs/no-silent-fallback.md).

## Original heterogeneous serving benchmark (v1.0)

These results showed the value of GPU + ANE serving and were measured before 1.5 adaptive
execution.

| Model | Throughput vs GPU-only | Short P99, GPU-only | Short P99, GPU + ANE |
|---|---:|---:|---:|
| laya | **2.92×** | 1538.0 ms | **108.5 ms** |
| laya-multilingual | **4.34×** | 2052.3 ms | **29.6 ms** |
| laya-typed-decisions | **4.57×** | 1592.9 ms | **79.5 ms** |

One Apple M4 Max, a short and a long request stream through one `Laya(execution="workers")`.
Short P99 is measured from arrival under open-loop bursty load, so queueing counts. The gain
comes from running both engines at once, not from raw ANE speed. Method and raw data:
[`benchmarks/v1.0.md`](benchmarks/v1.0.md).

## Supported models and platforms

| Model | max_len | MLX GPU | ANE buckets (explicit) | ANE buckets used by `auto` | Adaptive ANE execution |
|---|---:|---|---|---|---|
| [`convaiinnovations/laya`](https://huggingface.co/convaiinnovations/laya) | 512 | FP16 / FP32, any length | 64, 96, 128 | 64, 96, 128 | Yes |
| [`convaiinnovations/laya-multilingual`](https://huggingface.co/convaiinnovations/laya-multilingual) | 1024 | FP16 / FP32, any length | 64, 96, 128, 256 | 64, 96, 128 | No (worker-process ANE) |
| [`convaiinnovations/laya-typed-decisions`](https://huggingface.co/convaiinnovations/laya-typed-decisions) | 1024 | FP16 / FP32, any length | 64, 96, 128 | 64, 96, 128 | Yes |

Adaptive ANE execution applies with `execution="workers"` and `device="auto"`. Validated on one Apple M4 Max with macOS 26.6.2. On other Macs, `auto` stays on MLX until
ANE artifacts are built and calibrated there (`laya-apple calibrate`). Prebuilt artifacts
(`laya-apple artifacts fetch`) exist for Apple M4 Max, macOS 26, coremltools 9.0 only.
Details: [`docs/compatibility.md`](docs/compatibility.md).

## Community benchmarks

Every benchmark above ran on an M4 Max. The [community matrix](docs/community-benchmarks.md)
keeps results from other Macs separate, and already has an M4 Pro, an M4 and an M2 Pro (MLX
only). One command adds yours:

```bash
uv run python scripts/hardware_report.py --quick
```

The full steps, from `git clone` to the pull request, are in the
[guide](docs/community-benchmarks.md#add-your-mac).

## Reproduction

Each headline number above traces to a report, raw data, a command and an environment in
[`docs/reproducibility.md`](docs/reproducibility.md). The 1.6 prebuilt-artifact check is in
[`benchmarks/prebuilt-artifacts-1.6.0.md`](benchmarks/prebuilt-artifacts-1.6.0.md). The 1.5 evidence is in
[`research/coreml-adaptive-breaker/`](research/coreml-adaptive-breaker/README.md). The research
line that led to it, failed routes included, is mapped in [`research/README.md`](research/README.md).

## Limitations

- **One test machine.** Every benchmark, including the 1.5 validation and recovery runs, is
  from one Apple M4 Max on macOS 26.6.2. Routing thresholds are not assumed to hold on other
  Apple SoCs.
- **Adaptive execution hands off conservatively:** the first ANE requests of every GPU + ANE
  overlap run the 1.4 path. laya-multilingual, whose ANE runs in a worker process, does not
  use it.
- **The asynchronous path of 1.5 adaptive execution is not measured in `laya-apple serve`.**
  Serve uses adaptive execution by default. Its run-3 benchmark beside a local LLM ran with it
  enabled, but every one of 3,222 Neural Engine forwards ran on the 1.4 Core ML path
  ([`benchmarks/serve/m4-max-r3/tables.md`](benchmarks/serve/m4-max-r3/tables.md)).
- **Long and multi-question requests stay on the GPU,** which is faster for them. The ANE
  path is batch 1 only.
- **Isolation is partial.** Under concurrency, each stream's P99 is above its solo value.
- **Cold start** on a fresh artifact location costs 3–5 minutes of Core ML compile per
  model. `ane_startup="background"` serves on MLX in the meantime.
- **Prebuilt artifacts do not remove the cold start.** A fetched artifact still compiles on
  the device at first load: 273.5 s for laya-typed-decisions (buckets 64/96/128) in one run,
  against 2.7 s warm
  ([`research/coreml-compile-cache/screen.md`](research/coreml-compile-cache/screen.md)).
- **Core ML's compile cache grows without eviction:** 0.7–1.4 GB per bucket for every new
  artifact location (a fetch, an import, a moved cache). laya-apple never evicts it
  ([guide](docs/guide.md#artifact-lifecycle)).
- **Prebuilt artifacts exist for one platform profile only** (Apple M4 Max, macOS 26,
  coremltools 9.0), and have been fetched only on the machine that built them, with an empty
  cache ([`benchmarks/prebuilt-artifacts-1.6.0.md`](benchmarks/prebuilt-artifacts-1.6.0.md)).
  No independent check on a second machine of the same profile has been done yet. It is
  recommended, not required, before publishing ([`docs/publishing.md`](docs/publishing.md));
  each receiving machine still validates every artifact before registering it.
- **No quantized artifact ships.** In the W8 research only the laya-typed-decisions L64
  `w8-pt` result is reproduced (0.650 of FP16 latency); laya and laya-multilingual failed the
  parity gate ([`research/ane-w8/README.md`](research/ane-w8/README.md)).
- **The MLX fast-path gains (about 1–4%) come from a screen,** not a full benchmark
  ([`benchmarks/mlx-fast-path-screen/README.md`](benchmarks/mlx-fast-path-screen/README.md)).
- **Switchyard does not counterbalance round order:** with the standard seed, GPU + ANE ran
  first in all three runs.

Benchmark-specific limitations are documented with each experiment, for example option
order in [`docs/correctness.md`](docs/correctness.md#option-order), serve and client caveats
in [`docs/serve.md`](docs/serve.md#limits), and what has not been measured in
[`docs/compatibility.md`](docs/compatibility.md#not-measured).

## Contributing

The most useful first contribution is a benchmark from a Mac other than an M4 Max
([how](docs/community-benchmarks.md)).

[`CONTRIBUTING.md`](CONTRIBUTING.md) covers setup, test tiers and parity checks; coding
agents follow [`AGENTS.md`](AGENTS.md). Open work is labelled `good first issue`,
`help wanted` and `research`.

More: [user guide](docs/guide.md) · [stable API](docs/api.md) ·
[architecture](docs/architecture.md) · [changes](CHANGELOG.md) · [security](SECURITY.md).

Apache-2.0; see [`LICENSE`](LICENSE) and [`NOTICE`](NOTICE). Model weights are downloaded
from their pinned Hugging Face revisions and are not redistributed. This is an independent
project, not an official release of Convai Innovations, Apple or MLX.
