Metadata-Version: 2.4
Name: gemma-xdna2
Version: 0.1.0a2
Summary: Pinned full-model Gemma 3 1B inference on AMD XDNA2 with frozen PDI kernels
License-Expression: Apache-2.0 AND BSD-3-Clause AND MIT AND BSL-1.0
Classifier: Development Status :: 3 - Alpha
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3.12
Requires-Python: <3.13,>=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
License-File: BINARY_DISTRIBUTION.md
License-File: src/gemma_xdna2/licenses/AIE-API-LICENSE.txt
License-File: src/gemma_xdna2/licenses/Apache-2.0.txt
License-File: src/gemma_xdna2/licenses/Boost-COPYRIGHT.txt
License-File: src/gemma_xdna2/licenses/Cython-COPYING.txt
License-File: src/gemma_xdna2/licenses/Cython-LICENSE.txt
License-File: src/gemma_xdna2/licenses/ELFIO-LICENSE.txt
License-File: src/gemma_xdna2/licenses/GSL-LICENSE.txt
License-File: src/gemma_xdna2/licenses/MLIR-AIE-LLVM-LICENSE.txt
License-File: src/gemma_xdna2/licenses/NOTICE.txt
License-File: src/gemma_xdna2/licenses/XRT-LICENSE.txt
License-File: src/gemma_xdna2/licenses/XRT-NOTICE.txt
License-File: src/gemma_xdna2/licenses/pybind11-LICENSE.txt
Requires-Dist: numpy==2.4.6
Requires-Dist: transformers==5.13.0
Requires-Dist: huggingface-hub==1.22.0
Requires-Dist: jinja2==3.1.6
Dynamic: license-file

# Gemma XDNA2

Experimental full-model Gemma 3 1B IT inference on AMD XDNA2, using the accepted
`full26-c32768-selected` PDI from the restored September 2026 research archive.
Version `0.1.0a2` is prepared locally; it has not been published to PyPI or the Hub.

The wheel includes a C++ XRT session object, seven Cython AOT host modules,
the frozen PDI execution set, a private XRT 2.23.0 runtime and the Apache-licensed
userspace XDNA shim. No own Python source or bytecode is distributed. It does not install a kernel driver,
firmware, compiler, or system XRT. Weights and tokenizer files are in a separate
Hugging Face model bundle, staged in `release/huggingface`.

## Run the local release

Use CPython 3.12 on Linux x86_64. The tested host is Ubuntu 26.04.1, kernel
7.0.0-43-generic, `/dev/accel/accel0`, RyzenAI-npu6/aie2p, firmware 1.1.2.64.
The PyPI wheel is audited as `manylinux_2_39_x86_64`. It requires glibc >= 2.39
and libstdc++ providing GLIBCXX_3.4.32 and CXXABI_1.3.15. The manylinux tag
describes user-space ABI compatibility; other NPU/driver combinations remain
unqualified.

```bash
uv venv --python 3.12.16 .venv-run
uv pip install --python .venv-run/bin/python \
  dist/gemma_xdna2-0.1.0a2-cp312-cp312-manylinux_2_39_x86_64.whl
.venv-run/bin/gemma-xdna2 doctor --probe
.venv-run/bin/gemma-xdna2 verify release/huggingface
.venv-run/bin/gemma-xdna2 generate --model release/huggingface \
  --prompt 'What is the capital of France?' --max-tokens 24 --json
```

The user must have read/write access to the NPU device and at least 2 GiB of
memory-lock allowance. `doctor` reports both. A hard limit below 2 GiB needs an
administrator to raise it using `prlimit` for the launching process or a PAM
`limits.d` entry for future logins. This project never changes those settings
automatically. On the recovery host, `odroid` now has a persistent 4 GiB PAM
limit in `/etc/security/limits.d/90-gemma-xdna2.conf`; the current Codex parent retains an older 8 MiB hard limit. One-shot jobs
launched through the existing cron PAM configuration obtain the configured
4 GiB limit, remove their temporary cron entry and execute the validation.
A fresh SSH login itself was not exercised.

After publication, download using the exact Hub commit:

```bash
gemma-xdna2 download --repo yongseungyu/Gemma-3-1B-IT-XDNA2-Q4_1 \
  --revision <published-40-character-commit> --output ./gemma-model
gemma-xdna2 generate --model ./gemma-model --prompt 'Hello' --max-tokens 32
```

The repository name above is proposed. Its commit is assigned only when it is
published. Authentication uses the standard Hugging Face environment or token
store; no credential is bundled.

## Execution and numerical contract

All 26 transformer layers, embedding, normalization, RoPE, attention, MLP,
residuals, KV writes and full-vocabulary head run on the NPU. The host writes
parameters, starts/waits, reads logits, checks finiteness and chooses greedy
argmax. There is one persistent hardware context and one initial weight load.
Requests are serialized through a per-user NPU lock. There is no CPU fallback.

Projection weights use the archived native Q4_1 packet layout, with BF16 scale
and minimum per group. Original BF16 embedding/norms are retained; activations
and KV use the kernel's int8-g32 contract. This is a custom artifact format and
is consumed by this package, independently of Transformers model execution.

Prompt tokens are ingested through sequential full-model NPU steps. Optimized
batched prefill and native llama.cpp integration are future work. The image
has 32768-token allocation capacity. The 128--32K execution/timing sweep and
strict KV diagnostic findings are documented in `docs/context-sweep.md`;
complete long-context quality qualification is still pending. Generation is deterministic greedy
decoding; sampling is not implemented.

See [validation](docs/validation.md), [rebuild instructions](docs/rebuild.md) and
[release preparation](docs/release.md). Model quality remains `REPORT_ONLY`.

## Licenses

Project runtime software and restored kernels retain Apache-2.0; bundled
components retain their Apache/BSD/MIT/Boost licenses and notices.
Only compiled project code is distributed; this does not withdraw those
license grants or make binaries impossible to inspect. See
[BINARY_DISTRIBUTION.md](BINARY_DISTRIBUTION.md). The separate Gemma-derived weights are governed by the
[Gemma Terms of Use](https://ai.google.dev/gemma/terms). The model bundle contains
the terms, use policy, required notice, source attribution and modification log.
