Metadata-Version: 2.4
Name: eulernpu
Version: 0.1.0
Summary: eulernpu: NPU inference composition and simulation stack
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: click>=8.0
Requires-Dist: numpy>=1.26
Requires-Dist: pyyaml>=6.0
Requires-Dist: jsonschema>=4.20
Provides-Extra: demo
Requires-Dist: rich>=13.0; extra == "demo"
Provides-Extra: onnx
Requires-Dist: onnx>=1.15; extra == "onnx"
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0; extra == "dev"
Requires-Dist: rich>=13.0; extra == "dev"
Requires-Dist: ruff>=0.4; extra == "dev"
Dynamic: license-file

# eulernpu

**An inference-first NPU full-stack for edge AI — from a YAML spec, all
the way to real silicon on a $50 FPGA.**

`eulernpu` lets you describe a small inference accelerator and a model
as a reviewable YAML spec, then proves it on the host (`cpu_ref`),
simulates it on a functional NPU with cycle / memory estimates
(`npu_sim`), analyses PL offload for Zynq boards (`zynq_ps` /
`zynq_pl_stub`), and — when you want a physical proof — synthesises
the same graph through HLS → Vivado → xsdb and runs it on a real board.

```
   spec.yaml
      │
      ▼
   validate  ──►  compile  ──►  .npuart  ──►  run  /  sim  /  profile
                                   │                    │
                                   ▼                    ▼
                             explain                cycle & memory
                             (PL offload plan)      trace (JSON)
                                   │
                                   ▼
                             quantize (INT8 / INT4)
```

The same YAML that produces a `cpu_ref` golden also drives the
simulator, the board profiler, and the HLS template library that
generates RTL for real FPGAs. One spec, one graph, four backends,
one board.

---

## Table of contents

- [Why eulernpu](#why-eulernpu)
- [What it is / what it isn't](#what-it-is--what-it-isnt)
- [Highlights](#highlights)
- [Install](#install)
- [Quickstart — 60 seconds from spec to golden](#quickstart--60-seconds-from-spec-to-golden)
- [Tutorials — read these in order](#tutorials--read-these-in-order)
- [Silicon-proven demos](#silicon-proven-demos)
- [The CLI](#the-cli)
- [Architecture in one page](#architecture-in-one-page)
- [Testing](#testing)
- [License](#license)

---

## Why eulernpu

Every small-NPU project hits the same three cliffs:

1. **Spec drift.** The paper drawing, the RTL, and the simulator
   disagree; nobody knows which is authoritative.
2. **Silent mis-estimates.** A "quick cycle count" turns out to
   include a 2× init pass that nobody accounted for, and silicon is
   3× slower than expected. Or a `uint32_t` PMU sum overflows and
   the resulting speed-up is completely fictional.
3. **Board-shape lock-in.** A neat kernel that fits XC7Z020 falls
   off the DSP budget the moment you try it on a bigger graph.

`eulernpu` is built to make each of those cliffs *visible early*:

- **A single spec** is the ground truth. Validator, compiler, cost
  model, cycle model, HLS template library, and the runtime all
  consume the same YAML — there is no separate "hardware spec" that
  can drift.
- **Every measurement is deterministic and reproducible.** `cpu_ref`
  is byte-deterministic; `npu_sim` matches it within `1e-5`; the
  silicon cycle model is *anchored* to a real measurement (P06 on
  XC7Z020: predicted 456 cycles vs measured 460, ±1 %). A regression
  test fails CI if the model ever drifts more than ±10 %.
- **Board-realistic profiling is one command.**
  `eulernpu profile --all-boards` re-runs the same trace across
  every board profile in one table, so "will this fit ZU7EV?" is
  answered in a second, not in a full re-synthesis cycle.

If you have ever chased a fictitious 5× speed-up that turned out to
be a 32-bit counter wrap, that's the class of failure `eulernpu` is
designed to catch before you ever put a board on a bench.

---

## What it is / what it isn't

- ✅ A full-stack **inference accelerator** for small models
  (transformer / MLP / CNN / recurrent / SSM / robotics policy / LLM).
- ✅ **Operator-first, spec-first, host-first, board-realistic,
  measured design.** Every claim in this README has a test or a
  silicon capture behind it.
- ✅ Works on a **$50 FPGA** (QMTECH XC7Z020 CLG484-1). Scales up
  through XC7Z045 / XCZU7EV / Kria KV260 / XCZU9EG.

- ❌ Not a general-purpose GPU or OpenCL compatible.
- ❌ Not a graphics product.
- ❌ Not a training framework — the ML training happens in PyTorch
  (or whatever you like); `eulernpu` takes the trained model and
  puts it on hardware.
- ❌ Not a promise that every NPU is faster than a CPU. Sometimes
  the CPU wins, and `eulernpu` is designed to *measure honestly when
  it does*. See the LLM silicon tutorial for a worked example.

---

## Highlights

- **138 operators** in 17 groups (A – Q). Core math, conv / vision,
  attention (incl. flash / sliding-window / grouped-query),
  normalisation, MoE, recurrent, graph, multimodal, Mamba / SSM,
  KV-cache compression, autonomy / perception, vision-encoder,
  diffusion, speculative decoding.
- **4 backends** — `cpu_ref` (deterministic NumPy reference),
  `npu_sim` (functional sim + per-op cycle / MAC / memory trace),
  `zynq_ps` (ARM PS emulation), `zynq_pl_stub` (PL offload
  analysis with real-value fallback).
- **INT8 / INT4 quantization** with a calibration workflow
  (`eulernpu quantize`) and a wrap-safe fake-fixed-point host sim
  that matches silicon math byte-for-byte.
- **Board-realistic cycle model** anchored to real silicon
  measurements. `profile --all-boards` compares the same graph
  across 5 board profiles in one table.
- **Two-stage validation** — JSON Schema (Draft 7) + 20 semantic
  rules (S01 – S23) with a 3-line actionable error format.
- **Interop** — ONNX importer (`pip install eulernpu[onnx]`) and a
  Verilator testbench generator for plugging in your own RTL.
- **i18n CLI** — help / logs / errors localise via
  `--lang ko|en|zh|ja|es` (default `ko`); command names, op names,
  and rule codes stay English (they are API surface).

---

## Install

```bash
pip install eulernpu               # core
pip install eulernpu[demo]         # + rich, for terminal demos
pip install eulernpu[onnx]         # + onnx, for the ONNX importer

# development
pip install -e ".[dev]"            # tests + rich + typing
```

Requires **Python ≥ 3.10** and NumPy / Click / PyYAML / jsonschema.
Optional extras: `demo` (rich), `onnx` (onnx).

Silicon bring-up (KWS / bearing / LLM demos) additionally uses
**Vitis HLS 2025.1**, **Vivado 2025.1**, and **xsdb** on a Linux host
with a **QMTECH XC7Z020 CLG484-1** or equivalent Zynq-7000 / ZynqMP
board. None of these are needed to run the Python side (validate,
compile, sim, profile, quantize).

---

## Quickstart — 60 seconds from spec to golden

```bash
# 1. Show what's supported on this install
eulernpu info                       # backends + capability matrix
eulernpu info --ops                 # 138 operators grouped by category

# 2. Validate a spec YAML (schema + 20 semantic rules)
eulernpu validate examples/tiny_mlp/spec.yaml

# 3. Compile it into an artifact (.npuart)
eulernpu compile --spec examples/tiny_mlp/spec.yaml \
                 --target cpu_ref
# → build/tiny_mlp.npuart

# 4. Run and compare against a golden
eulernpu run --artifact build/tiny_mlp.npuart \
             --backend cpu_ref \
             --golden examples/tiny_mlp/golden.npz

# 5. Same graph, but now on the NPU simulator with a full trace
eulernpu sim --artifact build/tiny_mlp.npuart \
             --backend npu_sim --trace trace.json

# 6. Multi-board comparison of the same graph
eulernpu profile --artifact build/tiny_mlp.npuart --all-boards

# 7. INT8 quantize using a calibration dataset
eulernpu quantize --artifact build/tiny_mlp.npuart \
                  --calib-data data/calib/ \
                  --out build/tiny_mlp_int8.npuart
```

The `examples/` directory ships several models you can run this
sequence against (MLP, CNN, single-block Transformer, LSTM anomaly
detector, transformer policy, quantized CNN, MoE, decoder + KV
cache, and several op-level demos). Each ships a `spec.yaml`,
`weights.npz`, `input.npz`, and a `golden.npz` for the compare step.

---

## Tutorials — read these in order

The tutorials are the intended entry point. They are short (100 – 200
lines each), fully runnable, and available in **Korean and English**:

| # | Topic | Korean | English |
|---|-------|--------|---------|
| 01 | Overview — identity, spec layers, backends | [ko](docs/tutorials/ko/01_overview.md) | [en](docs/tutorials/en/01_overview.md) |
| 02 | Validate — schema + 20 semantic rules | [ko](docs/tutorials/ko/02_validate.md) | [en](docs/tutorials/en/02_validate.md) |
| 03 | Compile / Run — artifact → result → golden | [ko](docs/tutorials/ko/03_compile_run.md) | [en](docs/tutorials/en/03_compile_run.md) |
| 04 | `npu_sim` — cycle model + trace + profile | [ko](docs/tutorials/ko/04_npu_sim.md) | [en](docs/tutorials/en/04_npu_sim.md) |
| 05 | Quantize — INT8 / INT4 workflow | [ko](docs/tutorials/ko/05_quantize.md) | [en](docs/tutorials/en/05_quantize.md) |
| 06 | Model examples — catalog | [ko](docs/tutorials/ko/06_models.md) | [en](docs/tutorials/en/06_models.md) |
| 07 | FPGA overview — Zynq + 7-stage silicon flow | [ko](docs/tutorials/ko/07_fpga_overview.md) | [en](docs/tutorials/en/07_fpga_overview.md) |
| 08 | **KWS silicon** — canonical, 8.07× speed-up | [ko](docs/tutorials/ko/08_kws_silicon.md) | [en](docs/tutorials/en/08_kws_silicon.md) |
| 09 | Bearing silicon — deeper graph, 11.13× | [ko](docs/tutorials/ko/09_bearing_silicon.md) | [en](docs/tutorials/en/09_bearing_silicon.md) |
| 10 | LLM L2 hybrid silicon — honest slowdown | [ko](docs/tutorials/ko/10_llm_silicon_hybrid.md) | [en](docs/tutorials/en/10_llm_silicon_hybrid.md) |

If you only have ten minutes: read [01_overview](docs/tutorials/en/01_overview.md)
and [08_kws_silicon](docs/tutorials/en/08_kws_silicon.md).

---

## Silicon-proven demos

Three demos have been synthesised end-to-end and run on real silicon
(QMTECH XC7Z020 CLG484-1). Every number below comes from a UART
capture on a board, not a prediction:

| Demo | What | Silicon Result |
|------|------|----------------|
| **KWS** (tutorials [08](docs/tutorials/en/08_kws_silicon.md)) | DS-CNN + GRU keyword spotter (11 commands), full graph as one PL IP (Apple-ANE style) | 11 / 11 CPU ↔ NPU MATCH · **NPU 8.07× faster than ARM Cortex-A9** (3.17 ms vs 25.7 ms per inference) |
| **Bearing** (tutorial [09](docs/tutorials/en/09_bearing_silicon.md)) | 4-class CWRU vibration fault classifier (Conv1D × 3 + GAP + FC) | 40 / 40 MATCH · 200-window monitor 150 / 150 fault detect, 0 false alarm · **NPU 11.13× faster** (4.0 ms vs 44 ms) |
| **LLM** (tutorial [10](docs/tutorials/en/10_llm_silicon_hybrid.md)) | **10.77 M-parameter nanoGPT**, L2 hybrid: FFN block on PL, embed / attention + KV / LayerNorm / sampling on ARM | 5 / 5 byte-equal MATCH · live 50-char Shakespeare generation · **NPU 2.20× *slower* per token** (honest wrap-safe benchmark — bottleneck: 6 m_axi pointers share one `bundle=gmem` and serialise weight bursts) |

These three form a **silicon ladder**:

1. **KWS** proves the toolchain end-to-end on a graph that fits
   in BRAM: single IP, weights baked in as ROM, one `ap_start` per
   inference. Cleanest "the toolchain works" evidence.
2. **Bearing** carries the same L3 pattern to a **deeper graph** (3
   Conv1D layers) and hits a resource ceiling — first-pass II=4
   spilled to DSP 231 % / LUT 265 %; `ALLOCATION operation
   instances=fmul/fadd limit=4` + II=64 restored the budget.
3. **LLM** is a **deliberate departure** — a 10.77 M-parameter
   model does not fit in 140 KB of BRAM, so only the FFN block goes
   to PL and the rest stays on ARM. The measurement is *honestly
   slower* than the CPU on this chip; the wrap-safe benchmark
   (`BENCHMARK.md`) names the exact bottleneck and the four ways to
   flip the ratio on a larger board.

The LLM demo is deliberately included even though the NPU is
slower — because **that is what measuring instead of guessing looks
like**. The same toolchain that ships an 11× speed-up in one demo
tells you the truth when there isn't one in another.

---

## The CLI

Every feature is a subcommand of the single `eulernpu` entrypoint.
Hyphen-joined variants (`eulernpu-info`, `eulernpu-compile`, …) are
not supported.

| Command | What |
|---------|------|
| `eulernpu info [--ops] [--dtypes]` | Backend capabilities, operator groups, dtypes |
| `eulernpu validate <spec>` | Schema + semantic validation |
| `eulernpu compile --spec <yaml> --target <backend>` | Spec → `.npuart` artifact |
| `eulernpu run --artifact <art> --backend <backend>` | Execute (with optional `--golden`) |
| `eulernpu sim --artifact <art> --backend npu_sim` | Functional sim + trace + per-op cycles |
| `eulernpu profile --artifact <art> [--all-boards]` | Per-op profile (or 5-board comparison) |
| `eulernpu sweep --artifact <art> [--systolic 8,16,32]` | Board × systolic Pareto sweep |
| `eulernpu quantize --artifact <art> --calib-data <dir>` | INT8 quantization workflow |
| `eulernpu explain --artifact <art>` | Graph + memory plan + PL offload analysis |
| `eulernpu generate --artifact <lm.npuart>` | Autoregressive token generation (decoder LMs) |
| `eulernpu migrate-spec <spec> --from 0.4` | Bump `spec_version` + list new requirements |
| `eulernpu board smoke --target <zynq_ps\|zynq_pl_stub>` | Board smoke test (emulated) |
| `eulernpu benchmark / replay / calibrate / compress-cache` | Profiling utilities |

Every error message is three lines:

```
<Category>: <what happened>
Fix: <one-line guidance>
See: <doc path>
```

Full reference: [`docs/cli.md`](docs/cli.md).

---

## Architecture in one page

- [**`docs/architecture.md`**](docs/architecture.md) — a one-page
  summary: system layers, data flow, spec structure, compiler
  pipeline, artifact format, runtime dispatch, `npu_sim` cost model,
  PL partitioning, and the determinism / reproducibility
  guarantees the whole stack rests on.

Reading it before the tutorials makes CLI output and error messages
easier to place.

---

## Testing

```bash
python -m pytest                       # full suite
python -m pytest tests/hls/            # HLS-kernel equivalence + silicon anchor
python -m pytest tests/integration/    # end-to-end integration
```

Test strategy, coverage layers, and CI gates:
[`docs/testing.md`](docs/testing.md).

Highlights the suite enforces:

- Every valid spec fixture is compiled and executed on `cpu_ref`.
- Every invalid spec fixture is checked to fail with the expected
  rule code (regression assets — never deleted, never weakened).
- `npu_sim` numerical equivalence with `cpu_ref`
  (`atol = rtol = 1e-5`).
- Silicon anchor: `matmul_systolic` cycle model must stay within
  ±10 % of the P06 measured 460 cycles.

---

## License

`eulernpu` is dual-licensed.

### Open Source — AGPL v3

Available as free, open-source software under the
[GNU Affero General Public License v3.0](LICENSE) (AGPL-3.0).

Under the AGPL, you may freely use, modify, and distribute
`eulernpu` provided that any modifications and any software that
incorporates or links with `eulernpu` — **including network
services** — are also released under the AGPL.

Ideal for:

- Academic and research use
- Open-source projects
- Internal evaluation

### Commercial License

A **commercial license** is available for organisations that:

- Distribute `eulernpu` as part of a proprietary product or embedded
  system
- Provide SaaS / API services built on `eulernpu` without
  open-sourcing their stack
- Require OEM or redistribution rights (defence, industrial,
  medical devices)
- Need support agreements or IP indemnification

**Contact**: licensing@eulerwa.com
**Details**: [COMMERCIAL_LICENSE.md](COMMERCIAL_LICENSE.md)

> **Not sure which applies?** If your use of `eulernpu` is entirely
> internal and never exposed over a network, standard AGPL terms
> likely apply without a commercial license. For any commercial
> deployment or distribution, contact us.
