Metadata-Version: 2.4
Name: mblt-model-zoo
Version: 2.2.1
Summary: A codebase for pre-quantized AI models for Mobilint NPUs.
Author-email: "Mobilint Inc." <tech-support@mobilint.com>
License: BSD-3-Clause
Project-URL: Home, https://www.mobilint.com/
Project-URL: Repository, https://github.com/mobilint/mblt-model-zoo
Keywords: quantization,NPU,model zoo,pre-quantized models,inference,mobilint,mblt,aries,regulus,qb
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: System Administrators
Classifier: License :: OSI Approved :: BSD License
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: <3.13,>=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: mobilint-qb-runtime>=1.2.0
Requires-Dist: numpy>=1.26.0
Requires-Dist: torch>=2.4.1
Requires-Dist: opencv-python>=4.11.0.86
Requires-Dist: pillow>=11.1.0
Requires-Dist: faster-coco-eval
Requires-Dist: huggingface-hub
Requires-Dist: scipy
Requires-Dist: requests>=2.32.0
Requires-Dist: gdown>=5.2.0
Requires-Dist: tqdm
Requires-Dist: mblt-tracker>=1.0.1
Requires-Dist: PyYAML
Provides-Extra: transformers
Requires-Dist: transformers[serving]<=5.12.1,>=4.54.0; extra == "transformers"
Requires-Dist: sentencepiece; extra == "transformers"
Requires-Dist: torchvision; extra == "transformers"
Provides-Extra: melotts
Requires-Dist: transformers<=5.12.1,>=4.54.0; extra == "melotts"
Requires-Dist: g2p_en>=2.1.0; extra == "melotts"
Requires-Dist: anyascii>=0.3.2; extra == "melotts"
Requires-Dist: jamo>=0.4.1; extra == "melotts"
Requires-Dist: g2pkk>=0.1.1; extra == "melotts"
Requires-Dist: unidic>=1.1.0; extra == "melotts"
Requires-Dist: python-mecab-ko>=1.3.7; extra == "melotts"
Requires-Dist: soundfile; extra == "melotts"
Requires-Dist: gradio; extra == "melotts"
Requires-Dist: click; extra == "melotts"
Provides-Extra: onnxruntime
Requires-Dist: onnx; extra == "onnxruntime"
Requires-Dist: onnxruntime; extra == "onnxruntime"
Provides-Extra: onnxruntime-gpu
Requires-Dist: onnx; sys_platform != "darwin" and extra == "onnxruntime-gpu"
Requires-Dist: onnxruntime-gpu; sys_platform != "darwin" and extra == "onnxruntime-gpu"
Provides-Extra: qwen-asr
Requires-Dist: qwen-asr; extra == "qwen-asr"
Provides-Extra: qbcompiler
Requires-Dist: qbcompiler>=1.2.0; extra == "qbcompiler"
Requires-Dist: onnxruntime; extra == "qbcompiler"
Dynamic: license-file

# Mobilint Model Zoo

<!-- markdownlint-disable MD033 -->
<div align="center">
<p>
<a href="https://www.mobilint.com/" target="_blank">
<img src="https://raw.githubusercontent.com/mobilint/mblt-model-zoo/master/assets/Mobilint_Logo_Primary.png" alt="Mobilint Logo" width="60%">
</a>
</p>
</div>
<!-- markdownlint-enable MD033 -->

**mblt-model-zoo** is a curated collection of AI models optimized by [Mobilint](https://www.mobilint.com/)’s Neural Processing Units (NPUs).

Designed to help developers accelerate deployment, Mobilint's Model Zoo offers access to public, pre-trained, and pre-quantized models for vision, language, and multimodal tasks. Along with performance results, we provide pre- and post-processing tools to help developers evaluate, fine-tune, and integrate the models with ease.

## Installation

[![PyPI - Version](https://img.shields.io/pypi/v/mblt-model-zoo?logo=pypi&logoColor=white)](https://pypi.org/project/mblt-model-zoo/)
[![PyPI Downloads](https://static.pepy.tech/badge/mblt-model-zoo?period=total&units=INTERNATIONAL_SYSTEM&left_color=BLACK&right_color=GREEN&left_text=downloads)](https://clickpy.clickhouse.com/dashboard/mblt-model-zoo)
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/mblt-model-zoo?logo=python&logoColor=gold)](https://pypi.org/project/mblt-model-zoo/)

- Prepare environment equipped with Mobilint's NPU. In case you are not a Mobilint customer, please contact [us](mailto:tech-support@mobilint.com).
- Install **mblt-model-zoo** using pip:

```bash
pip install mblt-model-zoo
```

- If you want to install the latest version from the source, clone the repository and install it:

```bash
git clone https://github.com/mobilint/mblt-model-zoo.git
cd mblt-model-zoo
pip install -e .
```

Release notes are tracked in [CHANGELOG.md](CHANGELOG.md).

## Quick Start Guide

### Initializing a Vision Model

Vision models are loaded through `MBLT_Engine`. This is the same loading style used in
[`tests/vision`](tests/vision) and [`benchmark/vision`](benchmark/vision).

```python
from mblt_model_zoo.vision import MBLT_Engine

# Load a built-in model config.
# If model_path is empty, the MXQ file is downloaded from Hugging Face Hub and cached automatically.
model = MBLT_Engine(
    model_cls="resnet50",
    model_type="DEFAULT",
    model_path="",
    core_mode="global8",
)

# Load a different recipe defined in the model YAML.
model = MBLT_Engine(
    model_cls="resnet50",
    model_type="IMAGENET1K_V1",
    model_path="",
    core_mode="global8",
)

# Load from a local MXQ file instead of downloading from the Hub.
model = MBLT_Engine(
    model_cls="resnet50",
    model_type="DEFAULT",
    model_path="path/to/resnet50.mxq",
    core_mode="global8",
)

# Run a vision model with ONNX instead of MXQ.
# Local `.onnx` and `.mxq` paths auto-detect the framework from the suffix.
# If model_path is empty, the matching ONNX file is downloaded from the
# same Hugging Face repo and cached automatically.
model = MBLT_Engine(
    model_cls="alexnet",
    model_type="IMAGENET1K_V1",
    framework="onnx",
    model_path="",
)
```

`MBLT_Engine` accepts these main arguments:

- `model_cls`: Model name or YAML config path.
- `model_type`: Variant key defined in the model YAML. `DEFAULT` resolves to the default entry in that file.
- `framework`: Inference backend. When omitted, the engine infers `.mxq` and `.onnx` from `model_path` or `file_cfg.model_path` and otherwise falls back to `mxq`. `onnx` is supported for image classification, object detection, instance segmentation, and pose estimation.
- `model_path`: Local MXQ or ONNX path. Use `""` to download and cache the published artifact automatically for the selected framework.
- `mxq_path` and `onnx_path`: Backward-compatible explicit path aliases.
- `onnx_providers`: Optional ONNX Runtime provider order. By default, the engine prefers available GPU providers such as CUDA and falls back to CPU.
- `core_mode`: NPU execution mode. Supported values are `single`, `multi`, `global4`, and `global8`.

Model files are also available on our [HuggingFace Hub](https://huggingface.co/mobilint).

### Running Inference

You can pass an image path, PIL image, numpy array, or torch tensor to the preprocess pipeline.

```python
from mblt_model_zoo.vision import MBLT_Engine

model = MBLT_Engine(
    model_cls="resnet50",
    model_type="DEFAULT",
    model_path="",
    core_mode="global8",
)

image_path = "path/to/image.jpg"

try:
    input_img = model.preprocess(image_path)
    output = model(input_img)
    result = model.postprocess(output)

    result.plot(
        source_path=image_path,
        save_path="path/to/save/result.jpg",
        topk=5,
    )
finally:
    model.dispose()
```

For ONNX vision models, the preprocess and postprocess flow stays the same:

```python
from mblt_model_zoo.vision import MBLT_Engine

model = MBLT_Engine(
    model_cls="caformer_b36",
    framework="onnx",
)

try:
    input_img = model.preprocess("path/to/image.jpg")
    output = model(input_img)
    result = model.postprocess(output)
finally:
    model.dispose()
```

For object detection models such as YOLO, postprocessing thresholds are initialized from the model
YAML. You can either use those defaults directly:

```python
result = model.postprocess(output)
```

or override them once on the model before calling `postprocess()`:

```python
model.set_postprocess_thresholds(conf_thres=0.25)
result = model.postprocess(output)
```

Available vision models are documented in [mblt_model_zoo/vision/README.md](mblt_model_zoo/vision/README.md).

### Vision API Migration for 2.0.0

`mblt_model_zoo.vision` supports both the legacy top-level model imports and the task subpackage
imports. These imports are valid:

```python
from mblt_model_zoo.vision import ResNet50
from mblt_model_zoo.vision import YOLO11m
```

```python
from mblt_model_zoo.vision.image_classification import ResNet50
from mblt_model_zoo.vision.object_detection import YOLO11m
```

For new code, `MBLT_Engine` remains the preferred loading API:

```python
from mblt_model_zoo.vision import MBLT_Engine

model = MBLT_Engine(model_cls="resnet50", model_type="DEFAULT", model_path="", core_mode="global8")
```

The task subpackage imports remain available as compatibility wrappers around `MBLT_Engine`. You
can also inspect supported tasks and model names programmatically with `mblt_model_zoo.vision.list_tasks()`
and `mblt_model_zoo.vision.list_models()`.

For legacy class-style constructors, the old `product` argument is still accepted in `2.0.0` so
existing call sites do not fail immediately, but it is ignored by the YAML-backed model registry.
If you previously relied on `product` to choose a non-default artifact, migrate that selection to
explicit `model_cls`, `model_type`, or `model_path` values.

## Model List

We provide models quantized with our advanced quantization techniques. You can check whether a model is available in our [configuration directory](mblt_model_zoo/vision/models/).

## Optional Extras

When working with tasks other than vision, extra dependencies may be required. Those options can be installed via `pip install mblt-model-zoo[NAME]` or `pip install -e .[NAME]`.

Currently, these optional functions are only available on environment equipped with Mobilint's [ARIES](https://www.mobilint.com/aries).

|Name|Use|Details|
|-------|------|------|
|onnxruntime|For running vision models with `framework="onnx"` on CPU|Install with `pip install mblt-model-zoo[onnxruntime]`|
|onnxruntime-gpu|For running vision models with `framework="onnx"` with GPU-enabled ONNX Runtime|Install with `pip install mblt-model-zoo[onnxruntime-gpu]`|
|transformers|For using HuggingFace transformers related models|[README.md](mblt_model_zoo/hf_transformers/README.md)|
|MeloTTS|For using MeloTTS models|[README.md](mblt_model_zoo/MeloTTS/README.md)|
|qbcompiler|For generating mxq files with custom setting|[README.md](compile/README.md)|

The `qbcompiler` extra is strictly isolated from ordinary package use. qbcompiler is loaded only
when `compile_vision_model()` or `mblt-model-zoo compile` actually starts compilation. The base
package, vision APIs, compilation module import, and non-compile CLI commands continue to work
without qbcompiler installed; only a compilation request reports the installation error.

For the `transformers` extra, the repository also includes:

- functional test instructions in [tests/transformers/TEST.md](tests/transformers/TEST.md)
- benchmark script usage in [benchmark/transformers/README.md](benchmark/transformers/README.md)

> Note: The `MeloTTS` extra includes `unidic`, which requires an additional dictionary download step. Python packaging (PEP 517/518) does not support running arbitrary post-install commands automatically, so run `mblt-unidic-download` (or `python -m unidic download`) after installing the extra when needed.

## Command Line Interface

Installing this package exposes the `mblt-model-zoo` console command:

```bash
mblt-model-zoo --help
```

The CLI provides Mobilint-specific helper commands and delegates selected upstream Hugging Face
Transformers commands to the installed `transformers` package.

Compile a configured vision ONNX model with the optional compiler dependency:

```bash
mblt-model-zoo compile --model-cls alexnet
```

The matching Python API is `mblt_model_zoo.compile.vision.compile_vision_model`. See the
[vision compilation guide](compile/vision/README.md) for calibration datasets and options.
Invoking this API or command is the only point where qbcompiler is imported.

When paths are omitted, compilation stores downloaded ONNX models and compiled MXQ outputs under
`~/.mblt_model_zoo`, and uses registry datasets under `~/.mblt_model_zoo/datasets`. It does not
derive these defaults from the current checkout or working directory.

Compilation accepts one of three data entry levels: `--data-path` for a full organized image
dataset, `--subset-path` for already-sampled images, or `--calib-data-path` for ready preprocessed
`.npy` tensors. Later-stage input skips all earlier preparation stages.

### Vision Prediction And Validation

The vision CLI runs the same preprocess, NPU inference, postprocess, and plotting pipeline used by
the Python API. Use `predict` with a source image and a model name; the task is inferred from the
model configuration. `classify`, `detect`, `pose`, and `segment` are also accepted as aliases.

```bash
mblt-model-zoo predict --source ./cat.png --model resnet50
mblt-model-zoo predict --source ./street.jpg --model yolo11m --output ./result_detect.jpg
```

Vision commands accept a shared `--model-path` for local MXQ and local ONNX files. When
`--framework` is omitted, the CLI infers `.mxq` and `.onnx` suffixes and otherwise falls back to
MXQ. If the explicit framework conflicts with the local file suffix, the command fails with a
clear error. The compatibility flags `--mxq-path` and `--onnx-path` stay separate from
`--model-path`, so framework-specific resolution still works when both a local MXQ artifact and an
explicit ONNX runtime are involved.

```bash
mblt-model-zoo predict --source ./cat.png --model resnet50 --model-path ./resnet50.mxq
mblt-model-zoo predict --source ./cat.png --model resnet50 --model-path ./resnet50.onnx
mblt-model-zoo predict --source ./cat.png --model resnet50 --framework onnx
mblt-model-zoo predict --source ./cat.png --model resnet50 --framework onnx --mxq-path ./resnet50.mxq
mblt-model-zoo predict --source ./cat.png --model resnet50 --framework onnx --onnx-path ./resnet50.onnx
```

Prediction results are saved under `runs/vision/predict/` by default. Pass `--output` or
`--save-path` to choose a specific output file. Classification models accept `--topk`; object
detection, instance segmentation, and pose estimation models accept `--conf-thres` and
`--iou-thres`.

```bash
mblt-model-zoo predict --source ./cat.png --model resnet50 --topk 5
mblt-model-zoo predict --source ./street.jpg --model yolo11m --conf-thres 0.5 --iou-thres 0.5
```

Use `val` to validate a supported vision model on its benchmark dataset. Classification models use
ImageNet, while object detection, instance segmentation, and pose estimation models use COCO.
Validation also supports `--framework onnx`, the shared `--model-path` override, and the
framework-specific compatibility aliases.

```bash
mblt-model-zoo val --model resnet50
mblt-model-zoo val --model yolo11m --batch-size 8 --conf-thres 0.001 --iou-thres 0.7
mblt-model-zoo val --model resnet50 --model-path ./resnet50.mxq
mblt-model-zoo val --model resnet50 --model-path ./resnet50.onnx
mblt-model-zoo val --model resnet50 --framework onnx
mblt-model-zoo val --model resnet50 --framework onnx --mxq-path ./resnet50.mxq
mblt-model-zoo val --model resnet50 --framework onnx --onnx-path ./resnet50.onnx
```

Common NPU and artifact options are shared by the vision commands:

```bash
mblt-model-zoo predict \
  --source ./cat.png \
  --model resnet50 \
  --model-type DEFAULT \
  --model-path /path/to/model.mxq \
  --core-mode global8 \
  --dev-no 0
```

Use `--core-mode single`, `multi`, `global4`, or `global8` to select the NPU execution mode. For
manual placement, pass semicolon-separated values with `--target-cores`, such as `0:0;0:1`, or
`--target-clusters`, such as `0;1`. Full vision CLI details and supported model names are available
in [mblt_model_zoo/vision/README.md](mblt_model_zoo/vision/README.md).

### TPS Benchmark Helpers

The `tps` command measures token-per-second performance for Transformers-based text-generation and
image-text-to-text pipelines. It requires the `transformers` extra.

```bash
pip install "mblt-model-zoo[transformers]"
mblt-model-zoo tps measure --help
mblt-model-zoo tps sweep --help
```

Detailed TPS benchmark examples are available in
[benchmark/transformers/README.md](benchmark/transformers/README.md).

### MeloTTS Helpers

The `melo` command, also available as `melotts`, forwards arguments to the MeloTTS Click CLI. The
`melo-ui` command launches the MeloTTS Gradio WebUI. These commands require the `MeloTTS` extra.

```bash
pip install "mblt-model-zoo[MeloTTS]"
mblt-model-zoo melo --help
mblt-model-zoo melotts --help
mblt-model-zoo melo-ui --help
```

### Delegated Transformers Commands

When the first argument is one of `add-fast-image-processor`, `add-new-model-like`, `chat`,
`convert`, `download`, `env`, `run`, `serve`, or `version`, `mblt-model-zoo` delegates execution to
the installed Transformers CLI. For `chat` and `serve`, the CLI installs Mobilint model registration
hooks when the delegated Transformers backend loads models through the local serve command path.

## Verbose Option

By default, model initialization stays quiet. To print the model file size and MD5 hash whenever an MXQ model loads, set the environment variable `MBLT_MODEL_ZOO_VERBOSE` to a truthy value before running your script:

```bash
export MBLT_MODEL_ZOO_VERBOSE=true  # accepted values: true/1/yes/on (case-insensitive)
python your_script.py
```

### Example Verbose Output

```bash
Model Initialized
Model Size: 216.94 MB
Model Hash: 23c262c43b4c1c453dd0326e249480a0
Device Number: 0
Core Mode: single
Target Cores: [CoreId(cluster=Cluster.Cluster0, core=Core.Core0)]
Model Variant 0
        Input Shape: [(1, 200, 96), (1, 200, 96), (2, 200, 200)]
        Output Shape: [(1, 102400, 1)]
Model Variant 1
        Input Shape: [(1, 300, 96), (1, 300, 96), (2, 300, 300)]
        Output Shape: [(1, 153600, 1)]
Model Variant 2
        Input Shape: [(1, 400, 96), (1, 400, 96), (2, 400, 400)]
        Output Shape: [(1, 204800, 1)]
Model Variant 3
        Input Shape: [(1, 500, 96), (1, 500, 96), (2, 500, 500)]
        Output Shape: [(1, 256000, 1)]
Model Variant 4
        Input Shape: [(1, 600, 96), (1, 600, 96), (2, 600, 600)]
        Output Shape: [(1, 307200, 1)]
Model Variant 5
        Input Shape: [(1, 900, 96), (1, 900, 96), (2, 900, 900)]
        Output Shape: [(1, 460800, 1)]
```

Unset or set the variable to any other value to suppress these messages.

## License

The Mobilint Model Zoo is released under BSD 3-Clause License. Please see the [LICENSE](https://github.com/mobilint/mblt-model-zoo/blob/master/LICENSE) file for more details.

Additionally, the license for each model provided in this package follows the terms specified in the source link provided with it.

## Support & Issues

If you encounter any problems with this package, please feel free to contact [us](mailto:tech-support@mobilint.com).
