Metadata-Version: 2.4
Name: gigachat-audio-mlx
Version: 0.1.2
Summary: Native GigaChat Audio inference on Apple Silicon with MLX
Project-URL: Homepage, https://github.com/ai-babai/gigachat-audio-mlx
Project-URL: Repository, https://github.com/ai-babai/gigachat-audio-mlx.git
Project-URL: Issues, https://github.com/ai-babai/gigachat-audio-mlx/issues
Project-URL: Changelog, https://github.com/ai-babai/gigachat-audio-mlx/blob/main/CHANGELOG.md
Project-URL: Upstream Model, https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B
Project-URL: MLX BF16 Model, https://huggingface.co/ai-babai/gigachat-audio-mlx
Project-URL: Model Collection, https://huggingface.co/collections/ai-babai/gigachat-audio-mlx-6a5cab53ded991903b664437
Author: Maksim Popkov
License-Expression: MIT
License-File: LICENSE
License-File: THIRD_PARTY_NOTICES.md
Keywords: apple-silicon,audio-language-model,audio-understanding,gigachat,mlx,temporal-grounding
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Multimedia :: Sound/Audio :: Analysis
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: <3.14,>=3.12
Requires-Dist: huggingface-hub<2,>=1.3
Requires-Dist: mlx-lm<0.32,>=0.31.3
Requires-Dist: mlx<0.33,>=0.32
Requires-Dist: numpy<3,>=2.3
Requires-Dist: psutil<8,>=7
Requires-Dist: soundfile<0.15,>=0.14
Requires-Dist: soxr<2,>=1.0
Requires-Dist: transformers<6,>=5.14
Provides-Extra: dev
Requires-Dist: build<2,>=1.3; extra == 'dev'
Requires-Dist: pytest<10,>=9; extra == 'dev'
Requires-Dist: ruff<1,>=0.12; extra == 'dev'
Description-Content-Type: text/markdown

# GigaChat Audio MLX

**English** · [Русский](README.ru.md)

Local, offline audio understanding on Apple Silicon Macs. Ask questions about
audio, transcribe or translate speech, detect emotion, find moments by time, and
create timestamped summaries. This is an independent native
[MLX](https://github.com/ml-explore/mlx) port of the official
[GigaChat3.1-Audio-10B-A1.8B](https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B)
model from the
[GigaChat Audio paper](https://arxiv.org/abs/2607.10387). It runs without a
PyTorch, CUDA, or cloud runtime.

[PyPI](https://pypi.org/project/gigachat-audio-mlx/) ·
[Hugging Face models](https://huggingface.co/collections/ai-babai/gigachat-audio-mlx-6a5cab53ded991903b664437) ·
[Latest release](https://github.com/ai-babai/gigachat-audio-mlx/releases/latest) ·
[Full results](docs/release/v0.1.0/RESULTS.md) ·
[Upstream model](https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B) ·
[Original paper](https://arxiv.org/abs/2607.10387)

## At a glance: what each model needs

| Variant | Model size | Free disk to set aside | Measured total process memory | 5:17 audio on M4 Pro | Recommendation |
|---|---:|---:|---:|---:|---|
| **BF16** | 22.54 GB | about 25 GB | 22.84 GiB | 10.41 s · 30.5× realtime | **default for quality** |
| Q8/BF16 | 13.75 GB | about 16 GB | 14.80 GiB | 6.90 s · 46.0× realtime | experimental opt-in |

In plain terms: on the tested Mac, a 5-minute 17-second recording took about
10 seconds with BF16 or 7 seconds with Q8/BF16. Download time is not included.
The free-disk figures are practical rounded reserves; `download --dry-run`
reports the exact requirement before transferring anything.

The memory column is measured use by the whole process, not a promise that a
Mac with exactly that much unified memory will work. Both variants are verified
only on a 48 GB Mac today. 16/24/32 GB support will be stated only after it is
tested.

## Quick start

The verified environment is an Apple M4 Pro with 48 GB unified memory,
macOS 15.7.7, and native ARM Python 3.12 or 3.13. Broader M1–M5 and lower-memory
support is not yet claimed.

```bash
brew install uv
uv tool install gigachat-audio-mlx

gigachat-audio-mlx doctor
gigachat-audio-mlx download --variant bf16 --dry-run
gigachat-audio-mlx download --variant bf16 --yes

gigachat-audio-mlx run meeting.m4a \
  --variant bf16 \
  --task summarize \
  --max-tokens 768
```

`doctor` is read-only. The download dry run shows exact bytes, cache hits, free
disk space, and the destination before any network transfer.

## Why this port?

- Native MLX inference without PyTorch, CUDA, ONNX Runtime, Core ML, or a cloud
  API.
- One local runtime for audio QA, transcription, translation, emotion analysis,
  temporal grounding, and timestamped summaries.
- Data-only model repositories: no remote Python execution and no
  `trust_remote_code`.
- Immutable model revisions, a signed manifest of expected files, and SHA-256
  validation before loading.
- A quality-first BF16 model and a clearly labelled, optional Q8/BF16 variant.

## Model variants and quality trade-off

All resource figures below are direct measurements on the same 317.27-second
audio workload. Lower is better. The median of five clean processes is reported
for time and whole-process physical memory.

| Variant | Recommended use | Model size | Process memory | 5-minute time | Hugging Face |
|---|---|---:|---:|---:|---|
| **BF16** | **recommended quality-first default** | 22.54 GB | 22.84 GiB | 10.41 s | [`ai-babai/gigachat-audio-mlx`](https://huggingface.co/ai-babai/gigachat-audio-mlx) |
| Q8/BF16 | experimental opt-in | **13.75 GB** | **14.80 GiB** | **6.90 s** | [`ai-babai/...-q8-bf16`](https://huggingface.co/ai-babai/gigachat-audio-mlx-q8-bf16) |

Q8/BF16 is 39.0% smaller, used 35.2% less process memory, and was 33.7%
faster. It also produced a factual error in 29 of 64 reviewed responses versus
26 of 64 for BF16. BF16 therefore remains the default.

## Common tasks

The text instruction is part of every input. Use a preset or provide an exact
custom request:

```bash
# Russian transcription
gigachat-audio-mlx run speech.wav --variant bf16 --task transcribe

# Translate speech to English
gigachat-audio-mlx run interview.flac --variant bf16 --task translate-en

# Find a moment in time
gigachat-audio-mlx run meeting.m4a \
  --variant bf16 \
  --prompt "Find when the budget decision was approved. Return start and end timestamps."

# Timestamped summary
gigachat-audio-mlx run meeting.m4a \
  --variant bf16 \
  --task summarize \
  --max-tokens 768 \
  --stats
```

Available presets are `transcribe`, `translate`, `translate-ru`,
`translate-en`, `emotion`, `ground`, `summarize`, and `qa`. A custom `--prompt`
overrides the preset.

WAV and FLAC work in the default installation. MP3 is supported by the bundled
audio wheel; M4A uses the macOS-native decoder fallback. Stereo is mixed to
mono and resampled deterministically to 16 kHz with SoXR HQ. No Homebrew or
ffmpeg installation is required.

## Results available today

### MLX BF16 compared with MLX Q8/BF16

| Metric, lower is better | MLX BF16 · recommended | MLX Q8/BF16 · experimental | Change |
|---|---:|---:|---:|
| Model size, decimal GB | 22.54 | 13.75 | **−39.0%** |
| Whole-process memory, 5-minute profile | 22.84 GiB | 14.80 GiB | **−35.2%** |
| End-to-end time, 5-minute profile | 10.41 s | 6.90 s | **−33.7%** |
| Responses with a factual error | 26/64 (40.63%) | 29/64 (45.31%) | **+3 responses (+4.69 pp)** |

Model size is the disk payload, not RAM. Memory is the whole-process macOS physical
footprint. Published performance figures come from the verified M4 Pro / 48 GB
machine and are not universal Apple Silicon claims.

### Original PyTorch BF16 compared with the MLX port

| Check | Original PyTorch BF16 | MLX BF16 | MLX Q8/BF16 |
|---|---:|---:|---:|
| Model size, decimal GB | 23.73 | 22.54 | 13.75 |
| Exact decoded-text agreement | Reference | 41/50 vs Original | 45/50 vs MLX BF16 |

Exact text agreement is an implementation-parity diagnostic, not an aggregate
quality score. Original PyTorch has not yet been measured with the same
five-minute Mac resource protocol, so it is intentionally absent from the RAM
and speed comparison. That measurement is planned after publication.

### Results reported by the original authors

These values come from the upstream model card and were not rerun by this
project.

| Task and dataset | Metric | Original reported result |
|---|---:|---:|
| Audio QA · MMAU | Accuracy ↑ | 62.2 |
| Audio QA (Russian) · RuBQ | Accuracy ↑ | 60.0 |
| Emotion · Dusha podcast | Accuracy ↑ | 92.4 |
| ASR (Russian) · FLEURS ru | WER ↓ | 4.4 |
| ASR (English) · FLEURS en | WER ↓ | 6.5 |
| Translation · FLEURS ru→en | BLEU ↑ | 33.4 |
| Translation · FLEURS en→ru | BLEU ↑ | 26.0 |
| Temporal localization · 20–60 min | mIoU ↑ | 48.3 |
| Timed summary · 20–60 min | Overall ↑ | 67.9 |

See the [full result tables and measurement notes](docs/release/v0.1.0/RESULTS.md)
and the [upstream model card](https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B).

## Python API

```python
from gigachat_audio_mlx.runtime import GigaChatAudioRuntime

runtime = GigaChatAudioRuntime.load("/path/to/mlx-model")
```

Pinned Hugging Face snapshots can also be loaded without remote code:

```python
runtime = GigaChatAudioRuntime.from_pretrained(
    "ai-babai/gigachat-audio-mlx",
    revision="db430193908ea810d4873404b43a148d5cbe9753",
    local_files_only=True,
)
```

## Compatibility and limitations

- Verified: Apple M4 Pro, 48 GB unified memory, macOS 15.7.7 arm64, and Python
  3.12/3.13.
- Not yet claimed: all M1–M5 Macs, 16/24/32 GB configurations, Intel Mac,
  Linux, Windows, iOS, or 90-minute inputs.
- Long timestamped summaries can reach the output-token limit. Increase
  `--max-tokens` only after checking the memory estimate.
- Answers, translations, emotions, facts, and timestamps can be wrong. Verify
  important output against the source audio.
- Do not use output as the sole basis for medical, legal, employment, safety,
  or surveillance decisions. Users remain responsible for consent, privacy,
  copyright, and the terms governing their audio.

## Reproducibility and development

The source model is pinned to revision
`bf73d03a43bdf5118f5a4dbdc24ba6f56ac31cfb`. Model weights, datasets, caches,
private audio, and large raw benchmark outputs are not stored in Git. The
repository contains compact results, manifests, selection rules, and commands
needed to inspect the published evidence.

Start with [`benchmarks/README.md`](benchmarks/README.md) for the benchmark map
and [`docs/release/v0.1.0/RESULTS.md`](docs/release/v0.1.0/RESULTS.md) for the
current tables and limitations.

## License and attribution

The MLX runtime is MIT-licensed. Upstream model, paper, code, and dependency
provenance is recorded in [`THIRD_PARTY_NOTICES.md`](THIRD_PARTY_NOTICES.md).
Please cite both this software through [`CITATION.cff`](CITATION.cff) and the
original work:

- [GigaChat3.1-Audio-10B-A1.8B model](https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B)
- [GigaChat Audio: Time-aware Large Audio Language Model](https://arxiv.org/abs/2607.10387),
  by Aleksandr Kutsakov, Mariia Sadovina, Georgii Gospodinov, Alexandr
  Maximenko, Oleg Kutuzov, Pavel Bogomolov, and Fyodor Minkin
- [Upstream MIT license declaration](https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B)

This project is not an official release by the GigaChat Audio authors. Security
reports and contributions are covered by [`SECURITY.md`](SECURITY.md) and
[`CONTRIBUTING.md`](CONTRIBUTING.md).
