Metadata-Version: 2.4
Name: gigachat-audio-mlx
Version: 0.1.4
Summary: Native GigaChat Audio inference on Apple Silicon with MLX
Project-URL: Homepage, https://github.com/ai-babai/gigachat-audio-mlx
Project-URL: Repository, https://github.com/ai-babai/gigachat-audio-mlx.git
Project-URL: Issues, https://github.com/ai-babai/gigachat-audio-mlx/issues
Project-URL: Changelog, https://github.com/ai-babai/gigachat-audio-mlx/blob/main/CHANGELOG.md
Project-URL: Upstream Model, https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B
Project-URL: MLX BF16 Model, https://huggingface.co/ai-babai/gigachat-audio-mlx
Project-URL: Model Collection, https://huggingface.co/collections/ai-babai/gigachat-audio-mlx-6a5cab53ded991903b664437
Author: Maksim Popkov
License-Expression: MIT
License-File: LICENSE
License-File: THIRD_PARTY_NOTICES.md
Keywords: apple-silicon,audio-language-model,audio-understanding,gigachat,mlx,temporal-grounding
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Multimedia :: Sound/Audio :: Analysis
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: <3.14,>=3.12
Requires-Dist: huggingface-hub<2,>=1.3
Requires-Dist: mlx-lm<0.32,>=0.31.3
Requires-Dist: mlx<0.33,>=0.32
Requires-Dist: numpy<3,>=2.3
Requires-Dist: psutil<8,>=7
Requires-Dist: soundfile<0.15,>=0.14
Requires-Dist: soxr<2,>=1.0
Requires-Dist: transformers<6,>=5.14
Provides-Extra: dev
Requires-Dist: build<2,>=1.3; extra == 'dev'
Requires-Dist: pytest<10,>=9; extra == 'dev'
Requires-Dist: ruff<1,>=0.12; extra == 'dev'
Description-Content-Type: text/markdown

# GigaChat Audio MLX

**English** · [Русский](README.ru.md)

Local, offline audio understanding on Apple Silicon Macs. Ask questions about
audio, transcribe or translate speech, detect emotion, find moments by time, and
create timestamped summaries. This is an independent native
[MLX](https://github.com/ml-explore/mlx) port of the official
[GigaChat3.1-Audio-10B-A1.8B](https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B)
model from the
[GigaChat Audio paper](https://arxiv.org/abs/2607.10387). It runs without a
PyTorch, CUDA, or cloud runtime.

[PyPI](https://pypi.org/project/gigachat-audio-mlx/) ·
[Hugging Face models](https://huggingface.co/collections/ai-babai/gigachat-audio-mlx-6a5cab53ded991903b664437) ·
[Latest release](https://github.com/ai-babai/gigachat-audio-mlx/releases/latest) ·
[Full results](docs/release/v0.1.0/RESULTS.md) ·
[Upstream model](https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B) ·
[Original paper](https://arxiv.org/abs/2607.10387)

## At a glance: what each model needs

| Variant | Model size | Free disk to set aside | Measured total process memory | 5:17 audio on M4 Pro | Recommendation |
|---|---:|---:|---:|---:|---|
| **BF16** | 22.54 GB | about 25 GB | 22.84 GiB | 10.41 s · 30.5× realtime | **quality-first** |
| **Q8/BF16** | **13.75 GB** | **about 16 GB** | **14.80 GiB** | **6.90 s · 46.0× realtime** | **smaller and faster** |

In plain terms: on the tested Mac, a 5-minute 17-second recording took about
10 seconds with BF16 or 7 seconds with Q8/BF16. Download time is not included.
The free-disk figures are practical rounded reserves; `download --dry-run`
reports the exact requirement before transferring anything.

The memory column is measured use by the whole process, not a promise that a
Mac with exactly that much unified memory will work. Both variants are verified
only on a 48 GB Mac today. 16/24/32 GB support will be stated only after it is
tested.

## Why MLX matters on this Mac

Same Apple M4 Pro with 48 GB unified memory. The Original row is a safety stop,
not a completed speed benchmark; the MLX rows are medians of five clean runs.

| Runtime | Audio | Outcome | Whole-process footprint |
|---|---:|---|---:|
| Original PyTorch BF16 on MPS | 9.84 s | **No answer after 277 s; stopped** | **at least 51.92 GiB** |
| MLX BF16 | 317.27 s | Completed in **10.41 s** | 22.84 GiB |
| MLX Q8/BF16 | 317.27 s | Completed in **6.90 s** | 14.80 GiB |

The pinned upstream Transformers implementation is demonstrated for CUDA. On
MPS it exceeded the 48 GB Mac's safe memory budget and nearly filled system
swap even on the short smoke. We stopped before the 5-minute run, so no
Original-to-MLX speedup is claimed. The practical conclusion is simpler: MLX
makes this model usable on the tested Mac.

## Quick start

The verified environment is an Apple M4 Pro with 48 GB unified memory,
macOS 15.7.7, and native ARM Python 3.12 or 3.13. Broader M1–M5 and lower-memory
support is not yet claimed.

```bash
brew install uv
uv tool install gigachat-audio-mlx

gigachat-audio-mlx doctor
gigachat-audio-mlx download --variant bf16 --dry-run
gigachat-audio-mlx download --variant bf16 --yes

gigachat-audio-mlx run meeting.m4a \
  --variant bf16 \
  --task summarize \
  --max-tokens 768
```

`doctor` is read-only. The download dry run shows exact bytes, cache hits, free
disk space, and the destination before any network transfer.

## Why this port?

- Native MLX inference without PyTorch, CUDA, ONNX Runtime, Core ML, or a cloud
  API.
- One local runtime for audio QA, transcription, translation, emotion analysis,
  temporal grounding, and timestamped summaries.
- Data-only model repositories: no remote Python execution and no
  `trust_remote_code`.
- Immutable model revisions, a signed manifest of expected files, and SHA-256
  validation before loading.
- A quality-first BF16 model and a compact Q8/BF16 option for lower disk,
  memory, and processing time.

## Choose between BF16 and Q8/BF16

All resource figures below are direct measurements on the same 317.27-second
audio workload. Lower is better. The median of five clean processes is reported
for time and whole-process physical memory.

| Variant | Recommended use | Model size | Process memory | 5-minute time | Hugging Face |
|---|---|---:|---:|---:|---|
| **BF16** | **best measured answer quality** | 22.54 GB | 22.84 GiB | 10.41 s | [`ai-babai/gigachat-audio-mlx`](https://huggingface.co/ai-babai/gigachat-audio-mlx) |
| **Q8/BF16** | **less disk and memory, higher speed** | **13.75 GB** | **14.80 GiB** | **6.90 s** | [`ai-babai/...-q8-bf16`](https://huggingface.co/ai-babai/gigachat-audio-mlx-q8-bf16) |

Q8/BF16 is 39.0% smaller, used 35.2% less process memory, and needed 33.7%
less processing time. In the blinded review, it had factual errors in 3 more
cases than BF16.
Choose Q8/BF16 when size, memory, and speed matter; choose BF16 for the best
measured answer quality. The exact review counts are shown below.

## Common tasks

The text instruction is part of every input. Use a preset or provide an exact
custom request:

```bash
# Russian transcription
gigachat-audio-mlx run speech.wav --variant bf16 --task transcribe

# Translate speech to English
gigachat-audio-mlx run interview.flac --variant bf16 --task translate-en

# Find a moment in time
gigachat-audio-mlx run meeting.m4a \
  --variant bf16 \
  --prompt "Find when the budget decision was approved. Return start and end timestamps."

# Timestamped summary
gigachat-audio-mlx run meeting.m4a \
  --variant bf16 \
  --task summarize \
  --max-tokens 768 \
  --stats
```

Available presets are `transcribe`, `translate`, `translate-ru`,
`translate-en`, `emotion`, `ground`, `summarize`, and `qa`. A custom `--prompt`
overrides the preset.

WAV and FLAC work in the default installation. MP3 is supported by the bundled
audio wheel; M4A uses the macOS-native decoder fallback. Stereo is mixed to
mono and resampled deterministically to 16 kHz with SoXR HQ. No Homebrew or
ffmpeg installation is required.

## Results available today

### MLX BF16 compared with MLX Q8/BF16

| Metric, lower is better | MLX BF16 | MLX Q8/BF16 | Change |
|---|---:|---:|---:|
| Model size, decimal GB | 22.54 | 13.75 | **−39.0%** |
| Whole-process memory, 5-minute profile | 22.84 GiB | 14.80 GiB | **−35.2%** |
| End-to-end time, 5-minute profile | 10.41 s | 6.90 s | **−33.7%** |
| Responses with a factual error | 26/64 (40.63%) | 29/64 (45.31%) | **+3 responses (+4.69 pp)** |

Model size is the disk payload, not RAM. Memory is the whole-process macOS physical
footprint. Published performance figures come from the verified M4 Pro / 48 GB
machine and are not universal Apple Silicon claims.

### Original PyTorch BF16 compared with the MLX port

| Check | Original PyTorch BF16 | MLX BF16 | MLX Q8/BF16 |
|---|---:|---:|---:|
| Model size, decimal GB | 23.73 | 22.54 | 13.75 |
| Exact decoded-text agreement | Reference | 41/50 vs Original | 45/50 vs MLX BF16 |

Exact text agreement is an implementation-parity diagnostic, not an aggregate
quality score.

The post-publication Original resource check did not produce a valid latency
number on this Mac. The pinned official PyTorch/MPS path had not completed a
9.84-second smoke after 277 seconds; its process footprint had reached at least
51.92 GiB and system swap was nearly full. We stopped it before starting the
317.27-second profile. Therefore Original remains `N/A` for same-Mac time and
memory rather than being compared with a failed or swap-dominated run.

### Results reported by the original authors

These values come from the upstream model card and were not rerun by this
project.

| Task and dataset | Metric | Original reported result |
|---|---:|---:|
| Audio QA · MMAU | Accuracy ↑ | 62.2 |
| Audio QA (Russian) · RuBQ | Accuracy ↑ | 60.0 |
| Emotion · Dusha podcast | Accuracy ↑ | 92.4 |
| ASR (Russian) · FLEURS ru | WER ↓ | 4.4 |
| ASR (English) · FLEURS en | WER ↓ | 6.5 |
| Translation · FLEURS ru→en | BLEU ↑ | 33.4 |
| Translation · FLEURS en→ru | BLEU ↑ | 26.0 |
| Temporal localization · 20–60 min | mIoU ↑ | 48.3 |
| Timed summary · 20–60 min | Overall ↑ | 67.9 |

See the [full result tables and measurement notes](docs/release/v0.1.0/RESULTS.md)
and the [upstream model card](https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B).

## Python API

```python
from gigachat_audio_mlx.runtime import GigaChatAudioRuntime

runtime = GigaChatAudioRuntime.load("/path/to/mlx-model")
```

Pinned Hugging Face snapshots can also be loaded without remote code:

```python
runtime = GigaChatAudioRuntime.from_pretrained(
    "ai-babai/gigachat-audio-mlx",
    revision="db430193908ea810d4873404b43a148d5cbe9753",
    local_files_only=True,
)
```

## Compatibility and limitations

- Verified: Apple M4 Pro, 48 GB unified memory, macOS 15.7.7 arm64, and Python
  3.12/3.13.
- Not yet claimed: all M1–M5 Macs, 16/24/32 GB configurations, Intel Mac,
  Linux, Windows, iOS, or 90-minute inputs.
- Long timestamped summaries can reach the output-token limit. Increase
  `--max-tokens` only after checking the memory estimate.
- Answers, translations, emotions, facts, and timestamps can be wrong. Verify
  important output against the source audio.
- Do not use output as the sole basis for medical, legal, employment, safety,
  or surveillance decisions. Users remain responsible for consent, privacy,
  copyright, and the terms governing their audio.

## Reproducibility and development

The source model is pinned to revision
`bf73d03a43bdf5118f5a4dbdc24ba6f56ac31cfb`. Model weights, datasets, caches,
private audio, and large raw benchmark outputs are not stored in Git. The
repository contains compact results, manifests, selection rules, and commands
needed to inspect the published evidence.

Start with [`benchmarks/README.md`](benchmarks/README.md) for the benchmark map
and [`docs/release/v0.1.0/RESULTS.md`](docs/release/v0.1.0/RESULTS.md) for the
current tables and limitations.

## License and attribution

The MLX runtime is MIT-licensed. Upstream model, paper, code, and dependency
provenance is recorded in [`THIRD_PARTY_NOTICES.md`](THIRD_PARTY_NOTICES.md).
Please cite both this software through [`CITATION.cff`](CITATION.cff) and the
original work:

- [GigaChat3.1-Audio-10B-A1.8B model](https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B)
- [GigaChat Audio: Time-aware Large Audio Language Model](https://arxiv.org/abs/2607.10387),
  by Aleksandr Kutsakov, Mariia Sadovina, Georgii Gospodinov, Alexandr
  Maximenko, Oleg Kutuzov, Pavel Bogomolov, and Fyodor Minkin
- [Upstream MIT license declaration](https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B)

This project is not an official release by the GigaChat Audio authors. Security
reports and contributions are covered by [`SECURITY.md`](SECURITY.md) and
[`CONTRIBUTING.md`](CONTRIBUTING.md).
