Metadata-Version: 2.4
Name: echoai-helper
Version: 1.4.5
Summary: Real-time meeting transcription and interview assistance, on-device
Author: colakang
License: MIT
Project-URL: Homepage, https://github.com/colakang/echoai_helper
Project-URL: Issues, https://github.com/colakang/echoai_helper/issues
Keywords: transcription,asr,meeting-notes,diarization,funasr
Requires-Python: <3.14,>=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: sounddevice; sys_platform != "win32"
Requires-Dist: PyAudioWPatch; sys_platform == "win32"
Requires-Dist: audioop-lts; python_version >= "3.13"
Requires-Dist: standard-aifc; python_version >= "3.13"
Requires-Dist: numpy<2
Requires-Dist: scipy
Requires-Dist: torch
Requires-Dist: torchaudio
Requires-Dist: funasr
Requires-Dist: modelscope
Requires-Dist: huggingface_hub
Requires-Dist: loguru
Requires-Dist: onnxruntime
Requires-Dist: soundfile
Requires-Dist: openai
Requires-Dist: customtkinter==5.1.3
Requires-Dist: python-dotenv
Requires-Dist: pyyaml
Requires-Dist: pytz
Provides-Extra: litellm
Requires-Dist: litellm; extra == "litellm"
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Dynamic: license-file

# 🎙️ EchoAI Helper

[![PyPI](https://img.shields.io/pypi/v/echoai-helper.svg)](https://pypi.org/project/echoai-helper/)
[![GitHub Stars](https://img.shields.io/github/stars/colakang/echoai_helper?style=social)](https://github.com/colakang/echoai_helper/stargazers)
[![License](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
[![Python](https://img.shields.io/badge/python-3.12-blue.svg)](https://python.org)
[![macOS](https://img.shields.io/badge/macOS-13%2B-lightgrey.svg)](#-install)
[![Windows](https://img.shields.io/badge/Windows-10%2B-lightgrey.svg)](#windows)

Real-time meeting transcription and interview assistance, running on your own
machine. It records both sides of a conversation — your microphone and whatever
the meeting app is playing — transcribes them as they happen, labels who is
speaking, and exports readable notes.

Speech recognition is local. Audio never leaves the machine unless you ask a
language model to clean up the finished transcript.

<p>
<a href="https://www.producthunt.com/posts/echoai-interview-copilot?embed=true&utm_source=badge-featured&utm_medium=badge&utm_souce=badge-echoai&#0045;interview&#0045;copilot" target="_blank"><img src="https://api.producthunt.com/widgets/embed-image/v1/featured.svg?post_id=601490&theme=light" alt="EchoAI&#0032;Interview&#0032;Copilot&#0032; - Real&#0045;time&#0032;conversation&#0032;with&#0032;LLM&#0032;responses | Product Hunt" style="width: 250px; height: 54px;" width="250" height="54" /></a>
</p>

<p align="center">
<img width="800" alt="EchoAI Helper Interface" src="https://github.com/colakang/echoai_helper/raw/main/docs/images/ui.png">
</p>

---

## ✨ What it does

- **Local speech recognition** — FunASR / SenseVoice, on-device, GPU-accelerated
  where one is available (Metal on Apple Silicon, CUDA on Windows)
- **Automatic language detection** — Mandarin, Cantonese, English, Japanese and
  Korean, switching per utterance, including mid-sentence code-switching
- **Both sides of the call** — your microphone and the far end, on separate tracks
- **Speaker labelling** — voices on the far-end track are told apart as the
  meeting runs, and re-grouped properly at export once you give it the real
  headcount
- **Pause-based segmentation** — sentences are cut where people actually pause,
  not on a fixed timer, so the model sees whole utterances
- **Crash-safe recording** — every settled sentence is written to disk as it is
  produced; a crash costs the last line, not the meeting
- **LLM cleanup on export** — an optional pass that fixes mis-heard words using
  the surrounding conversation, through an API key or through a coding-agent CLI
  on a subscription you already pay for
- **Markdown and JSON export** — Markdown to read, JSON as a complete record
- **Noise kept out of the transcript** — room noise that reaches the model comes
  back as a bare "."; 169 of 1312 lines in one real meeting, now dropped
- **Pause your own track** — muting yourself in the meeting app does not reach
  this one, and pausing roughly halves the model's work
- **Recovers a lost microphone** — a Bluetooth headset that drops out is
  detected and capture is rebuilt, rather than silently recording nothing
- **Live reply suggestions** — for interviews, where a prompt is wanted while the
  other person is still talking

## 💡 Two modes

| | Meeting notes | Live interview |
|---|---|---|
| Text appears | at each pause | as you speak (~0.6s) |
| Model calls | one per sentence | roughly three times as many |
| Best for | an accurate record | a prompt you can act on |

Switch in the app; the several settings that differ move together.

## 🎬 Demo

https://github.com/user-attachments/assets/0d627e4a-960b-4628-8bbc-8d892f02cfd1

---

## ⚡ Install

### Before you start (macOS)

Two things a fresh Mac tends to be missing. Both are one command, and the first
is easy to mistake for something having gone wrong.

**Command Line Developer Tools.** macOS ships stubs for `git` and the
compilers; touching one pops a dialog saying the tools must be installed. It
can appear part-way through an install, which reads like a failure and is not.
Get it over with first:

```bash
xcode-select --install
```

If they are already there it says so and exits non-zero, which looks like a
failure and is not.

**uv**, which is how this is installed:

```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
```

It fetches its own Python, which matters: this needs 3.12 or 3.13, and the
Python on your Mac is probably 3.9 (Apple's) or 3.14 (Homebrew's).

<details>
<summary>Homebrew — optional, and what it changes</summary>

Not required. `echoai-helper setup` installs the virtual audio device either
way: with Homebrew if you have it, and otherwise by fetching the vendor's
package directly and verifying its checksum before anything runs.

Having it means the audio driver is tracked like anything else you installed,
so `brew uninstall --cask blackhole-2ch` removes it cleanly. Without it the
driver is still perfectly removable, just by hand.

```bash
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
```

</details>

### macOS

Then:

```bash
uv tool install echoai-helper     # uv fetches a suitable Python itself
echoai-helper setup               # audio routing — one password prompt
echoai-helper install-launcher    # adds an icon to Launchpad, and opens it
```

After that, launch from Launchpad or run `echoai-helper`.

**If the icon does nothing**, the app is reporting something it cannot show
you. It writes everything to `~/Library/Logs/EchoAI Helper.log`; the last few
lines say what stopped it:

```bash
tail -20 "$HOME/Library/Logs/EchoAI Helper.log"
```

**Apple Silicon.** Transcription runs on Metal and keeps up comfortably.
**Intel Macs install and run, but see [known limitations](#-known-limitations)** —
there is no Metal path, so they fall back to the CPU.

`setup` installs a virtual audio device and builds the Multi-Output that lets
you hear a meeting while it is being recorded. macOS asks for a password once,
because that installs an audio driver — nothing else needs a privilege, and
Audio MIDI Setup is not involved.

The app takes the audio output while it runs and gives it back silently when it
quits, including after a crash. `echoai-helper setup --restore` does it by hand;
`--status` shows what routing is in place.

<details>
<summary>Prefer Homebrew?</summary>

A formula is in [`packaging/`](packaging/echoai-helper.rb) for a tap. Homebrew
can declare the virtual audio device as a dependency, which removes the one step
that needs a password.
</details>

### Windows

```powershell
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"    # if you need uv
uv tool install echoai-helper
echoai-helper
```

No audio setup step: Windows exposes WASAPI loopback directly, so the far end of
a call is capturable without a virtual device. The macOS-only commands (`setup`,
`install-launcher`) report that there is nothing to do. See
[known limitations](#-known-limitations) — this path has not been re-tested since
the segmentation rework.

### What has actually been tested

Stated precisely, because "should work" and "has been run" are different
claims and the difference matters when you are deciding whether to install it.

| | Install | Live meeting | Notes |
|---|---|---|---|
| **M4 Mac mini, 16GB, macOS 26.3, Python 3.12** | ✅ | ✅ | The development machine. Every measurement in this README comes from it. |
| **Other Apple Silicon** (M1–M3) | — | — | Same wheels, same Metal path. Expected to work; not run. |
| **Intel Mac** | ✅ resolves | ❌ | Installs — `uv` falls back to torch 2.2.2 — but no Metal, so CPU only. See [known limitations](#-known-limitations). |
| **Windows 10/11** | ✅ resolves | ❌ | Dependencies resolve. The capture path has not been run since the segmentation rework. |

"Live meeting" means real calls, in Mandarin, Cantonese and English, with the
transcript exported afterwards — not a smoke test. The longest was 84 minutes
and 1311 lines.

The Mac mini has no built-in microphone, so the microphone track has only been
exercised through a Bluetooth headset and a wired input. Disconnecting and
reconnecting that headset mid-recording is tested, because that is how the
microphone was found to die silently in the first place.

If you run it somewhere not on this list, an issue saying so — working or
not — is genuinely useful.

### Upgrading

```bash
uv tool upgrade echoai-helper
```

Not `uv tool install` again — that is a no-op on something already installed
and says so, which is easy to read as "you are up to date" when you are not:

```
`echoai-helper` is already installed
```

`uv` caches the package index, so an upgrade run soon after a release can land
on the version *before* the one you came for — it reports `Updated … -> v1.4.3`
and looks entirely successful. Running it again gets there. To go straight to
the newest:

```bash
uv tool install --force --refresh echoai-helper
```

`--refresh` belongs to `install`. There is no `uv tool upgrade --refresh`; it
exits with an error.

Re-run `echoai-helper install-launcher` after upgrading if you use the
Launchpad icon. The bundle holds a generated script rather than a copy of the
program, so it keeps working across upgrades — but the icon and the launch
script itself are only rewritten when you run that again.

### First run

Speech models (~1.5GB) download on first launch. Nothing else is needed.

<details>
<summary>Why <code>uv</code> rather than <code>pip</code></summary>

Because this needs Python 3.12 or 3.13, and the Python you have is probably
neither. macOS ships 3.9; Homebrew currently installs 3.14. `uv` fetches a
suitable interpreter itself, and its builds include tkinter, so there is no
separate Tk step either.

**If you try `pip install echoai-helper` on the wrong version** you get:

```
ERROR: Could not find a version that satisfies the requirement echoai-helper
       (from versions: none)
ERROR: No matching distribution found for echoai-helper
```

which reads as "no such package". It means "not for this Python". Nothing is
wrong with the name.

The upper bound is onnxruntime, which publishes no macOS arm64 wheel for 3.14.
The lower one is the vendored `src/custom_speech_recognition`, which imports
`aifc` and `audioop` — removed from the standard library in 3.13, and supplied
by backports that install only when they are missing.

</details>

---

## 🎯 Using it

1. Open the app. If audio routing is not in place, it offers to finish it.
2. Pick a mode, and set the number of people if you know it.
3. Hold your meeting. Nothing needs touching — though **Pause Mic** is there for
   the stretches you are muted anyway, and it roughly halves the model's work.
4. **Export** — one dialog covers format, cleanup, which model to clean with,
   and merging over-split speakers.

Cleanup runs in the background with a progress bar and an estimate, and can be
stopped: whatever finished is kept.

### Choosing a cleanup backend

| | Cost | Speed (measured) |
|---|---|---|
| **API** (`conf.yaml`) | per token | 5–8s per batch of lines |
| **Claude CLI** | included in a subscription | 20–50s per batch |

Both are offered at export, and the dialog turns the per-batch figure into an
estimate for the transcript in hand. The live reply suggestions always use the
configured API — a CLI takes seconds per answer, which is too late to be useful
while someone is still talking.

### Past recordings

```bash
echoai-helper sessions              # list them
echoai-helper sessions --export 0   # export one again
echoai-helper sessions --delete 0
```

Re-exporting is the point: a different format, another pass of cleanup, a
different number of speakers, without re-recording anything. An unfinished
session from the last 12 hours is offered on the next launch.

---

## ⚙️ Configuration

Model settings live in `conf.yaml`; everything else is in the app. Installed
from a wheel the shipped copy sits inside the package, so make yourself an
editable one:

```bash
echoai-helper config     # writes conf.yaml where you can reach it, and prints the path
```

That copy overrides the defaults. You will not usually need it — both values
below already ship as `auto`.

```yaml
FunASR:
  model_name: "iic/SenseVoiceSmall"
  device: "auto"        # cuda, then mps, then cpu
  language: "auto"      # zh, en, yue, ja, ko

LLM:
  provider: "openai"    # openai | litellm | cli
```

Both `auto` values are load-bearing rather than lazy defaults:

- **`device: "auto"`** — measured on an M4, dual-track real-time factor is 2.04
  on cpu (falling behind twice over) against 0.35 on Metal. Landing on cpu by
  accident means transcription that cannot keep up. Naming a device explicitly is
  also wrong on every machine that does not have it, and this file travels.
- **`language: "auto"`** — pinning a language does not bias the model, it forces
  the syllables onto words of that language. A Cantonese call transcribed with
  `language: "en"` comes back as fluent nonsense.

An OpenAI key goes in `.env` (see `.env.example`), or in a file called `.llm`
holding nothing else. Both are gitignored.

---

## 🔍 Troubleshooting

```bash
echoai-helper check-audio           # what is being captured, and from where
echoai-helper setup --status        # what routing is in place
```

**Nothing from the far end (macOS).** The meeting app has its own audio settings
and remembers them. Set its speaker to `EchoAI Meeting`.

**Nothing from the microphone over Bluetooth.** A Bluetooth headset can only send
its microphone to one device; if it is on a phone call, the Mac gets silence.
This now recovers on its own: the app notices within about five seconds that the
device has stopped delivering, waits without touching anything while no
microphone is available, and rebuilds capture as soon as one is. You will see it
in the log:

```
[WARN] You: no audio for 31s — the device is gone. Rebuilding capture.
[INFO] You: capture restored on 'Your Headset'
```

**You are muted in the meeting but still being transcribed.** Expected — muting
in Zoom or WeChat silences your outgoing audio, not this app's own input stream.
Use **Pause Mic**. It resets to off every launch, deliberately: a pause that
survived a restart would look like recording and not be.

**More speakers than people.** Expected during the meeting, and fixable
afterwards. A voice print reflects what is being said as much as who is saying
it, so the same person reading out a number and the same person talking come
back as different speakers. Set the number of people before the meeting, or
merge at export, where all the prints are clustered together instead of one
utterance at a time.

The prints are recorded whether or not speaker labels are switched on, so
turning labels off does not cost you the ability to sort the speakers out
later.

---

## 🛠️ Development

```bash
git clone https://github.com/colakang/echoai_helper.git
cd echoai_helper
uv venv --python 3.12 .venv
uv pip install --python .venv/bin/python -e .
.venv/bin/python -m pytest tests/ -q
.venv/bin/python main.py            # run from the checkout
```

[`docs/macos-audio-setup.md`](docs/macos-audio-setup.md) covers the audio
routing, what has been measured, and where the sharp edges are.

---

## 📝 Known limitations

- **Recovering the microphone interrupts the meeting audio for ~0.4s.** When a
  device dies, PortAudio can only be recovered by restarting it, which
  invalidates every open stream — so the far-end track is rebuilt too, whether
  or not anything was wrong with it. Measured at 378ms. It buys back a
  microphone track that would otherwise be dead for the rest of the call.
- **Live speaker labels are a rough guide; the export is where they are
  settled.** Measured on one speaker replayed through the app, the voice print
  depends on *what* is being said, not only on who is saying it: two turns of
  ordinary speech score 0.741 against each other, two turns of read-out digits
  score 0.746 — and the same person's speech against their own digits scores
  **0.473**, below the threshold that decides "same speaker". 82% of
  same-person pairs across different content are judged to be different people.

  No threshold fixes that. On the same recording the same-person range
  (0.276–0.988) sits entirely inside the different-person range (0.276–0.820),
  so every setting either splits one person or merges two.

  What does work is re-clustering at export, which sees all the voice prints
  at once and is given the real number of people, rather than deciding one
  utterance at a time. **Set the headcount before the meeting, or merge at
  export** — the voice prints are recorded either way, including when speaker
  labels are switched off.

  Utterances shorter than 2 seconds carry no voice print at all and cannot be
  re-grouped afterwards. There is nothing measurable in audio that short; the
  gate is set where a speaker starts matching themselves more than they match
  a stranger.
- **The Windows capture path is untested** since the segmentation rework. Its
  detector threshold was lowered to pass silence through, because segmentation
  now happens on pauses and a recogniser that only reports speech never delivers
  them — reasoned, not verified. Reports welcome.
- **Intel Macs fall back to the CPU.** PyTorch has shipped no Intel-Mac build
  since 2.2.2, and there is no Metal path on Intel regardless; `uv` resolves to
  that older torch and installs cleanly. But an M4 already measures a dual-track
  real-time factor of 2.04 on CPU — falling behind twice over — and an Intel CPU
  is slower again. Expect transcription not to keep up with a live meeting.
  Untested: reasoned from the wheel availability and the CPU measurement.
- **macOS audio routing is a shared setting.** Selecting the Multi-Output changes
  the output for every app, and macOS sometimes moves it back after sleep.
  Checked at every launch.

## 🤝 Contributing

Pull requests welcome; for anything substantial, open an issue first. See
[CONTRIBUTING.md](CONTRIBUTING.md).

## 🙌 Credits

- [FunASR](https://github.com/modelscope/FunASR) — speech recognition and speaker
  embeddings
- [silero-vad](https://github.com/snakers4/silero-vad) — voice activity detection
- [WhisperLiveKit](https://github.com/QuentinFuxa/WhisperLiveKit) — the
  LocalAgreement idea behind stable partial transcripts
- [BlackHole](https://existential.audio/blackhole/) — virtual audio device on macOS
- [CustomTkinter](https://github.com/TomSchimansky/CustomTkinter) — interface
- [Ecoute](https://github.com/SevaSk/ecoute) — the original inspiration
- [@zixing0131](https://github.com/zixing0131) — core audio processing

## 📞 Contact

[echo365.ai](https://www.echo365.ai) · [Issues](https://github.com/colakang/echoai_helper/issues)

## 📄 License

MIT — see [LICENSE](LICENSE).
