01 Install
Python 3.11+ required. Local LLMs need Ollama; cloud LLMs need only an API key (Groq free tier works).
pip install jarvisepip install "jarvise[all]" # voice + google + microsoft + browserpip install -e . # from source (development).env and ./data/ — nothing leaves your machine unless you choose a cloud LLM.02 First three commands
jarvise -tutorial
Opens this guide in your browser — the full flow, all commands, every feature.
jarvise --way
Interactive cloud setup: pick Groq / OpenRouter / Google, paste key, pick model.
jarvise
Everyday use: auto-detects your best local model, starts voice + CLI together.
jarvise -tutorialjarvise --wayjarvise --modeljarvise --ollama3.2 -run # pull model if needed, then runjarvise --way --text-only # no mic / speaker03 Command reference
Launch flags
| Command | What it does |
|---|---|
jarvise -tutorial | Opens this interactive guide in the browser |
jarvise --way | Pick cloud provider (Groq / OpenRouter / Google) interactively |
jarvise --model | Auto-detect the best local Ollama model for your hardware |
jarvise --<name> -run | Pull <name> if missing, then run it locally |
jarvise --text-only | Skip the voice pipeline — keyboard only |
jarvise --verbose | Debug logs: STT, TTS, LLM timing, tool calls |
jarvise --disable-router | Bypass fast-path router, send everything to the LLM |
jarvis | Alias of jarvise (same entry point) |
In-session shortcuts (type at the prompt)
| Type | Action |
|---|---|
suggest | Show proactive suggestions learned from your patterns |
triage | Email triage — urgent / important / promotional buckets |
conflicts | Detect calendar conflicts |
suggest times | Meeting slots that avoid your focus blocks (9–12) |
exit / quit | Shutdown JARVIS |
04 System flow — what happens to your words
Every input, whether spoken or typed, walks the same pipeline. Simple commands skip the LLM entirely via the router.
Space, speak, release · text — type at You: prompt16kHz → VAD filters silence/noise → 1.0s silence timeout →
Faster-Whisper STT → text + confidence. Say "stop" to interrupt TTS.
"what time is it" answers in milliseconds. Everything else → ReAct agent.
Thought → Action or a final answer. Streaming — TTS starts on the first sentence.
Action: tool: {args}, checks risk tier, executes, feeds Observation back to the LLM.05 Voice interaction
Push-to-talk
Hold Space to record. Release → VAD confirmed speech → 1.0s silence window → Whisper transcribes → JARVIS answers out loud. Hardware-aware model (tiny/base/small/medium) picked from your VRAM.
Streaming + instant replies
TTS speaks the first sentence as soon as the LLM streams it. Greetings like "hello" / "thanks" hit a quick-response table — zero LLM calls. Short queries skip RAG.
06 Human-in-the-loop safety
Every action is classified. You always approve the risky ones.
Confirm? [y/N] with a clear summary. 5-minute undo window.execute_code, financial, legal. Blocked until you approve; every decision appended to data/audit.jsonl.07 Memory & RAG
ChromaDB
Conversations embedded with MiniLM, similarity-searched each turn and injected into the prompt as context.
MEMORY.md
Human-editable file of distilled facts. Ask JARVIS to “remember…” — it lands here.
Preferences
Learned command→tool mappings improve router accuracy over time. forget_stale prunes old memories.
08 All 9 features
Voice pipeline
STT, TTS, push-to-talk, VAD, interrupt, hardware-aware models.
HITL engine
3-tier risk, confirmations, undo tracker, JSONL audit log.
RAG memory
ChromaDB vectors, MEMORY.md, preferences, forgetting.
Proactive suggestions
Bigram + tool-pattern learning → subtle follow-up tips after each reply.
Calendar + email
Triage (urgent/promo), conflict detection, meeting slots, draft queue.
Browser automation
15 actions: navigate, click, fill, screenshot, tabs + multi-step research with citations.
Code sandbox
Python/shell with blocked imports, diff preview before write, execution history.
Multi-agent
Researcher/coder/reviewer/planner roles, parallel or pipeline, token budgets.
Task planner
Goal → dependency graph, statuses, auto-unblock, replan, persisted plans.
09 All 19 tools
| Tool | Risk | What it does |
|---|---|---|
browser | GREEN | Navigate, extract, click, fill, screenshot, tabs |
research | GREEN | Multi-step search → extract → compiled answer with citations |
web_search | GREEN | DuckDuckGo search |
filesystem | YELLOW | Read / write / delete files (write+delete confirmed) |
execute_code | RED | Sandboxed python/shell/pip, diff preview, history |
google_calendar | YELLOW | List/create/update Google Calendar events |
google_email | YELLOW | Gmail read and send |
outlook | YELLOW | Microsoft Outlook / Exchange |
system_monitor | GREEN | CPU, RAM, GPU live stats |
get_time / get_date | GREEN | Current time and date (router fast-path) |
pwd | GREEN | Current working directory |
remember / recall | GREEN | Save and semantic-search memories |
list_memories / forget | GREEN | List or delete stored memories |
calendar_email | YELLOW | Unified triage, conflicts, meeting slots, drafts |
multi_agent | YELLOW | Spawn sub-agents for parallel work |
planner | GREEN | Create/track/replan task plans |
10 Troubleshooting
| Problem | Fix |
|---|---|
| Voice pipeline failed at startup | Run jarvise --way --text-only, or install voice extras: pip install "jarvise[voice]" |
| Ollama not responding | ollama serve then ollama list |
| Groq API error | Re-check key: jarvise --way re-prompts and rewrites .env |
| Responses feel slow | Use --way with Groq (fastest). Greetings are instant; long queries skip RAG |
| No speaker output | Check data/kokoro_models/ has kokoro-v1.0.int8.onnx + voices-v1.0.bin |
| Windows console encoding errors | Run inside a UTF-8 terminal, or use --text-only |
| Want full logs | jarvise --way --verbose |