Metadata-Version: 2.4
Name: lexical-workspace
Version: 0.1.1
Summary: Local AI workspace: LLM chat with RAG, persistent memory, image & audio generation, agents, and an interactive CLI.
Author-email: Samyak <samyak@example.com>
License: MIT
Requires-Python: <3.13,>=3.10
Description-Content-Type: text/markdown
Requires-Dist: llama-cpp-python==0.2.90
Requires-Dist: sentence-transformers==3.1.1
Requires-Dist: diffusers==0.30.3
Requires-Dist: accelerate==0.34.2
Requires-Dist: transformers==4.41.2
Requires-Dist: torch==2.2.2
Requires-Dist: chromadb==0.5.5
Requires-Dist: faiss-cpu==1.9.0
Requires-Dist: markdown-it-py==3.0.0
Requires-Dist: pymdown-extensions==10.9
Requires-Dist: rich==13.7.1
Requires-Dist: prompt-toolkit==3.0.47
Requires-Dist: click==8.1.7
Requires-Dist: pyyaml==6.0.1
Requires-Dist: tqdm==4.66.5
Requires-Dist: numpy==1.26.4
Requires-Dist: pydantic==2.8.2
Requires-Dist: pydantic-settings==2.4.0
Requires-Dist: gguf==0.19.0
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-asyncio; extra == "dev"
Requires-Dist: pytest-mock; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Requires-Dist: mypy; extra == "dev"
Provides-Extra: gpu
Requires-Dist: llama-cpp-python[cuda]; extra == "gpu"
Provides-Extra: audio
Requires-Dist: ace-step; extra == "audio"
Provides-Extra: games
Requires-Dist: python-chess; extra == "games"
Requires-Dist: pygame; extra == "games"
Provides-Extra: integrations
Requires-Dist: google-auth>=2.23.0; extra == "integrations"
Requires-Dist: google-auth-oauthlib>=1.1.0; extra == "integrations"
Requires-Dist: google-api-python-client>=2.100.0; extra == "integrations"
Requires-Dist: requests>=2.31.0; extra == "integrations"

# Lexical LLM Assistant

A fast, **CPU-only** local LLM assistant that learns from your Markdown files and
conversations. Everything runs on your own machine — no GPU, no API keys, no
network calls by default.

- **CPU-Only Inference** — `llama-cpp-python` with GGUF models (no GPU required)
- **Markdown Learning** — ingest `.md` files with header-aware chunking + hybrid search
- **Persistent Memory** — remember facts across sessions (`/memory`, `remember`, auto-capture)
- **Hybrid Retrieval** — BM25 (sparse) + FAISS dense cosine + cross-encoder rerank
- **Local Image Generation** — Stable Diffusion (txt2img / img2img), interactive settings
- **Local Audio Generation** — MusicGen / ACE-Step text-to-music, interactive settings
- **LoRA Training** — optional fine-tuning on your own data
- **Agent Mode** — local tool-use (calculator, file read, doc search, open app, shell)

---

## Table of Contents

1. [Quick Start](#quick-start)
2. [Running the CLI](#running-the-cli)
3. [Command Reference](#command-reference)
   - [System](#system)
   - [Chat & Memory](#chat--memory)
   - [Documents](#documents)
   - [Models](#models)
   - [LoRA](#lora)
   - [Images](#images)
   - [Audio](#audio)
   - [Agent](#agent)
   - [Config & Stats](#config--stats)
4. [Natural-Language Memory](#natural-language-memory)
5. [Configuration](#configuration)
6. [CPU Optimization](#cpu-optimization)
7. [LoRA Fine-Tuning](#lora-fine-tuning-optional)
8. [Project Structure](#project-structure)
9. [Running Tests](#running-tests)
10. [Troubleshooting](#troubleshooting)
11. [License & Acknowledgments](#license--acknowledgments)

---

## Quick Start

### 1. Install Dependencies

```bash
cd lexical-llm
pip install -r requirements.txt
# Optional: install as a package so `lexical` / `lexical-train` / ... entry
# points are available.
pip install -e .
# Optional: enable the ACE-Step audio backend (slow on CPU; needs newer deps)
pip install ace-step
```

> **Dependency note:** `torch==2.2.2` (CPU), `transformers==4.41.2`, and
> `diffusers==0.30.3` are pinned on purpose. **Do not bump diffusers** — the audio
> and image backends are validated against these versions. ACE-Step is kept as an
> opt-in extra so its (potentially newer) `transformers` requirement never affects
> MusicGen or image generation unless you install it *and* select it.

### 2. Get a Model

Download a small GGUF model (1B–3B params, Q4_K_M quantization recommended) into
`models/`:

```bash
# Llama 3.2 3B Instruct (~2GB)
wget -P models \
  https://huggingface.co/bartowski/Llama-3.2-3B-Instruct-GGUF/resolve/main/Llama-3.2-3B-Instruct-Q4_K_M.gguf

# Phi-3 Mini 3.8B (~2.3GB)
wget -P models \
  https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/resolve/main/Phi-3-mini-4k-instruct-q4.gguf

# Gemma 2 2B (~1.5GB)
wget -P models \
  https://huggingface.co/bartowski/gemma-2-2b-it-GGUF/resolve/main/gemma-2-2b-it-Q4_K_M.gguf
```

You can also let the CLI download one for you later with
`/download-model <url or hf-repo:file>`.

### 3. (Optional) First-Time Setup

```bash
python -m lexical_llm.cli --ingest     # alias: lexical-setup (creates dirs + config)
```

`/setup` inside the CLI launches an interactive wizard (model pick + download +
`config.yaml`). Non-interactive contexts fall back to silent setup.

### 4. Add Documents

```bash
mkdir -p data/docs
cp your-notes/*.md data/docs/
```

### 5. Ingest

```bash
python -m lexical_llm.scripts.ingest        # or: lexical-ingest
```

…or run `/ingest` from inside the CLI (rebuilds the hybrid index automatically).

### 6. Run

```bash
python -m lexical_llm.cli
```

After `pip install -e .` you can also just run `lexical`.

---

## Running the CLI

| Action | Command |
|--------|---------|
| Launch chat | `python -m lexical_llm.cli` (or `lexical`) |
| Custom config | `python -m lexical_llm.cli --config path/to/config.yaml` |
| Override model | `python -m lexical_llm.cli --model models/foo.gguf` |
| Set thread count | `python -m lexical_llm.cli --threads 6` |
| Ingest then exit | `python -m lexical_llm.cli --ingest` |
| Standalone ingest | `python -m lexical_llm.scripts.ingest` (or `lexical-ingest`) |
| Standalone setup | `python -m lexical_llm.cli --ingest` (or `lexical-setup`) |
| LoRA training | `python -m lexical_llm.train_lora ...` (or `lexical-train`) |

Relative paths (config, model, docs) resolve against the **project root**, so the
CLI works from any working directory.

---

## Command Reference

All slash commands are tab-completable. Type `/help` inside the CLI for the live
list. Commands are grouped below.

### System

| Command | Syntax | Description |
|---------|--------|-------------|
| Help | `/help` (also `/?`) | Show the in-CLI help panel |
| Exit | `/exit` (also `/quit`) | Quit the assistant |
| Clear | `/clear` | Clear conversation history |
| Setup | `/setup` | Run the first-time setup wizard (dirs + config) |
| Reload | `/reload` | Reload the model and search indexes |

```text
/help
/exit
/clear
/setup
/reload
```

### Chat & Memory

| Command | Syntax | Description |
|---------|--------|-------------|
| Memory list | `/memory` | List all stored facts (with IDs, source, tags) |
| Memory add | `/memory add <fact>` | Store a fact manually |
| Memory delete | `/memory del <id>` | Delete a fact by its ID |
| Memory search | `/memory search <query>` | Keyword + semantic search over facts |
| Memory auto | `/memory auto <on|off>` | Toggle automatic fact capture from chat |

Plain text messages are normal chat. Prefix a line with `remember` to store a fact
directly. Auto-memory (on by default) extracts durable facts from ordinary
sentences ("I like Python", "my project is…").

```text
/memory
/memory add I prefer dark mode in editors
/memory del 07a1fc32f97c481a
/memory search which language do i like
/memory auto off
remember I work on a Rust project called Ferry
```

See [Natural-Language Memory](#natural-language-memory) for the `remember` /
`note:` / `actually` forms.

### Documents

| Command | Syntax | Description |
|---------|--------|-------------|
| Ingest | `/ingest` | Ingest every file in `data/docs` |
| Ingest path | `/ingest <path>` | Ingest a specific file or directory |
| Reload | `/reload` | Reload model **and** indexes (after editing docs) |

After `/ingest`, the hybrid (BM25 + FAISS) retriever is rebuilt and saved
automatically — there is no separate "build index" step.

```text
/ingest
/ingest data/docs/notes/
/ingest my-standalone-file.md
```

**Retrieval relevance gate:** results below `retrieval.similarity_threshold`
(0.3) are filtered out, so irrelevant queries return nothing instead of a
junk citation.

### Models

| Command | Syntax | Description |
|---------|--------|-------------|
| Model info | `/model` | Show current model path, context, threads, params + installed list |
| Load model | `/model <path>` | Load a different GGUF (absolute or name under `models/`) |
| Models picker | `/models [n|name]` | Interactive selector of installed GGUF files |
| Download | `/download-model <url|repo:file> [name.gguf]` | Fetch a GGUF into `models/` |
| Threads show | `/threads` | Show current CPU thread count |
| Threads set | `/threads <n>` | Set thread count (reloads model; re-applies active LoRA) |

```text
/model
/model models/Phi-3-mini-4k-instruct-q4.gguf
/models
/models 2
/download-model https://huggingface.co/owner/model/resolve/main/m.q4_k_m.gguf
/download-model owner/model.q4_k_m.gguf
/threads
/threads 6
```

> **Model path resolution:** absolute paths are used as-is; relative names always
> resolve inside the canonical `models/` directory (never the repo root or a system
> path). Changing threads reloads the model and **reapplies any active LoRA**.

### LoRA

| Command | Syntax | Description |
|---------|--------|-------------|
| List | `/lora` | List LoRA adapters found in `lora/` |
| Load | `/lora <path>` | Load a LoRA adapter (`.gguf` or `*/adapter*.bin`) |
| Unload | `/lora none` | Unload the active adapter |

```text
/lora
/lora lora/my-adapter
/lora none
```

### Images

| Command | Syntax | Description |
|---------|--------|-------------|
| Text-to-image | `/image <prompt>` | Generate an image from a text prompt (local SD) |
| Image-to-image | `/img2img <path> <prompt>` | Transform an image (photo → anime, etc.) |
| Image-to-image (alias) | `/image2image <path> <prompt>` | Same as `/img2img` |
| Select model | `/img-model` | List & select the Stable Diffusion checkpoint |

Both `/image` and `/img2img` ask **"Use recommended settings? [Y/n]"** after you
enter the prompt. Press Enter (or `y`) to use the defaults; `n` opens an
interactive tweaker:

| Setting | Range | Default | Notes |
|---------|-------|---------|-------|
| Steps | 1–150 | 25 | Diffusion steps |
| Guidance | 1.0–30.0 | 7.5 | Classifier-free guidance scale |
| Max resolution | 256–2048 | 768 | Width = height (px) |
| Strength | 0.0–1.0 | 0.6 | **img2img only** — 0 = keep original, 1 = fully redo |

`/img2img` accepts the path first (`/img2img img.png anime style`) or last
(`/img2img anime style img.png`); with no args it prompts interactively.

```text
/image a serene mountain lake at sunset, cinematic
/img2img photo.jpg in the style of a 1980s anime
/image2image photo.jpg oil painting of a harbor
/img-model
```

### Audio

| Command | Syntax | Description |
|---------|--------|-------------|
| Generate | `/audio <prompt>` | Generate audio/music (MusicGen or ACE-Step) |
| Settings shortcut | `/audio --sec N <prompt>` | Skip the panel, set clip length to N seconds |
| Select model | `/audio-model [n|name|backend|repo]` | List & select the audio backend/model |

`/audio` opens a **settings panel** (clip length in seconds + guidance scale)
unless you use the `--sec N` shortcut. Recommended defaults: MusicGen guidance
3.0; ACE-Step guidance 7.5 / steps 100; clip length 8.0s (configurable). Press
`n` to tweak:

| Setting | Range | Default | Backend |
|---------|-------|---------|---------|
| Clip length (sec) | 1.0–30.0 | 8.0 | both |
| Guidance | 1.0–15.0 | 3.0 (MG) / 7.5 (ACE) | both |
| Steps | 1–200 | 100 | ACE-Step only |

`/audio-model` lists local models in `models/audio/` (and `models/`). Pick by
number/name, type `musicgen` or `acestep` to use that backend's default, or type
any HuggingFace repo id (`owner/model`). Unsupported entries (a lone weight file,
a folder without `config.json`) are flagged `✗` with a reason.

```text
/audio a calm lo-fi beat
/audio --sec 12 an epic orchestral score
/audio-model
/audio-model acestep
/audio-model facebook/musicgen-medium
```

> **CPU caveat:** MusicGen-small handles short clips reasonably; ACE-Step (3.5B)
> is **very slow and memory-hungry on CPU** — a hardware limit, not a bug. If a
> generation produces near-silent audio, the CLI warns (the checkpoint is likely
> broken/untrained rather than a code error).

### Agent

| Command | Syntax | Description |
|---------|--------|-------------|
| Toggle | `/agent [on|off]` | Enable/disable local tool-use agent mode |

```text
/agent on
/agent off
/agent          # show current status
```

When **on**, chat is routed through a tool-use loop. Available local tools:
`calculator`, `read_file`, `search_docs`, `open_app`, `open_file`, `shell`. OS
actions (`open_app` / `open_file`) and the `shell` tool are configurable in
`AgentConfig`.

### Config & Stats

| Command | Syntax | Description |
|---------|--------|-------------|
| Config | `/config` | Print the active configuration (YAML) |
| Stats | `/stats` | Show session stats (facts, chunks, LoRA, last gen speed) |

```text
/config
/stats
```

---

## Natural-Language Memory

Beyond `/memory add`, the assistant captures facts from ordinary chat:

- `remember I prefer Python over JavaScript` — store a fact explicitly.
- `note: my API key is in .env` — shorthand for "remember".
- `actually, the meeting is at 3pm not 2pm` — correction; updates/adds a fact.
- Automatic: first-person statements and preferences ("I like Rust", "my project
  is…") are captured in the background when auto-memory is enabled.

Facts are stored in `data/memory/user_facts.json` and searched with a
keyword + (optional) semantic fallback.

---

## Configuration

`config.yaml` lives at the project root. Relative paths resolve to the project
root. Key defaults:

```yaml
llm:
  model_path: models/llama-3.2-3b-instruct-q4_k_m.gguf
  n_ctx: 32768            # context window (tokens)
  n_threads: 0           # 0 = auto (CPU count - 1)
  n_gpu_layers: 0        # keep 0 for CPU-only
  temperature: 0.7
  max_tokens: 2048

embedding:
  model_name: sentence-transformers/all-MiniLM-L6-v2
  device: cpu

reranker:
  enabled: true
  device: cpu

retrieval:
  top_k: 5               # candidates retrieved before rerank
  top_k_rerank: 3        # results kept after rerank
  similarity_threshold: 0.3   # relevance gate (filters junk hits)

memory:
  file_path: data/memory/user_facts.json
  auto_memory: true      # capture durable facts from chat
  embed_facts: true      # semantic search over facts (falls back to keyword)

ingestion:
  docs_directory: data/docs

vector_store:
  persist_directory: data/index/chromadb

audio:
  backend: musicgen                  # "musicgen" | "acestep"
  musicgen_model: facebook/musicgen-small
  acestep_model: ACE-Step/ACE-Step-3.5B
  duration: 8.0                      # default clip length (sec)
  acestep_steps: 100
  acestep_guidance: 7.5
  output_dir: data/audio

agent:
  enabled: false
  shell_enabled: true
  open_enabled: true
```

Most fields can also be set via environment variables with a prefix, e.g.
`AUDIO_BACKEND=acestep`, `LLM_N_THREADS=6`.

---

## CPU Optimization

### Thread count
Auto-detected by default (`CPU cores - 1`). Override with `/threads <n>` or
`--threads N`. More threads = faster generation, up to your physical core count.

### Model selection for older CPUs
Intel/AMD x86 CPUs are auto-detected at startup. If your machine lacks AVX2, the
CLI selects a compatible `llama.cpp` build automatically.

| CPU Generation | Recommended Models |
|----------------|--------------------|
| 3rd–4th Gen (Haswell/Skylake) | 1B–3B Q4_K_M (~1.5–2GB) |
| 5th–6th Gen | 3B–7B Q4_K_M (~2–4GB) |
| 7th Gen+ | 7B–13B Q4_K_M (~4–8GB) |

### Memory budget (8 GB RAM example)
- Model (3B Q4): ~2 GB
- Embeddings: ~200 MB
- FAISS index: ~100 MB per 10k chunks
- KV cache: ~2 MB per 1k context
- **Total for 8 GB RAM**: use a 3B model and keep `n_ctx` modest.

---

## LoRA Fine-Tuning (Optional)

For deeper adaptation beyond RAG:

```bash
# 1. Prepare training data (JSONL, messages format)
cat > data/train.jsonl << 'EOF'
{"messages": [{"role": "user", "content": "What is our API?"}, {"role": "assistant", "content": "Our API is REST-based..."}]}
{"messages": [{"role": "user", "content": "How to deploy?"}, {"role": "assistant", "content": "Use docker-compose..."}]}
EOF

# 2. Train a LoRA adapter
python -m lexical_llm.train_lora \
  --data data/train.jsonl \
  --model models/llama-3.2-3b-instruct-q4_k_m.gguf \
  --output lora/my-adapter \
  --epochs 3
# (alias: lexical-train ...)

# 3. Use it in the CLI
/lora lora/my-adapter
```

---

## Project Structure

```text
lexical-llm/
├── config.yaml                # Configuration (relative paths → project root)
├── requirements.txt           # Python dependencies
├── pyproject.toml             # Package metadata + entry points
├── data/
│   ├── docs/                 # Markdown files to ingest
│   ├── index/                # Vector indexes (FAISS + BM25 + Chroma)
│   ├── memory/               # User facts (JSON)
│   ├── images/               # Generated images
│   └── audio/                # Generated audio (.wav)
├── models/                   # GGUF models
│   ├── audio/                # Local audio model folders / weights
│   └── image/                # Local image checkpoints
├── lora/                     # LoRA adapters
├── src/lexical_llm/
│   ├── __init__.py
│   ├── config.py             # Configuration management
│   ├── model.py              # llama.cpp wrapper
│   ├── ingest.py             # Markdown parsing / chunking / embeddings
│   ├── memory.py             # User memory store
│   ├── retrieval.py          # Hybrid search (BM25 + FAISS + rerank)
│   ├── imagegen.py           # Stable Diffusion (txt2img / img2img)
│   ├── audiogen.py           # MusicGen / ACE-Step
│   ├── agent.py              # Local tool-use agent
│   ├── cli.py                # Interactive CLI
│   ├── train_lora.py         # LoRA training
│   ├── scripts/ingest.py     # Standalone ingest entry point
│   └── utils.py              # Utilities
└── tests/                    # Unit tests
```

---

## Running Tests

```bash
pytest tests/ -v
```

Tests use mocked models/embedders (no downloads). Audio and image tests exercise
the CLI/settings paths with fake pipelines so they run without GPU or network.

---

## Troubleshooting

### Model fails to load
- Check `model_path` in `config.yaml` (or your `--model` argument).
- Confirm the GGUF file is valid.
- A relative model name must exist under `models/`.

### Out of memory
- Reduce `n_ctx` (default 32768 — large for low-RAM machines; 4096–8192 is often
  plenty).
- Use a smaller model (1B–3B) or heavier quantization (Q3_K / Q2_K).
- Close other applications.

### Slow generation
- Increase `/threads` up to your physical core count.
- Use a smaller context window.
- Lower the model size or quantization level.
- Audio: ACE-Step is inherently slow on CPU; prefer MusicGen-small for quick clips.

### Silent audio output
- The model checkpoint is likely broken, partially merged, or untrained — the CLI
  warns when peak amplitude is near zero. Re-download a complete model folder
  (config.json + weights + tokenizer for MusicGen).

### Import errors
```bash
pip install -r requirements.txt --force-reinstall
```
Do **not** bump `torch` / `transformers` / `diffusers` from the pinned versions.

### Agent tools not working
- OS actions and the shell tool are gated by `agent.shell_enabled` /
  `agent.open_enabled` in `config.yaml`.

---

## License & Acknowledgments

**MIT License** — see the `LICENSE` file for details.

- [llama.cpp](https://github.com/ggerganov/llama.cpp) — CPU inference engine
- [sentence-transformers](https://www.sbert.net/) — embeddings
- [ChromaDB](https://www.trychroma.com/) — vector store
- [FAISS](https://github.com/facebookresearch/faiss) — dense similarity search
- [rank-bm25](https://github.com/dorianbrown/rank_bm25) — sparse BM25 search
- [diffusers](https://github.com/huggingface/diffusers) — image generation
- [transformers](https://github.com/huggingface/transformers) — MusicGen / ACE-Step
- [rich](https://github.com/Textualize/rich) + [prompt_toolkit](https://github.com/prompt-toolkit/python-prompt-toolkit) — the terminal UI
