Metadata-Version: 2.4
Name: spec-probe
Version: 0.1.12
Summary: Verify C/C++ modules implement functional requirements from spec (LangGraph + LLM)
Author: taikt
License-Expression: MIT
Project-URL: Homepage, https://github.com/taikt/spec-probe
Project-URL: Documentation, https://github.com/taikt/spec-probe/blob/main/QUICKSTART.md
Project-URL: Repository, https://github.com/taikt/spec-probe
Project-URL: Issues, https://github.com/taikt/spec-probe/issues
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: langgraph>=0.2
Requires-Dist: langchain-core>=0.3
Requires-Dist: langchain-openai>=0.2
Requires-Dist: pydantic>=2.0
Requires-Dist: pydantic-settings>=2.0
Requires-Dist: loguru>=0.7
Requires-Dist: python-dotenv>=1.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: httpx>=0.27
Requires-Dist: requests>=2.31
Requires-Dist: markitdown>=0.1
Requires-Dist: mammoth>=1.8
Requires-Dist: atlassian-python-api>=3.40
Requires-Dist: markdown>=3.5
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23; extra == "dev"
Provides-Extra: rag
Requires-Dist: langchain-community>=0.3; extra == "rag"
Requires-Dist: langchain-huggingface>=0.1; extra == "rag"
Requires-Dist: sentence-transformers>=3.0; extra == "rag"
Requires-Dist: faiss-cpu>=1.8; extra == "rag"
Dynamic: license-file

# spec-probe

Check code against functional requirements in a spec (`.md` / `.docx` / `.pdf`). Use **VS Code Copilot** chat (`/spec-cover`, …) or the **`spec-probe`** CLI.

---

## Workflow Overview


| Type                         | Step                | Action                                                | Description                                               |
| :----------------------------- | :-------------------- | :------------------------------------------------------ | :---------------------------------------------------------- |
| **One-time** *(Run once)*    | 1. Install & Skills | `pip install spec-probe && spec-probe-skill --vscode` | Sets up package,`.venv`, and Copilot skills               |
|                              | 2. Auth & Keys      | `spec-probe-copilot-login` + Exacode API key          | Sets up Copilot & Exacode credentials in `.env`           |
|                              | 3. Configure Paths  | `~/.config/spec-probe/config.yaml`                    | Sets `module_path`, `spec_path`, `related_modules`, codegraph |
|                              | 4. Index Code       | `spec-probe-codegraph`                                | Builds symbol & call-graph index (excluding tests/mocks)  |
|                              | 5. Domain Pack      | `/spec-pack` *(in chat)*                              | Extracts domain glossary, acronyms, and heuristics        |
|                              | 6. Parse Spec       | `/spec-cover parse spec summary` *(in chat)*          | Parses spec into requirements catalog in`reports/specs/`  |
| **Daily** *(Run repeatedly)* | 1. Verify FRs       | `/spec-cover verify FR ...` *(in chat)*               | Verifies requirements; generates draft results in cache   |
|                              | 2. Audit Report     | `/spec-audit` *(in chat)*                             | **Mandatory quality gate**: Audits draft results with **Gemini 3.8 Flash**, validates code evidence, confirms true gaps |
|                              | 3. Export Report    | `/spec-cover write report` *(in chat)*                | Exports final verified `.md` and `.html` report to user report dir |

---

## 1. One-Time Setup

### Step 1: Install & Skills

```bash
pip install spec-probe
spec-probe-skill --vscode
```

- Installs `spec-probe` and sets up Copilot skills in `~/.copilot/skills/`.
- Initializes user directory and `.venv` under `~/.config/spec-probe/`.
- **Reload VS Code** after running `spec-probe-skill`.

Optional for Linux: `sudo apt install ripgrep` (faster grep search).

### Step 2: Authenticate & API Keys (`.env`)

#### A. GitHub Copilot Token (Required for Verification & Review)

```bash
spec-probe-copilot-login
```

1. Open `https://github.com/login/device` in your browser.
2. Enter the 8-character code displayed in the terminal.
3. Token is saved automatically to `COPILOT_GITHUB_TOKEN` in `~/.config/spec-probe/.env`.

#### B. Exacode API Key (Required for High-Volume Jobs)

Exacode is an LGE-internal free LLM gateway. It powers high-frequency pipeline jobs (`enrich`, `search`, `rescue`, `report`, `wiki`) without token cost or external rate limits.

1. Open `http://exacode-chat.lge.com/key/` (or clear and re-create your key).
2. Copy **Key 2 (EXACODE SWE API)**.
3. Add to `~/.config/spec-probe/.env`:
   ```bash
   EXACODE_API_KEY="your_key_2_here"
   EXACODE_BASE_URL="http://exacode-chat.lge.com/v1"
   ```

*(Details: [Exacode for Copilot Chat](http://collab.lge.com/main/spaces/~tai2.tran/pages/3695262088/Exacode+for+Copilot+Chat))*

### Step 3: Configure Target Paths (`config.yaml`)

Edit `~/.config/spec-probe/config.yaml` (use absolute paths):

```yaml
input:
  module_path: /abs/path/to/your/app
  spec_path: /abs/path/to/spec.docx
  related_modules:
    # paths:
    #   - /abs/path/to/shared/service
    #   - /abs/path/to/core/service

output:
  # Directory to store reports (default: ~/.config/spec-probe/reports)
  # report_dir: /path/to/my/reports

codegraph:
  # Base codebase / monorepo root to index with codegraph (optional)
  # When set, running `spec-probe-codegraph` indexes this root directly.
  # If omitted, defaults to indexing module_path and related_modules.paths.
  # root_path: /abs/path/to/codebase/root
```

### Step 4: Index Code with Codegraph

Builds an AST symbol table and call-graph database essential for accurate C/C++ cross-module symbol resolution:

```bash
spec-probe-codegraph
```

- When `codegraph.root_path` is set in `config.yaml`, it indexes the codebase root directly with a single command.
- If not set, it automatically indexes `module_path` and `related_modules.paths`.
- You can also pass paths directly: `spec-probe-codegraph /path1 /path2`. Tests and mocks are excluded automatically. Re-running syncs changes.

### Step 5: Draft Domain Pack (Run once per module)

In VS Code Copilot chat:

1. **Select Model in Chat Picker:** Choose **`Gemini 3.8 Flash`** (recommended for huge 265k context window and fast code analysis) or **`GPT-6 Luna`**.
2. Run slash command:
   ```
   /spec-pack
   ```

*Why run via Copilot Chat Agent instead of internal API calls?*
Domain Pack creation is an **agentic, multi-step synthesis**: it scans AST/symbols, reads 3–8 core architecture and state-machine files, cross-checks spec vocabulary, drafts project-specific rules, and validates patterns with `spec_pack.py validate --strict`. Running as an interactive Copilot Chat agent lets the developer review and tailor domain heuristics with human-in-the-loop flexibility.

### Step 6: Parse Spec (Run once per spec)

In VS Code Copilot chat:

```
/spec-cover parse spec summary
```

Parses the requirements document into a structured catalog cached under `~/.config/spec-probe/reports/specs/` so subsequent verifications are fast.

> **Zero Token Cost by Default (0 LLM Tokens):** Spec parsing is 100% deterministic local Python execution (using regex, AST, and table parsing via `load_spec_file`). No external LLM API is called, consuming 0 tokens. In Copilot Chat, any chat model picker works equally well (e.g. lightweight models like Haiku or Flash).
>
> *(Novel / Unstructured Spec Fallback)*: If a specification lacks standard tables and numbered headers, `spec-probe parse --spec <path> --llm-fallback` uses per-section extraction (`llm.jobs.parse`) to accurately capture requirements without context overflows.

---

## 2. Daily Workflow (VS Code Copilot Chat)

Once one-time setup is complete, use chat for regular verification:

### Step 1: Verify Requirements (`/spec-cover`)

- Verify a specific requirement:
  ```
  /spec-cover verify FR-001
  ```
- Or verify by requirement text:
  ```
  /spec-cover check requirement: The module shall initialize within 500ms
  ```
- Or verify a section:
  ```
  /spec-cover verify section 3.2
  ```

*(Note: In Copilot chat, typing naturally without `/spec-cover` like `verify FR-001` also works via auto-intent matching, but `/spec-cover ...` is recommended for explicit routing).*

During verification, results and a temporary draft report are automatically persisted to `~/.config/spec-probe/cache/` (`last_verify_session.json` and `draft_verify_latest.md`). No unwanted report files are written to your project output directory yet.

### Step 2: Audit Report (`/spec-audit`) — **Mandatory Quality Gate**

> **Crucial & Required Step:** Automated scan results from Step 1 are treated as unconfirmed hypotheses. This step audits the temporary draft results, eliminates AI hallucinations, validates code citations line-by-line, and confirms true implementation gaps with zero risk of false positives.
>
> *(Note on token cost: `/spec-audit` is extremely token-efficient — it does not read the entire codebase. It runs targeted ripgrep searches only on suspected gaps and NOT_FOUND items, consuming very few tokens while boosting accuracy to 100%).*

In VS Code Copilot chat:

1. **Select Model in Chat Picker:** Choose **`Gemini 3.8 Flash`** (recommended for deep reasoning and whole-file code context) or **`GPT-6 Luna`**.
2. Run slash command:
   ```
   /spec-audit
   ```

*Why run via Copilot Chat Agent with a strong model?*
Like `/spec-pack`, `/spec-audit` is an **interactive agentic review**. It treats tool verdicts and "missing" bullets as hypotheses, actively reads surrounding source code, traces evidence chains across caller-callee graphs, resolves ambiguous symbols, confirms true implementation gaps, and reports findings directly in the chat.

### Step 3: Export Final Report File

Once you have reviewed the audited findings in chat, export the official report:

```
/spec-cover write report
```

Generates the final verified `.md` and standalone `.html` reports under your configured report directory (e.g. `reports/verifications/` or path configured in `config.yaml`). Because this is run after auditing, your published reports will always represent the highest-accuracy ground truth.

---

## 3. Technical Tuning & LLM Jobs (`config.json`)

`config.json` is dedicated to technical pipeline tuning and slotting LLM models for each phase.

### LLM Pipeline Phases (`llm.jobs`)

The hybrid strategy balances high reasoning quality for critical verification with free high-throughput inference for pre/post-processing:

| Phase / Slot | Role in Pipeline | Recommended Model | Why? |
| :--- | :--- | :--- | :--- |
| **`verify`** | **Core verification**: Analyzes code candidates against requirement to determine verdicts (`IMPLEMENTED`, `PARTIAL`, `NOT_FOUND`) and cite code evidence. | Copilot (`gemini-3.8-flash`, `gpt-6-luna`) | Requires deep multi-step code reasoning and high accuracy without excessive cost. |
| **`final_review`** | **Secondary review loop**: Inspects full source files to eliminate false positives and catch subtle implementation details. | Copilot (`gemini-3.8-flash`) | Superior large-context code comprehension (265k tokens); fast and highly reliable for inspecting whole source files. |
| **`rescue`** | **Not-found rescue**: Deeper search across caller trees when initial passes yield no candidates. | Copilot (`gemini-3.8-flash`) | **Targeted fallback reasoning**: Only runs when an FR is `NOT_FOUND`, so token usage remains low while candidate recovery rate increases significantly. |
| **`enrich`** | **Requirement analysis**: Extracts C/C++ function signatures, symbols, keywords, and acronyms from requirement text. | Exacode (`Chat-EXACODE-A`) | **Free (internal LGE model)**; high-frequency preprocessing without token cost or external rate limits. |
| **`search`** | **Code search agent**: Ranks and filters candidate files and functions. | Exacode (`Chat-EXACODE-A`) | **Free (internal LGE model)**; iterative search query generation saves external Copilot quota. |
| **`report`** | **Report generation**: Formats executive summaries, gap tables, and markdown output. | Exacode (`Chat-EXACODE-A`) | **Free (internal LGE model)**; fast text formatting and summarization. |
| **`wiki`** | **Domain knowledge**: Generates and retrieves architecture terminology. | Exacode (`Chat-EXACODE-A`) | **Free (internal LGE model)**; internal knowledge indexing. |
| **`translate`** | **Translation**: Translates requirements or reports if multi-language output is enabled. | Exacode (`Chat-EXACODE-A`) | **Free (internal LGE model)**; fast translation without token limits. |
| **`parse`** | **Spec extraction fallback**: Hybrid per-section extraction for novel/unstructured spec layouts. | Exacode (`Chat-EXACODE-A`) or Copilot (`gpt-6-luna`) | **Free on Exacode**; only called on unstructured sections, keeping token overhead minimal. |

### Key Pipeline Parameters (`pipeline`)

- `parallel_workers`: Number of concurrent workers for batch verification (default: 1 or 4).
- `context_lines`: Number of source code context lines included around grep matches (default: 40).
- `batch_size`: Batch size when processing large specifications (default: 20).
- `final_review_enabled`: Enables the secondary whole-file verification pass (default: `true`).

---

## 4. Updating spec-probe

When a new version is released, updating is a single command:

```bash
spec-probe update
```

- Automatically upgrades the package via `pip` (or `git pull` if running from a local dev clone).
- Refreshes IDE skills and slash commands in `~/.copilot/skills/` and `~/.cursor/skills/`.
- **Preserves 100% of your configurations**: never overwrites `config.yaml`, `config.json`, or `.env`.
- To check if an update is available without installing:
  ```bash
  spec-probe update --check-only
  ```

---

## CLI (optional)

```bash
spec-probe verify --module /abs/app --spec /abs/spec.md --fr FR-001
spec-probe check "The module shall …"
spec-probe parse --spec /abs/spec.md --summary
```

Paths default from `~/.config/spec-probe/config.yaml` when omitted.
