Metadata-Version: 2.5
Name: portage-mcp
Version: 0.1.0
Summary: Generate an MCP server from any CLI tool's own --help output.
Project-URL: Homepage, https://github.com/jayaprakash2207/Portage-_MCP
Project-URL: Repository, https://github.com/jayaprakash2207/Portage-_MCP
Project-URL: Issues, https://github.com/jayaprakash2207/Portage-_MCP/issues
Author: Jayaprakash A R
License: MIT
License-File: LICENSE
Keywords: automation,cli,code-generation,mcp,model-context-protocol
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Code Generators
Requires-Python: >=3.10
Requires-Dist: mcp>=1.0
Provides-Extra: dev
Requires-Dist: mypy>=1.11; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest-timeout>=2.3; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.6; extra == 'dev'
Description-Content-Type: text/markdown

<p align="center">
  <img src="https://raw.githubusercontent.com/jayaprakash2207/Portage-_MCP/main/assets/portage-banner.svg" alt="Portage — every CLI, one command from your AI assistant" width="100%">
</p>

<p align="center">
  <b>Turn any command-line tool into an MCP server — by reading its own <code>--help</code>. No per-tool code.</b>
</p>

<p align="center">
  <a href="https://github.com/jayaprakash2207/Portage-_MCP/actions/workflows/ci.yml"><img alt="CI" src="https://github.com/jayaprakash2207/Portage-_MCP/actions/workflows/ci.yml/badge.svg"></a>
  <img alt="tests" src="https://img.shields.io/badge/tests-360%20passing-2ea44f">
  <img alt="integration" src="https://img.shields.io/badge/integration-16%20real%20CLIs-2ea44f">
  <img alt="coverage" src="https://img.shields.io/badge/coverage-~91%25-2ea44f">
  <img alt="mypy" src="https://img.shields.io/badge/mypy-strict-1f6feb">
  <img alt="ruff" src="https://img.shields.io/badge/lint-ruff-d7ff64">
  <img alt="python" src="https://img.shields.io/badge/python-3.10%2B-1f6feb">
  <img alt="mcp" src="https://img.shields.io/badge/MCP-SDK%202.x-7C3AED">
  <img alt="license" src="https://img.shields.io/badge/license-MIT-2ea44f">
</p>

<p align="center">
  <a href="#-quick-start">Quick start</a> ·
  <a href="#-how-it-works">How it works</a> ·
  <a href="#-security-model">Security</a> ·
  <a href="#-cli">CLI</a> ·
  <a href="#-configuration">Config</a> ·
  <a href="#-remote-deployment">Remote</a> ·
  <a href="#-roadmap">Roadmap</a>
</p>

---

## What is this?

There are thousands of command-line tools — `ffmpeg`, `jq`, `ripgrep`, `git`,
`curl`, your own scripts — and almost none of them have a
[Model Context Protocol](https://modelcontextprotocol.io) server, so an AI
assistant can't use them. Writing one by hand for every CLI is repetitive busywork.

**Portage does it automatically.** Point it at a CLI, and it:

```
<tool> --help  ──▶  parse (sections · usage · options · commands · type inference)
               ──▶  normalized CLI IR  ──▶  JSON Schema / MCP tool definitions
               ──▶  stdio (or HTTP) MCP server  ──▶  your AI assistant
                                              └──▶  safe, allow-listed execution
```

One generic pipeline handles every CLI. There is **no** `if tool == "git"` anywhere —
a test enforces it.

> **Status — local MVP complete.** Discovery, `--help` **and** man-page parsing,
> schema generation, the stdio/HTTP MCP server, the execution engine, and the
> full safety layer are implemented and tested (360 unit tests, 16 integration
> against real `jq` / `ripgrep` / `curl` / `git` / `ffmpeg`, ~91 % coverage,
> `ruff` + `mypy --strict` clean). The remote front door is scaffolded and
> verified locally; edge deploy is a manual one-liner.

---

## ⚡ Quick start

```bash
git clone https://github.com/jayaprakash2207/Portage-_MCP.git
cd Portage-_MCP
python -m venv .venv && . .venv/bin/activate        # Windows: .venv\Scripts\activate
pip install -e ".[dev]"

portage doctor                     # environment check
portage inspect jq                 # discovery + parsed IR + generated tools, as JSON
```

### Connect it to Claude

**Claude Code**

```bash
claude mcp add portage -- portage serve jq ripgrep curl git
```

**Claude Desktop** — add to `claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "portage": { "command": "portage", "args": ["serve", "jq", "ripgrep", "curl", "git"] }
  }
}
```

Restart the client. The generated tools appear in the tool list. With the default
config a tool call returns a **structured preview** (validation + authorization +
the exact `argv`) and runs nothing — enabling real execution is a deliberate,
per-CLI opt-in (see [Configuration](#-configuration)).

---

## 🧭 How it works

```mermaid
flowchart TD
    A["CLI on your machine"] -->|"help capture"| B["discovery"]
    A -.->|"man tool"| B
    B --> C["parser<br/>sections · usage · options<br/>commands · type inference"]
    C --> D["normalized CLI IR (CliProgram)"]
    D --> E["schema generator"]
    E --> F["MCP tool defs<br/>JSON Schema + arg_specs"]
    F --> G["stdio / HTTP MCP server"]
    G <--> H["AI assistant"]
    H -->|"tools/call"| I["safety pipeline"]
    I --> J["schema validation"]
    J --> K["per-value policy"]
    K --> L["allow-list — default deny"]
    L --> M["safe argv build — no shell"]
    M --> N["sandbox + rlimits + timeout"]
    N --> O["subprocess"]
    O --> P["structured result + audit event"]
```

Each stage is its own module and depends only on the shared data model — parsing,
schema generation, protocol handling and execution never import one another.

| Layer | Module | What it does |
|---|---|---|
| **Discovery** | [`discovery.py`](src/portage/discovery.py) | Resolve a CLI on `PATH`, capture `--help` → `-h` safely (timeouts, help-on-stderr, non-zero exit, truncation). |
| **Parse** | [`parser/`](src/portage/parser/) | Layered `--help` **+ man-page** parser → `CliProgram` IR. Reports `ParseConfidence`; keeps anything it can't classify as an `UnknownConstruct` instead of guessing. |
| **Merge** | [`merge.py`](src/portage/merge.py) | Deterministically fold the man-page IR into the `--help` IR — richer descriptions win, `--help` stays authoritative for structure. |
| **Schema** | [`schema.py`](src/portage/schema.py) | IR → draft-2020-12 JSON Schema (`additionalProperties: false`), deterministic names (`git remote add` → `git_remote_add`), collision-safe, plus `arg_specs` reconstruction metadata. |
| **Serve** | [`service.py`](src/portage/service.py) · [`server.py`](src/portage/server.py) | Transport-agnostic registry + an `mcp` 2.x adapter. stdio and streamable HTTP. |
| **Execute** | [`executor.py`](src/portage/executor.py) | Structured `argv` builder + `shell=False`, `stdin`-closed, timeout-bounded runner. POSIX `setrlimit`. |
| **Safety** | [`validation`](src/portage/validation.py) · [`value_policy`](src/portage/value_policy.py) · [`authorization`](src/portage/authorization.py) · [`sandbox`](src/portage/sandbox.py) · [`audit`](src/portage/audit.py) · [`pipeline`](src/portage/pipeline.py) | The one path from a tool call to a process. |

---

## ✨ Features

<table>
<tr>
<td width="50%" valign="top">

**Parsing that doesn't lie**
- GNU / POSIX / BSD option styles, `--opt=VAL`, `--opt[=WHEN]`, `--[no-]flag`
- enums from `{a,b,c}` / `<a|b|c>` / quoted "one of" lists
- repeatable options, documented defaults, `--arg NAME VALUE` (arity 2)
- positional args, variadics, nested subcommands
- `ParseConfidence` per program / command / option
- unclassifiable fragments preserved, never invented

</td>
<td width="50%" valign="top">

**Execution you can trust**
- no code path builds a command string — ever
- every value → distinct `argv` elements; flag & value never joined
- **default deny**: nothing runs without an explicit allow-list
- JSON Schema validation → per-value policy → allow-list → sandbox
- POSIX rlimits + wall-clock timeout + output cap
- structured audit events (argument *names*, never values)
- dry-run preview of the exact `argv`

</td>
</tr>
<tr>
<td width="50%" valign="top">

**Two documentation sources**
- `--help` first, `man <tool>` where available
- overstrike / ANSI cleanup, boilerplate-tail trimming
- deterministic merge; conflicts surfaced, not dropped

</td>
<td width="50%" valign="top">

**Local first, remote ready**
- stdio for Claude Desktop / Code
- `portage serve --http` → Starlette/uvicorn at `/mcp`
- Cloudflare Worker front door (`deploy/`), verified end-to-end via `wrangler dev`

</td>
</tr>
</table>

---

## 🔒 Security model

Portage lets an AI assistant run real commands, so the execution path is the
product. Every `tools/call` goes through, in order:

| # | Gate | Guarantee |
|---|---|---|
| 1 | **Schema validation** | Arguments checked against the generated draft-2020-12 schema. Unknown fields, wrong types, bad enums, missing required → structured rejection. |
| 2 | **Per-value policy** | Optional `value_rules`: max length, required / forbidden regex, "path must resolve under". Broken patterns fail closed. |
| 3 | **Allow-list — default deny** | Nothing runs unless a `CliConfig` sets `execution_enabled: true` **and** an `allowed_commands` prefix matches **and** every emitted flag is in `allowed_options`. An empty rule never matches. |
| 4 | **Safe `argv` construction** | Values become individual `argv` elements from `arg_specs`. No shell. No `--flag=value` joining. No string interpolation. Verified inert against `;` `\|` `&&` `$()` backticks newlines quotes redirection path-traversal. |
| 5 | **Sandbox** *(opt-in)* | Wrap in `bubblewrap` / `firejail` / `docker`: read-only root, private `/tmp`, no network by default. `mode: require` refuses to run if no launcher is present. |
| 6 | **Resource limits + timeout** | POSIX `setrlimit` (CPU / memory / file size / nproc); every run bounded and killed on overrun; output capped. |
| 7 | **Audit** | A structured `AuditEvent` per call — timestamp, tool, executable, command path, validation & authorization results, mode, exit code, duration. Argument **names only**. |

The executable is chosen by Portage from the tool definition and passed to the
engine as an absolute path — an MCP caller cannot select or redirect it.

<details>
<summary><b>What is <i>not</i> yet covered</b></summary>

- Container / restricted-user isolation on the deployed engine (the sandbox
  wrappers exist; a permitted command still runs as the engine's user).
- Cross-argument policy ("`--output` and the positional must share a dir").
- rlimits are POSIX-only; on Windows only the timeout + output cap apply.
- `--version`-style flags that bypass a required positional can't be modelled in
  JSON Schema, so such a call is rejected as missing-required.

Full list in [`TEST_REPORT.md`](TEST_REPORT.md).
</details>

---

## 🖥 CLI

| Command | Does |
|---|---|
| `portage doctor [tool]` | Environment check — interpreter, `mcp` SDK, `man`, optional CLI lookup. |
| `portage discover <tool>` | Capture the CLI's help text; print the structured `DiscoveryResult`. |
| `portage parse <tool>` | Discover + parse → the normalized `CliProgram` IR as JSON. |
| `portage generate <tool>` | Discover + parse → the generated MCP tool definitions + schemas. |
| `portage inspect <tool>` | **One-shot debug view:** discovery + man status + IR + tools + an optional dry-run. Nothing executes. |
| `portage call <tool> <cli> --json '{…}'` | Run one tool through the safety pipeline (dry-run unless `--execute` **and** policy permits). |
| `portage serve <cli>… [--config F] [--http]` | Run the MCP server (stdio, or streamable HTTP at `/mcp`). |

```console
$ portage inspect jq --no-man
{
  "discovery": { "status": "ok", "help": { "command_display": "jq --help", ... } },
  "ir": { "name": "jq", "confidence": "high", "global_options": [ ... ], "positionals": [{ "name": "filter", "required": true }] },
  "tools": [ { "name": "jq", "input_schema": { "type": "object", "additionalProperties": false, ... } } ]
}
```

---

## ⚙ Configuration

`portage serve --config portage.json`. Everything not listed stays denied.

```json
{
  "server_name": "portage",
  "clis": [
    {
      "name": "git",
      "use_man_page": true,
      "discover_subcommands": true,
      "subcommand_depth": 2,
      "policy": {
        "execution_enabled": true,
        "allowed_commands": [["git", "status"], ["git", "log"], ["git", "show"]],
        "allowed_options": ["--oneline", "--stat", "--short", "-n"],
        "timeout_seconds": 20,
        "value_rules": [{ "json_name": "max_count", "pattern": "\\d{1,4}" }],
        "resource_limits": { "cpu_seconds": 15, "memory_mb": 512, "max_processes": 64 },
        "sandbox": { "mode": "auto", "backend": "bubblewrap", "allow_network": false }
      }
    },
    { "name": "jq", "policy": { "execution_enabled": false } }
  ]
}
```

A ready-to-adapt version is in [`deploy/portage.example.json`](deploy/portage.example.json).

---

## 🌐 Remote deployment

Cloudflare Workers can't run native binaries, so the design is two parts:

```
MCP client ──HTTP──▶ Portage-Protocol (Cloudflare Worker: auth + reverse proxy)
                          │
                          ▼  HTTPS
                     Portage-Engine (this package: portage serve --http)
                          │
                          ▼
                     the real CLI
```

- **Engine:** anywhere that runs Python 3.10+ and the CLIs — `portage serve --config … --http --host 0.0.0.0 --port 8080`.
- **Worker:** [`deploy/cloudflare-worker/`](deploy/cloudflare-worker/) — type-checks with `tsc`, and the whole chain is verified locally (`test-local.ps1` / `test-local.sh`): health route, `401` without the bearer token, a real MCP `initialize` proxied through. Edge deploy: `npx wrangler secret put PORTAGE_ENGINE_TOKEN && npx wrangler deploy`.

Details in [`deploy/README.md`](deploy/README.md).

---

## 🧪 Development

```bash
ruff check .                     # lint
mypy                             # type-check (strict)
pytest -q                        # 360 unit tests — no network, no CLIs needed
pytest -q --run-integration -m integration   # + real jq/rg/curl/git/ffmpeg (+ bubblewrap)
pytest -q --cov=portage          # coverage
```

<p>
  <img alt="tests"     src="https://img.shields.io/badge/unit-360-2ea44f">
  <img alt="integ"     src="https://img.shields.io/badge/integration-16-2ea44f">
  <img alt="modules"   src="https://img.shields.io/badge/src%20modules-21-1f6feb">
  <img alt="coverage"  src="https://img.shields.io/badge/coverage-~91%25-2ea44f">
  <img alt="loc"       src="https://img.shields.io/badge/checked%20by-mypy%20--strict-1f6feb">
</p>

---

## 🗺 Roadmap

- [x] CLI discovery + `--help` capture
- [x] Layered `--help` parser → normalized IR
- [x] JSON Schema / MCP tool generation
- [x] stdio MCP server + tool discovery
- [x] Execution engine — structured `argv`, no shell
- [x] Safety layer — validation · value policy · **default-deny** allow-list · audit · dry-run
- [x] Man-page parsing + deterministic merge
- [x] End-to-end against 5 real CLIs, no per-tool code
- [x] POSIX resource limits + bubblewrap/firejail/docker sandbox (fail-closed)
- [x] Recursive subcommand discovery (`git remote add` → `git_remote_add`)
- [x] Streamable HTTP transport + Cloudflare Worker front door (verified locally)
- [x] Usage-line alternation decomposition (`[-p | --paginate | -P]` → options)
- [x] PyPI packaging (`python -m build` + `twine check` pass; trusted-publishing release workflow)
- [ ] `wrangler deploy` to the edge (needs a hosted Engine URL)
- [ ] Container / restricted-user isolation actually exercised on a Linux host
- [ ] `tbl`-formatted man tables; a live `docker`-backend sandbox test
- [ ] Cut the first GitHub Release → auto-publish to PyPI · submit to MCP registries

---

## Why "Portage"?

A *portage* is carrying a boat overland between two waterways — bridging things
that don't otherwise connect. Portage carries CLI functionality across into the
MCP waterway so AI assistants can use it.

## License

MIT — see [LICENSE](LICENSE). Contributions welcome; see
[CONTRIBUTING.md](CONTRIBUTING.md).
