Metadata-Version: 2.4
Name: onsense
Version: 0.3.0
Summary: Turn your Android phone into a camera and sensor source for PC AI agents (onSense MCP broker)
Author: shdev
License: MIT
License-File: LICENSE
Keywords: ai-agent,android,camera,claude,mcp
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Multimedia :: Video :: Capture
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Utilities
Requires-Python: >=3.10
Requires-Dist: cryptography>=42
Requires-Dist: httpx>=0.27
Requires-Dist: mcp>=1.2.0
Requires-Dist: qrcode>=7.4
Requires-Dist: zeroconf>=0.131
Provides-Extra: test
Requires-Dist: pytest>=7; extra == 'test'
Description-Content-Type: text/markdown

# onsense — Phone Camera & Sensors for PC AI Agents

**onsense** is the PC-side MCP broker for [onSense](https://play.google.com/store/apps/details?id=com.shdev.onsense).
Install the Android app on your phone, run `uvx onsense pair` on your PC, and your AI client (Claude, Cursor, or any MCP-compatible tool) can see through your phone's camera and read its sensors — all over local Wi-Fi, with no cloud relay.

---

## Architecture

```text
Android phone (onSense app)           PC (this package)           AI client
  HTTP provider :8080        ←→    stdio MCP broker          Claude / Cursor / …
  camera frames, photos,           onsense serve               natural-language or
  sensors, QR pairing              onsense clip :8770          /eye tool calls
```

- The **phone** runs an HTTP provider on port 8080. It exposes camera frames, recent photos, and sensor readings, all gated by a pairing token.
- The **PC package** (`onsense`) is a stdio MCP broker. It translates MCP tool calls from your AI client into HTTP requests to the phone.
- Discovery uses mDNS (`_onsense._tcp.local.`). If the phone's IP changes, the broker rediscovers it automatically — no manual reconfiguration needed.
- An optional **clip bridge** daemon on port 8770 lets the phone push camera frames, files, and clipboard content to the PC (and, if enabled, lets the phone pull from the PC clipboard).

---

## Requirements

- Python 3.10+ and [uv](https://docs.astral.sh/uv/) (provides `uvx`)
- The onSense Android app installed on a phone on the **same local Wi-Fi network** as your PC
- An MCP-compatible AI client such as [Claude Code](https://claude.ai/code) or Cursor

---

## Quick Start

### 1. Pair your phone

Run this once. The PC prints a QR code; scan it from the onSense app ("Scan PC QR"). The phone pushes its address and token to the PC, which registers the MCP server automatically.

```bash
uvx onsense pair
```

After pairing, restart your AI client once so it picks up the new `onsense` MCP server.

### 2. Use it

Ask your AI client naturally:

> "Take a photo of what's in front of my phone."
> "What are the current sensor readings?"
> "Show me the last 5 photos on my phone."

The MCP tools are called automatically. You can also invoke them directly as `/eye` or whichever slash command your client supports.

### 3. Diagnose problems

```bash
uvx onsense doctor           # checks Python, uv, MCP, Claude registration, mDNS, phone reachability
uvx onsense doctor --base http://192.168.1.5:8080 --token YOUR_PAIRING_TOKEN
```

---

## Subcommands

| Command | What it does |
| --- | --- |
| `uvx onsense pair` | Display a QR code on the PC; phone scans it and pushes `{base, token}`; registers MCP automatically |
| `uvx onsense pair "onsense://pair?base=...&token=..."` | Parse a pairing URI directly (phone-displayed QR → manual copy) |
| `uvx onsense pair --img screenshot.png` | Decode a pairing QR from a screenshot file (requires opencv) |
| `uvx onsense serve` | Run the MCP server (stdio). Also starts the clip daemon automatically unless `--no-clip` is passed |
| `uvx onsense serve --no-clip` | Run the MCP server only, without starting the clip daemon |
| `uvx onsense clip` | Run the clip bridge daemon standalone (port 8770) |
| `uvx onsense clip --allow-pull` | Enable phone→pull: phone can GET /clip to retrieve the PC clipboard |
| `uvx onsense clip --set-clipboard` | Auto-inject received content into the PC OS clipboard (off by default) |
| `uvx onsense doctor` | Diagnose installation, connectivity, and phone reachability |

---

## Other MCP clients (Claude Desktop, Cursor, Codex)

`uvx onsense pair` auto-registers the server with **Claude Code** (via `claude mcp add`). The MCP server itself is standard stdio MCP, so any MCP-compatible client works once configured — only the auto-registration is Claude Code-specific. For other clients, run `uvx onsense pair` once to obtain your phone's `PHONE_BASE` / `PHONE_TOKEN` (also saved to `~/.onsense/pair.json`), then add the server manually:

**Claude Desktop / Cursor** — add to the client's MCP config (`claude_desktop_config.json` or `.cursor/mcp.json`):

```json
{
  "mcpServers": {
    "onsense": {
      "command": "uvx",
      "args": ["onsense", "serve"],
      "env": { "PHONE_BASE": "http://<phone-ip>:8080", "PHONE_TOKEN": "<token>" }
    }
  }
}
```

**Codex** — add to `~/.codex/config.toml`:

```toml
[mcp_servers.onsense]
command = "uvx"
args = ["onsense", "serve"]
env = { PHONE_BASE = "http://<phone-ip>:8080", PHONE_TOKEN = "<token>" }
```

Restart the client once after adding the server.

---

## MCP Tools

These tools are exposed to your AI client after pairing:

| Tool | Description |
| --- | --- |
| `get_live_frame()` | Capture the current camera frame from the phone (returns JPEG image) |
| `read_sensors()` | Return phone sensor readings as JSON: battery level/charging state, ambient light (lux), accelerometer (x/y/z) |
| `recent_photos(limit=10)` | List recent photos on the phone: id, name, date_added, size, width, height |
| `get_photo(id, max_width=1024)` | Fetch a specific photo by id (downscaled to max_width); use ids from `recent_photos` |

If the phone's IP changes, the broker retries using mDNS autodiscovery before surfacing an error.

---

## Clip Bridge (Phone ↔ PC File & Clipboard)

`onsense serve` automatically starts a clip daemon on **port 8770**. You can also run it standalone with `onsense clip`.

### Phone → PC push (POST /clip)

The onSense Android app can push content to the PC:

- **Images** are saved to disk as `latest.jpg` (in `ONSENSE_CLIP_DIR`, default `<tempdir>/onsense/`).
- **Text files** are saved to disk.
- **Other files** (video, PDF, etc.) are saved to disk by filename.
- If `--set-clipboard` is active, images and text are also injected into the PC OS clipboard so you can paste with Ctrl+V immediately.

By default, `--set-clipboard` is **off** — files are saved to disk but the clipboard is not touched.

### PC → Phone pull (GET /clip)

Off by default. Enable with:

```bash
uvx onsense clip --allow-pull
```

When enabled, the phone can GET /clip to retrieve the current PC clipboard content (copied files first, then images, then text). Returns 204 if the clipboard is empty.

### Ports and environment variables

| Variable | Default | Purpose |
| --- | --- | --- |
| `PHONE_BASE` | (from pairing) | Phone HTTP base URL, e.g. `http://192.168.1.5:8080` |
| `PHONE_TOKEN` | (from pairing) | Pairing auth token |
| `ONSENSE_CLIP_DIR` | `<tempdir>/onsense` | Directory where pushed files are saved |
| `ONSENSE_CLIP_MAX_MB` | `200` | Maximum incoming file size in MB (0 = unlimited) |
| `ONSENSE_CLIP_ALLOW_PULL` | `0` | Set to `1` to enable PC→phone pull without the CLI flag |
| `ONSENSE_CLIP_SET_CLIPBOARD` | `0` | Set to `1` to auto-inject into OS clipboard without the CLI flag |
| `ONSENSE_TEST_FRAME` | (unset) | Local JPEG path returned by `get_live_frame` when phone is unreachable |

---

## Security

**Local network only.** The PC-side services bind to all interfaces but reject connections from non-private IP addresses at the application layer. They are not intended to be exposed to the internet.

**HMAC request signing.** The pairing token is never sent in cleartext. Every authenticated request is signed with HMAC-SHA256 over `METHOD\npath\ntimestamp\nnonce\nalgorithm`, keyed by a signing key derived from the pairing token via HKDF-SHA256 (a key separate from the encryption key), and carries `X-Ts` / `X-Nonce` / `X-Sig` / `X-Enc` headers. Servers verify in constant time, reject timestamps outside a ±300 s window, and reject reused nonces — so a sniffed request cannot be replayed and the token cannot be stolen off the wire. New Android installs generate a 128-bit random token, stored on the phone and in `~/.onsense/pair.json` (chmod 600) after pairing. (The `/health` endpoint is unauthenticated.) See [PROTOCOL.md](PROTOCOL.md) for the exact wire format.

**Pull and clipboard injection are off by default.** `GET /clip` (phone pulls PC clipboard) and OS clipboard auto-injection (phone push → Ctrl+V) are disabled unless you explicitly pass `--allow-pull` or `--set-clipboard`.

**Encrypted bodies (AES-256-GCM).** Sensitive payloads — camera frames, photos, sensor data, files, and clipboard content — are encrypted end-to-end with AES-256-GCM. The key is derived from the pairing token via HKDF-SHA256 (a key separate from the signing key); each message uses a fresh 96-bit nonce, and the GCM tag authenticates the body and binds it to its request (so tampering or response substitution is rejected). A passive sniffer on the same Wi-Fi sees only ciphertext. Low-sensitivity metadata stays plaintext: the open `/version` and `/health` endpoints, HTTP error responses, and the saved-file path returned after a push. (Note: large transfers currently buffer the body in memory while encrypting/decrypting — chunked streaming for very large files is a future step. Transport-level TLS is also a possible future hardening.)

**File size cap.** Incoming pushes are rejected if they exceed `ONSENSE_CLIP_MAX_MB` (default 200 MB). Set to `0` to remove the cap.

---

## License

MIT. See [LICENSE](LICENSE).

---

## 한국어 요약

**onsense**는 Android 폰(onSense 앱)을 PC AI 에이전트의 카메라·센서로 연결하는 PC측 MCP 브로커입니다.

- `uvx onsense pair` — 폰과 한 번 페어링하면 Claude·Cursor에 MCP 서버가 자동 등록됩니다.
- `uvx onsense doctor` — 연결·설치 문제를 진단합니다.
- `uvx onsense serve` — AI 클라이언트가 호출하는 MCP 서버(stdio)를 실행합니다. 동시에 클립 브리지 데몬(:8770)도 자동 기동합니다.
- `uvx onsense clip` — 폰↔PC 파일·클립보드 브리지를 단독으로 실행합니다.

폰과 PC는 **같은 로컬 Wi-Fi**에 있어야 하며, 개발자 운영 클라우드 서버는 없습니다. 통신은 현재 로컬 HTTP(비암호화)이므로 신뢰할 수 있는 네트워크에서 사용하세요.
