Metadata-Version: 2.5
Name: leak-audit
Version: 0.1.0
Summary: Local-first AI agent that finds recurring charges, hidden fees and forgotten subscriptions in your email, bank statements and SMS.
License: MIT
License-File: LICENSE
Keywords: agent,gemini,india,llm,personal-finance,privacy,subscriptions
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Office/Business :: Financial
Requires-Python: >=3.12
Requires-Dist: defusedxml>=0.7.1
Requires-Dist: fastapi>=0.143.3
Requires-Dist: google-api-python-client>=2.201.0
Requires-Dist: google-auth-oauthlib>=1.5.0
Requires-Dist: google-genai>=1.0
Requires-Dist: pdfplumber>=0.11.10
Requires-Dist: pikepdf>=10.17.0
Requires-Dist: pydantic>=2.7
Requires-Dist: python-dotenv>=1.0
Requires-Dist: pyyaml>=6.0.3
Requires-Dist: rich>=15.0.0
Requires-Dist: sqlite-utils>=4.2.1
Requires-Dist: typer>=0.12
Requires-Dist: uvicorn>=0.54.0
Description-Content-Type: text/markdown

# Leak

**An open-source, local-first AI agent that reads your email, bank statements and SMS alerts, finds the money quietly leaking out — forgotten subscriptions, price hikes, trials that turned paid, duplicates, charges after you cancelled, forex fees — and drafts the cancellation emails for you to approve.**

India-first (₹, HDFC/ICICI/SBI/Axis, UPI autopay, bank SMS), region-agnostic by design. The agent harness is built by hand — no LangChain, LlamaIndex or CrewAI — and is meant to be read.

[![Leak: privacy masking, findings, evidence, approvals — click for the 49-second walkthrough](docs/assets/demo.gif)](docs/assets/leak-demo.mp4)

![₹24,635/year found on the synthetic demo inbox](docs/assets/dashboard.png)

<sub>All screenshots and the video use the synthetic demo dataset (made-up persona, accounts and numbers).</sub>

```text
$ uvx leak-audit demo
 1  cult.fit       zombie: charged 3× after you cancelled     ₹17,988/yr   dispute
 2  JioHotstar     duplicate: already in your Airtel Black     ₹3,588/yr   cancel
 3  Google One     duplicate: billed on two accounts           ₹1,560/yr   cancel
 4  Amazon Prime   duplicate: already in your Airtel Black     ₹1,499/yr   cancel
 …  Netflix price hike (+₹1,800/yr), ChatGPT forex fees (₹867/yr), recurring subscriptions to review

₹24,635/year in potential savings — every row cites the statement line, email or SMS behind it.
```

---

## A quick tour

| Findings, with the savings number | Evidence drawer: the exact source lines |
|---|---|
| ![Findings](docs/assets/dashboard.png) | ![Evidence](docs/assets/evidence.png) |
| **Approvals: drafts are queued, never sent** | **What the LLM saw: post-redaction payloads** |
| ![Approvals](docs/assets/approvals.png) | ![What the LLM saw](docs/assets/llm.png) |

## Privacy & safety

Leak is built on the assumption that your financial data is nobody's business.

- **Local-first.** Everything — documents, ledger, findings, traces — lives in one SQLite file under `.leak/` on your machine. There is no Leak server and no telemetry.
- **Redacted before any LLM sees it.** PAN, Aadhaar (Verhoeff-checked), card numbers (Luhn-checked, full or masked), account numbers, phone numbers, emails, personal UPI IDs, addresses and your own names are replaced with tokens like `[CARD_2]` at ingestion. The token → value map stays in a local vault (`chmod 600`) so *your* dashboard can unmask.
- **Fail-closed egress guard.** Every payload is redacted again and re-scanned right before it leaves the process; if anything still looks like PII, the call is refused. A test plants a tool that leaks raw text on purpose and asserts nothing reaches the model ([`tests/test_privacy_egress.py`](tests/test_privacy_egress.py)).
- **See exactly what the model saw.** The trace records the post-guard payload of every step; the dashboard's *What the LLM saw* page shows it verbatim.
- **No send tool.** The agent can *queue* a Gmail draft or a calendar reminder; nothing happens until you press Approve. There is no code in this repository that sends email. Gmail access is read-only unless you opt in to `gmail.compose` (drafts).
- **Grounded findings only.** A finding is accepted only if every evidence id it cites exists and its amount appears in the cited text.
- **Bring your own OAuth client.** Leak ships no Google client ID, so no third party ever holds a token to your inbox.
- **Local dashboard hardening.** Binds to `127.0.0.1`, rejects non-localhost `Host` headers (DNS rebinding), sends no CORS headers, and requires a per-process token for approvals and unmasking.

Statements never leave unredacted either: for banks without a parser, Leak sends the *redacted text* of the statement, never the PDF. Scanned (image-only) PDFs are refused rather than uploaded.

## Quickstart

### Try it on synthetic data (one command)

```bash
uvx leak-audit demo
```

(Until the package is on PyPI: clone the repo and run `uv run leak demo`.)

Scans a synthetic inbox, four password-protected bank/card statements and an SMS backup, extracts receipts (replayed from a recorded cassette: ₹0, no key needed), runs the detectors, prints the report and opens the dashboard. With `GEMINI_API_KEY` set, the Pro-model auditor agent also reviews every finding (~₹30).

### Your own data

```bash
git clone https://github.com/<you>/leak && cd leak
uv sync
cp config.example.toml config.toml        # model ids, budgets, your names (always masked)
echo 'GEMINI_API_KEY=...' > .env          # https://aistudio.google.com/apikey
(cd web && npm install && npm run build)  # dashboard, once

uv run leak gmail auth                                   # see "Gmail setup" below
uv run leak scan --gmail                                 # add --limit 100 to sample first
uv run leak scan --statements ~/Downloads/statements     # PDF passwords: LEAK_PDF_PASSWORDS="a,b" in .env
uv run leak scan --sms ~/Downloads/sms-backup.xml        # Android "SMS Backup & Restore" export
uv run leak extract                                      # Flash: receipts, renewals, price changes (cached)
uv run leak report                                       # detectors + Pro auditor → ranked report
uv run leak serve                                        # dashboard on http://127.0.0.1:8765
```

<details>
<summary><b>Gmail setup (bring your own OAuth client, ~5 minutes)</b></summary>

1. In [Google Cloud Console](https://console.cloud.google.com/) create a project and enable the **Gmail API**.
2. **Google Auth Platform → Branding**: fill in an app name and your email. **Audience**: *External*, keep it in **Testing**, and add your Gmail address under **Test users**. (You do not need Google verification for your own use.)
3. **Clients → Create client → Desktop app**, download the JSON to `~/.config/leak/client_secret.json`.
4. `uv run leak gmail auth`. Google will warn the app is unverified — it's yours; choose *Advanced → Go to …*.
5. To let Leak create drafts, set `[gmail] enable_actions = true` and run `leak gmail auth` again (adds `gmail.compose`).

Testing-mode tokens expire after 7 days; Leak notices and asks you to log in again. Google Workspace users can pick *Internal* instead and skip both the test-user list and the expiry.
</details>

## How the harness works

```mermaid
flowchart LR
  subgraph Ingest["Ingest (local)"]
    G[Gmail] --> R
    P[PDF statements<br/>pikepdf + pdfplumber<br/>bank plugins] --> R
    S[SMS backup XML] --> R
    R[Redactor<br/>PII → tokens<br/>vault stays local] --> DB[(SQLite)]
  end
  DB --> X[Extractor<br/>SMS rules · Flash batches<br/>content-hash cache]
  X --> L[Ledger<br/>merchant catalog · FX<br/>cross-source dedupe]
  L --> D[Detectors<br/>recurring · price hike · trial<br/>duplicate · zombie · forex]
  D --> A
  subgraph A["Auditor agent (leak/harness)"]
    direction TB
    LOOP[loop.py<br/>plan → act → observe] --> T[tools.py<br/>registry + schemas]
    LOOP --> PERM[permissions.py<br/>auto · needs_approval · forbidden]
    LOOP --> B[budget.py<br/>steps · $ · ₹]
    LOOP --> C[context.py<br/>compaction]
    LOOP --> TR[trace.py<br/>JSONL + replay]
    LOOP --> GR[grounding.py<br/>evidence checks]
    LOOP -. egress guard .-> LLM[(Gemini Pro)]
  end
  A --> F[Findings]
  A --> Q[Approval queue]
  Q -- you approve --> ACT[Gmail draft · .ics reminder]
  F --> UI[Dashboard]
  Q --> UI
```

The whole agent is one loop ([`leak/harness/loop.py`](leak/harness/loop.py)):

```text
while True:
    budget.start_step()                       # step ceiling → BudgetExceeded
    payload = egress_guard(context)           # redact + re-scan; refuse on PII
    trace("llm_request", payload)             # exactly what left the machine
    response = llm.generate(payload, tools)
    budget.charge(response.usage)             # $ / ₹ ceiling → BudgetExceeded
    if not response.tool_calls: return response.text
    for call in response.tool_calls:
        auto           → validate args, run, feed result back
        needs_approval → enqueue for a human, tell the model it did NOT run
        forbidden      → never shown to the model; refused if called anyway
    context.compact()
```

| Component | File | What it teaches |
|---|---|---|
| Loop | [`loop.py`](leak/harness/loop.py) | Plan → tool call → observe; where every safety property is enforced in code |
| Tools | [`tools.py`](leak/harness/tools.py) | A Python function + Pydantic model becomes an LLM tool; JSON schema generated, args validated, errors fed back instead of crashing |
| Permissions | [`permissions.py`](leak/harness/permissions.py) | `auto` / `needs_approval` / `forbidden`; config can tighten, never loosen; approval queue |
| Budget | [`budget.py`](leak/harness/budget.py) | Step + cost ceilings from real token usage (cached tokens priced separately), shown in $ and ₹ |
| Context | [`context.py`](leak/harness/context.py) | What goes into the prompt each step; compaction of old tool output |
| Trace | [`trace.py`](leak/harness/trace.py) | Append-only JSONL flight recorder; `leak trace replay` pretty-prints, `ReplayProvider` re-runs a loop deterministically for $0 |
| Grounding | [`grounding.py`](leak/harness/grounding.py) | Claims must cite evidence that exists and supports them; rejections go back to the model to fix |

`leak/harness/` is 833 lines, 543 of them code — the rest is comments written for whoever reads it next. It has no dependency on Gemini: providers implement a five-line `LLMProvider` protocol ([`leak/llm/provider.py`](leak/llm/provider.py)), and tests run the whole thing against a scripted fake model. `mypy --strict` clean.

Model ids are config, not code (`config.example.toml`): extraction on `gemini-3.5-flash-lite`, the auditor on `gemini-3.1-pro-preview` at the time of writing.

## Evals

`uv run python evals/run_evals.py` — runs offline by replaying a recorded cassette (no key, $0), and in CI on every push.

| Metric | Result | Set |
|---|---|---|
| Extraction F1 (charges, renewals, price changes, cancellations) | **1.000** (42/42, 0 FP) | 22 emails + 21 SMS, labels generated with the data |
| Detection precision / recall | **1.000 / 1.000** | 18 planted findings incl. zombie, trial→paid, bundle duplicate, forex |
| Cross-source dedupe: missed merges | **0** | 155 ledger entries, 27 confirmed by 2–3 sources |
| SMS handled by deterministic rules | 21 / 21 | no LLM call |

| Cost (live Gemini, measured) | Tokens | Cost |
|---|---|---|
| Extraction, first run (22 emails → 3 Flash-Lite calls) | 11.3k in / 7.6k out | **$0.022 · ₹1.96** (≈ ₹0.09/email) |
| Extraction, any re-run | — | **₹0** (content-hash cache) |
| Bank statements (4 PDFs, 153 txns) | — | **₹0** (deterministic parsers) |
| Auditor agent, one `leak report` (20 steps, Pro) | 292k in (67% cached) / 8k out | **$0.33 · ₹29** |

These fixtures are synthetic and small, and were written alongside the parsers and prompt — read the scores as "no regressions", not "works on every inbox". Harder, independently written fixtures are the most valuable contribution you can make; see [`evals/README.md`](evals/README.md).

## Supported sources

| Source | How | Notes |
|---|---|---|
| HDFC Bank savings | parser `hdfc_savings` | direction from running balance |
| Axis Bank savings | parser `axis_savings` | |
| ICICI Bank credit card | parser `icici_credit_card` | foreign-currency column → forex detector |
| SBI Card | parser `sbi_card` | |
| Any other bank | LLM fallback on **redacted** text | rows must cite a line that contains their amount |
| Password-protected PDFs | `pikepdf`, decrypted in memory only | passwords via `LEAK_PDF_PASSWORDS` |
| Gmail | read-only API, incremental sync | drafts only with opt-in `gmail.compose` |
| Android SMS | "SMS Backup & Restore" XML | DLT senders only; OTPs and personal chats dropped before storage |

**Add your bank:** most parsers are ~20 lines — see [`docs/adding-a-bank.md`](docs/adding-a-bank.md). The merchant catalog ([`leak/ledger/merchants.yaml`](leak/ledger/merchants.yaml)) takes PRs too: aliases, typical prices, cancel links, and what telecom/credit-card bundles include.

## CLI

| Command | |
|---|---|
| `leak demo` | Whole pipeline on synthetic data, then the dashboard |
| `leak scan [--gmail] [--statements PATH] [--sms FILE] [--limit N]` | Ingest + redact into the local store |
| `leak extract` | Receipts, renewals, price changes, cancellations → ledger |
| `leak ledger [--merchant netflix]` | The normalized, de-duplicated ledger |
| `leak report [--no-agent]` | Detectors + auditor agent → ranked findings |
| `leak serve` | Dashboard: findings + evidence drawer, approvals, agent runs, what the LLM saw |
| `leak actions list \| approve ID \| reject ID` | The approval queue from the terminal |
| `leak trace list \| replay RUN \| rerun RUN` | Inspect or deterministically re-run any agent run |

## Development

```bash
uv sync && uv run pytest -q          # offline, no API key
uv run ruff check . && uv run mypy leak
uv run python evals/run_evals.py     # add --live to record new model output
(cd web && npm run dev)              # dashboard with hot reload, proxies /api to `leak serve`
(cd video && npm install && npm run shots && npm run render)   # screenshots + Remotion video, see video/README.md
uv run python demo/generate_sample_data.py   # regenerate synthetic data + labels
```

## Roadmap

- **Account Aggregator** (RBI AA framework) as a consented, structured alternative to PDFs and SMS
- **More banks**: Kotak, Yes, IDFC FIRST, IndusInd, AU, HSBC, Amex India, OneCard; Paytm/PhonePe wallet exports
- **Ollama-only mode**: same `LLMProvider` protocol, fully offline extraction and auditing
- **WhatsApp / Telegram bot** for the monthly "here's what leaked" summary and one-tap approvals
- **"Unused" detector**: no product-usage emails in 90 days for a paid service
- OCR for scanned statements (locally), encrypted vault (OS keychain), UPI mandate list import

## License

MIT — see [LICENSE](LICENSE). Security reports: [SECURITY.md](SECURITY.md).
