Metadata-Version: 2.4
Name: whisper-misp
Version: 0.0.1
Summary: Whisper enrichment module for MISP (misp-modules expansion/hover).
Author: Whisper Security
License: MIT
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pymisp<3,>=2.5
Requires-Dist: requests>=2.32
Dynamic: license-file

# MISP Whisper Module

Enrichment module for [MISP](https://www.misp-project.org/) built on the
[misp-modules](https://github.com/MISP/misp-modules) framework. Click
**Add enrichment** on an IP, domain, hostname, or AS attribute and the
module pulls the DNS, WHOIS, BGP, and threat context Whisper holds for it,
then returns it as MISP attributes and objects your analysts can pivot on.
Hovering over a supported attribute shows a compact live threat verdict.

Whisper is the internet's infrastructure graph: DNS, BGP, WHOIS, hosting,
and threat intel pre-joined into one queryable map. The module brings that
graph to the attribute you're looking at.

## Table of Contents

- [Introduction](#introduction)
- [Installation](#installation)
- [Configuration](#configuration)
- [Usage](#usage)
- [Behavior](#behavior)
  - [Supported Attribute Types](#supported-attribute-types)
  - [Expansion vs Hover](#expansion-vs-hover)
  - [Data Flow](#data-flow)
  - [Enrichment Mapping](#enrichment-mapping)
  - [Performance budgets](#performance-budgets)
- [Local development](#local-development)
- [Agent-activity log source (second tier)](#agent-activity-log-source-second-tier)
- [Additional Information](#additional-information)

## Introduction

The module is a standard misp-modules **expansion + hover** module speaking
the `misp_standard` format. For each enrichment request it runs scoped
Cypher queries against the WhisperGraph API, translates the results into
MISP attributes, objects (`asn`, `domain-ip`), and object references, and
returns them for MISP to ingest onto the event. Re-enrichment is
idempotent: output UUIDs are UUIDv5s derived from Whisper node identities,
so running the same enrichment twice updates instead of duplicating.

## Installation

The module ships as one module file plus two support packages
(`whisper_core`, `whisper_misp`).

**Container deployments (misp-docker):** mount the module file into the
misp-modules container's custom-module hook and install the packages into
its interpreter:

```yaml
# docker-compose override for the misp-modules service
volumes:
  - "./whisper-misp/modules/expansion/:/custom/expansion/:Z"
  - "./whisper-misp/whisper_core/:/custom/whisper_core/:Z"
  - "./whisper-misp/whisper_misp/:/custom/whisper_misp/:Z"
environment:
  PYTHONPATH: "/custom"
```

**Native misp-modules installs:** install the packages and copy the module
file next to the other expansion modules:

```bash
pip install git+https://github.com/whisper-sec/whisper-misp.git
cp modules/expansion/whisper.py <misp-modules>/misp_modules/modules/expansion/
```

Restart misp-modules afterwards; `curl -s localhost:6666/modules` should
list `whisper`.

## Configuration

All settings live in the MISP UI under **Administration → Server Settings →
Plugin → Enrichment** (or via `cake Admin setSetting`):

| Setting | Required | Default | Description |
| --- | --- | --- | --- |
| `Plugin.Enrichment_whisper_enabled` | Yes | `false` | Enable the module. |
| `Plugin.Enrichment_whisper_api_key` | Yes | – | Your Whisper API key, sent as `X-API-Key`. Never logged. |
| `Plugin.Enrichment_whisper_api_url` | No | `https://graph.whisper.security` | WhisperGraph API base URL. |
| `Plugin.Enrichment_whisper_max_tlp` | No | `tlp:amber+strict` | TLP ceiling — attributes tagged above it are refused before any value leaves MISP. `tlp:red` disables the gate. |
| `Plugin.Enrichment_whisper_timeout` | No | `8` | Wall-clock budget in seconds for the enrichment flows. Keep it below MISP's `Plugin.Enrichment_timeout`. |

Also enable the framework plumbing once: `Plugin.Enrichment_services_enable`,
`Plugin.Enrichment_hover_enable`, and point `Plugin.Enrichment_services_url` /
`_port` at your misp-modules instance.

## Usage

- **Expansion**: open an event, click the enrichment icon on a supported
  attribute (or *Enrich Event*), pick **Whisper**, review, ingest.
- **Hover**: point at a supported attribute value — the popover shows the
  live Whisper threat verdict (score, level, flags, feed listings).
- **API**: `POST /events/enrichEvent/<id>` with `{"whisper": 1}` enriches
  every supported attribute on the event through the same machinery.

## Behavior

### Supported Attribute Types

| MISP attribute | Whisper anchor |
| --- | --- |
| `ip-src`, `ip-dst` | `IPV4` / `IPV6` |
| `domain`, `hostname` | `HOSTNAME` |
| `AS` | `ASN` (values normalized: `15169`, `as15169`, `AS015169` → `AS15169`) |

Single-address CIDR values (`/32`, `/128`) collapse to the bare IP; wider
ranges return an informative note instead of an error.

### Expansion vs Hover

MISP's hover endpoint sends no `event_id`; the expansion flow always does.
The module branches on that: hover runs a **single** threat-context query
(5-second cap by default; a configured `timeout` overrides it) and returns
one compact verdict note, while expansion runs the full multi-query pull. ASN and CIDR hovers answer without any API call.
The TLP gate applies to both — a hover is still egress.

### Data Flow

For an expansion request the module:

1. Enforces the TLP ceiling, then builds a Whisper client (no in-request
   retries — every flow is budgeted instead).
2. Runs the per-type queries (see mapping below), each with an HTTP
   timeout capped at the remaining wall-clock budget.
3. Translates results into MISP output via a deterministic converter and
   returns `{"results": {"Attribute": [...], "Object": [...]}}`.
4. Anything the budget cut is named in an analyst-visible truncation note;
   content-free results return an honest status note instead of echoing
   the seed.

Per type: **IP** runs the one-hop neighbourhood plus threat-context and
network-context (announcing ASN, prefix, BGP flags) queries. **Domain**
runs up to 19 queries across prioritized categories — DNS direct facts,
threat evidence, capped pivots (nameserver-for / mail-server-for /
subdomains / inbound CNAMEs, 25 each with overflow notes), web links both
directions, SPF policy, WHOIS phones, and registered-lookalike detection.
**AS** runs the one-hop neighbourhood.

### Enrichment Mapping

| Whisper data | MISP output |
| --- | --- |
| Domain + its resolved IPs | `domain-ip` object (reference → seed) |
| ASN (+ human-readable name) | `asn` object with `description` (reference → seed, relationship from the Whisper edge, e.g. `announced-by`) |
| Related IPs | `ip-dst` attribute |
| Related domains/hostnames | `domain` attribute |
| WHOIS emails | `whois-registrant-email` attribute |
| Registrar (`HAS_REGISTRAR`/`PREV_REGISTRAR`) | `whois-registrar` attribute |
| Registrant org (`REGISTERED_BY`) | `whois-registrant-org` attribute |
| Other organizations | `text` attribute |
| Threat/network/SPF/WHOIS-phone/lookalike/overflow context | `text` attributes with the category as the comment |

Attributes can't carry object references in MISP, so for attribute-mapped
neighbours the Whisper edge type lands in the attribute **comment**
(`Whisper: nameserver-for-domain`, `Whisper: links-to-inbound`, …) —
nothing is lost, analysts can filter on it.

### Performance budgets

MISP enrichment is synchronous (MISP kills module calls at
`Plugin.Enrichment_timeout`, default 10s), so every flow runs under a
wall-clock budget: each query carries a per-call HTTP timeout equal to the
remaining budget and in-request retries are disabled. In the IP and domain
flows, whatever the budget cuts is reported in an analyst-visible
truncation note instead of failing the enrichment; the single-query AS and
hover flows surface an overrun as a clean transport error.

| Flow | Queries | Budget (default) | Overrun behavior |
| --- | --- | --- | --- |
| hover (any type) | ≤ 1 | 5s per call (or `timeout` if set) | transport error |
| `ip-src`/`ip-dst` | ≤ 3 | 8s wall clock | supplements skipped + truncation note |
| `domain`/`hostname` | ≤ 19 | 8s wall clock | categories skipped + truncation note |
| `AS` | 1 | 8s per call | error via transport mapping |

Typical live latencies (p95 over 10 runs against the production API):
hover 0.06s, IP 0.12s, AS 0.05s, domain 1.16s.

## Local development

See [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) for the Docker dev stack
(MISP + misp-modules + Traefik with the module hot-mounted), the test
layers (`make test`, `make test-integration`, `make e2e`), and the
CI setup including the MISP/misp-modules upstream-parity workflow.

```
whisper-misp/
├── modules/expansion/whisper.py   # the misp-modules module (handler/introspection/version)
├── whisper_core/                  # shared Whisper client, Cypher builders, result parser
├── whisper_misp/                  # MISP output layer: converter, enrichment flows, TLP gate
├── tests/                         # unit suite + integration/perf suites (real API)
├── scripts/                       # e2e_setup.sh + e2e_misp.py (full analyst-loop E2E)
└── docs/website/misp/             # website-ready documentation set
```

## Agent-activity log source (second tier)

Alongside enrichment, the repo ships a second, keyed integration tier:
`scripts/agent_logs_push.py`, a single-shot cron poller that pulls your
Whisper agents' activity logs (DNS lookups, egress connections, identity
allocations) from the control plane and files them into a rolling
per-UTC-day MISP event. Closed connections land as `network-connection`
objects, answered DNS as `passive-dns` objects, unanswered DNS as
`domain` attributes tagged `whisper:dns-decision=<decision>`, and
alloc/gap notes as `text` — everything `to_ids=False` with deterministic
UUIDs, so re-runs converge instead of duplicating. The enrichment module
is untouched. Full page:
[docs/website/misp/agent-activity.md](docs/website/misp/agent-activity.md).

## Additional Information

- Architecture deep dive: [docs/architecture.md](docs/architecture.md)
- Website documentation drafts: [docs/website/misp/](docs/website/misp/)
- Known limitations: URL and file-hash attributes are out of scope
  (Whisper has no direct anchor for them); location data (country/city) is
  not yet emitted; large ASNs' one-hop neighbourhoods are dominated by
  routing records and may return only a status note.
- License: [MIT](LICENSE).
