Metadata-Version: 2.4
Name: tsume
Version: 0.1.0
Summary: A security-first toolkit for the full first-response threat intel workflow - triage, monitor, detonate.
Project-URL: Homepage, https://github.com/iamavu/tsume
Project-URL: Repository, https://github.com/iamavu/tsume
Author: Avantika
License-Expression: MIT
Keywords: cli,ioc,security,threat-intel
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Security
Requires-Python: >=3.11
Requires-Dist: keyring>=24.3
Requires-Dist: pydantic>=2.5
Requires-Dist: requests>=2.31
Requires-Dist: rich>=13.7
Requires-Dist: typer>=0.12
Provides-Extra: all
Requires-Dist: fastapi>=0.111; extra == 'all'
Requires-Dist: fpdf2>=2.7; extra == 'all'
Requires-Dist: jinja2>=3.1; extra == 'all'
Requires-Dist: pypdf>=4.0; extra == 'all'
Requires-Dist: python-multipart>=0.0.9; extra == 'all'
Requires-Dist: uvicorn[standard]>=0.30; extra == 'all'
Requires-Dist: yara-python>=4.5; extra == 'all'
Provides-Extra: dev
Requires-Dist: fpdf2>=2.7; extra == 'dev'
Requires-Dist: httpx>=0.27; extra == 'dev'
Requires-Dist: hypothesis>=6.100; extra == 'dev'
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pre-commit>=3.7; extra == 'dev'
Requires-Dist: pypdf>=4.0; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: responses>=0.25; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Requires-Dist: types-requests; extra == 'dev'
Provides-Extra: extract
Requires-Dist: pypdf>=4.0; extra == 'extract'
Provides-Extra: report
Requires-Dist: fpdf2>=2.7; extra == 'report'
Provides-Extra: web
Requires-Dist: fastapi>=0.111; extra == 'web'
Requires-Dist: jinja2>=3.1; extra == 'web'
Requires-Dist: python-multipart>=0.0.9; extra == 'web'
Requires-Dist: uvicorn[standard]>=0.30; extra == 'web'
Provides-Extra: yara
Requires-Dist: yara-python>=4.5; extra == 'yara'
Description-Content-Type: text/markdown

# Tsume (爪)

A security-first toolkit for the full first-response threat intel workflow — triage, monitor, detonate.

Inspired by [malwoverview](https://github.com/alexandreborges/malwoverview) and
[mihari](https://github.com/ninoseki/mihari).

## Install

Not published to PyPI — install from a checkout of this repo.

```bash
pip install -e "."          # CLI only
pip install -e ".[web]"     # + web dashboard (tsume web)
pip install -e ".[yara]"    # + local YARA scanning (tsume scan)
pip install -e ".[extract]" # + PDF IOC extraction (tsume extract)
pip install -e ".[report]"  # + PDF report export (tsume report)
pip install -e ".[all]"     # + everything above
```

`yara-python` requires a C compiler on platforms without a prebuilt wheel; if
the install fails, `tsume scan` will tell you it needs the extra rather than
crashing. `fpdf2` (report) and `pypdf` (extract) are pure Python.

Working on Tsume itself? Add `dev` to any of the above for tests/lint/type-checking,
e.g. `pip install -e ".[dev,all]"`.

## Providers

Run `tsume providers` for a live view of every provider, the IOC types it supports,
and whether its API key is currently configured — the table below adds the
env var name and quirks/rate-limit notes that don't fit in a terminal table.

| Provider | Env var | Supports | Notes |
|---|---|---|---|
| `virustotal` | `TSUME_VIRUSTOTAL_API_KEY` | hash, ip, url, domain | default for all types |
| `abuseipdb` | `TSUME_ABUSEIPDB_API_KEY` | ip | default for ip |
| `urlhaus` | `TSUME_URLHAUS_API_KEY` | hash, ip, url, domain | default for hash/url/domain; requires an abuse.ch Auth-Key |
| `malwarebazaar` | `TSUME_MALWAREBAZAAR_API_KEY` | hash | opt-in via `--provider`; requires an abuse.ch Auth-Key |
| `threatfox` | `TSUME_THREATFOX_API_KEY` | hash, ip, url, domain | opt-in via `--provider`; requires an abuse.ch Auth-Key |
| `shodan` | `TSUME_SHODAN_API_KEY` | ip | opt-in via `--provider`; informational (open ports/CVEs), not a malicious/clean verdict source |
| `greynoise` | `TSUME_GREYNOISE_API_KEY` (optional) | ip | opt-in via `--provider`; Community API works without a key (10 lookups/day), a key raises the quota (up to 50/week) |
| `hybrid-analysis` | `TSUME_HYBRID_ANALYSIS_API_KEY` | hash | opt-in via `--provider` |
| `alienvault` | `TSUME_ALIENVAULT_API_KEY` (optional) | hash, ip, url, domain | opt-in via `--provider`; OTX general lookups work without a key |
| `urlscanio` | `TSUME_URLSCANIO_API_KEY` | ip, url, domain | opt-in via `--provider`; anonymous search works but result detail (verdicts) needs a key |
| `triage` | `TSUME_TRIAGE_API_KEY` | hash, ip, url, domain | opt-in via `--provider`; Hatching Triage sandbox search — matches any sample that *contacted* an ip/url/domain during sandbox execution, not indicators that are themselves malicious (a benign IP a piece of malware happened to resolve DNS against can come back "malicious") |
| `ipinfo` | `TSUME_IPINFO_API_KEY` (optional) | ip | opt-in via `--provider`; informational only (org/geo/ASN), no verdict; works without a key at a lower rate limit |
| `polyswarm` | `TSUME_POLYSWARM_API_KEY` | hash, ip, url, domain | opt-in via `--provider`; raw REST against Polyswarm's API (no SDK dependency, so it stays on the SSRF-guarded session) |
| `whois` | none needed | ip, domain | opt-in via `--provider`; informational only, no verdict; RDAP over plain HTTPS (bootstrapped via rdap.org), no `python-whois`/`ipwhois` dependency |
| `malshare` | `TSUME_MALSHARE_API_KEY` | hash | opt-in via `--provider`; response shape confirmed live |
| `pulsedive` | `TSUME_PULSEDIVE_API_KEY` (optional) | ip, url, domain | opt-in via `--provider`; network-indicator OSINT (not hashes — Pulsedive's indicator model is network infrastructure); unauthenticated requests work at a much lower rate limit |

### CVE providers (`tsume cve`)

| Provider | Env var | Notes |
|---|---|---|
| `nist` | `TSUME_NIST_API_KEY` (optional) | default; NVD's public API works unauthenticated at a lower rate limit |
| `vulncheck` | `TSUME_VULNCHECK_API_KEY` | opt-in via `--provider`; checks VulnCheck's KEV (known exploited vulnerabilities) list; response shape confirmed live |
| `vulncheck-nvd` | `TSUME_VULNCHECK_NVD_API_KEY` | opt-in via `--provider`; VulnCheck's NVD++ index (`nist-nvd2`) — a mirror of NVD data enriched with VulnCheck's own CPE matching, same CVSS-based verdict as `nist`; based on published API docs, not yet confirmed against a live key |
| `vulncheck-xdb` | `TSUME_VULNCHECK_XDB_API_KEY` | opt-in via `--provider`; VulnCheck's Exploit Database (XDB) — public proof-of-concept exploit code found for the CVE, a distinct signal from KEV (public PoC existing doesn't mean it's been used in the wild, so this never scores MALICIOUS on its own); based on published API docs, not yet confirmed against a live key |

### Family/actor providers (`tsume family`)

| Provider | Env var | Notes |
|---|---|---|
| `malpedia` | `TSUME_MALPEDIA_API_KEY` | default; requires manual account approval from Malpedia — the free-registration flow doesn't grant API access. Response shape unconfirmed against a live key. Family names use Malpedia's own convention (e.g. `win.emotet`, not `Emotet`) |
| `mitre-attack` | none needed | default; enriches with MITRE ATT&CK groups/software and their techniques, entirely offline against a local dataset — run `tsume attack-update` first (see below) |

Only the "default" providers above run when `--provider` is omitted, to keep a bare
`tsume hash <value>` fast and light on API quota. Pass `--provider` to query any other
provider — repeatable (`--provider shodan --provider greynoise`) or comma-separated
(`--provider shodan,greynoise`), whichever you prefer.

These built-in defaults can be overridden per IOC type in the config file's
`[defaults]` section (see below) — useful if you'd rather always query ThreatFox,
or drop AbuseIPDB from your default IP lookups.

## Configuration

API keys are read in this order:

1. Environment variable: `TSUME_<PROVIDER>_API_KEY` (hyphens in the provider name become underscores, e.g. `hybrid-analysis` → `TSUME_HYBRID_ANALYSIS_API_KEY`)
2. OS keyring (service name `tsume`, username = provider name)
3. `~/.tsume.conf` (plaintext, last resort — a warning is printed if used)

The config file is standard INI format, one section per provider:

```ini
[virustotal]
api_key = your-key-here

[hybrid-analysis]
api_key = your-key-here
```

To override which providers run by default for each IOC type (instead of the
built-in defaults listed above), add a `[defaults]` section with a comma-separated
provider list per type. Any type you don't list keeps the built-in default:

```ini
[defaults]
hash = virustotal, urlhaus, threatfox
ip = abuseipdb, shodan, greynoise
url = urlhaus
domain = urlhaus
```

This only changes what runs when `--provider` is omitted — `--provider` on the
command line always takes precedence over both the config file and the built-in
defaults.

## Usage

### Lookups

```bash
tsume hash <md5|sha1|sha256> [--provider ...] [--format json|table] [--no-cache] [--cache-ttl SECONDS]
tsume ip <address> [--provider ...] [--format json|table] [--no-cache] [--cache-ttl SECONDS]
tsume url <url> [--provider ...] [--format json|table] [--no-cache] [--cache-ttl SECONDS]
tsume domain <domain> [--provider ...] [--format json|table] [--no-cache] [--cache-ttl SECONDS]
tsume cve <cve-id> [--provider ...] [--format json|table] [--no-cache] [--cache-ttl SECONDS]
tsume family <name> [--provider ...] [--format json|table] [--no-cache] [--cache-ttl SECONDS]
```

Every lookup is served from and saved to a persistent disk cache (`~/.tsume_cache.db`,
default 1 hour TTL); `--no-cache` bypasses it and `tsume cache-clear` empties it entirely.

### Batch, local file analysis, and reporting

```bash
tsume batch <file> [--type hash|ip|url|domain|cve|family] [--provider ...] [--format json|table]
tsume scan <file> --rules <rule-file-or-dir> [--format json|table]           # requires the `yara` extra
tsume extract <file.pdf|file.eml> [--type pdf|eml] [--output <file>] [--format json|table]
tsume report <output.html|output.pdf> [--ioc-type ...] [--verdict ...] [--provider ...] [--search ...] [--limit N]
```

`batch` reads one IOC per line (`#` comments allowed) and auto-detects each
line's type unless `--type` forces one. `extract` pulls hash/IP/URL/CVE/domain-shaped
strings out of a PDF or email — including "defanged" indicators like `hxxp://` and
`1[.]2[.]3[.]4` — and `--output` writes a clean line-per-IOC file ready to feed
straight into `batch`. `report` exports a filtered slice of lookup history.

### MITRE ATT&CK

```bash
tsume attack-update [--url <bundle-url>]     # downloads MITRE's public ATT&CK dataset (~45MB), once
tsume attack <technique-id|group-or-software-name>
```

`attack-update` fetches the STIX bundle to `~/.tsume_attack.json`; nothing else
in Tsume downloads it automatically. `tsume attack T1055.011` looks up a technique
by ID; `tsume attack "Evil Corp"` matches a group or software by name or alias.

### Scheduled monitoring & alerting

```bash
tsume monitor add <name> --type <ioc-type> --value <value> [--provider ...] [--alert-on malicious|change] [--webhook <url>]
tsume monitor list
tsume monitor enable|disable|remove <rule-id>
tsume monitor run       # runs every enabled rule once; invoke this from cron/Task Scheduler
tsume monitor history   # shows fired alerts
```

A rule re-checks its IOC each time `monitor run` is invoked and only alerts when
the outcome is *new* (a changed verdict, or a newly-malicious result it hadn't
already recorded) — a scheduled cron job won't re-alert on an unchanged result
every time it runs. `--webhook` POSTs a JSON payload to any URL (Slack/Discord
incoming webhooks, a self-hosted relay, etc.) when a rule fires.

### Triage cloud detonation

```bash
tsume detonate <file> [--network internet|drop|tor|vpn] [--timeout SECONDS] [--wait] [--yes]
tsume detonate-status <sample-id>
```

Uploads a local file to Triage's cloud sandbox for full dynamic analysis (not
just hash reputation). Prompts for confirmation before every submission —
depending on your Triage account tier, samples may be **public** and visible
to other Triage users; Tsume reports "private"/"public"/"unknown" rather than
guessing when the API doesn't say. `--wait` blocks and polls until reported;
otherwise check back later with `detonate-status`.

### Web dashboard

```bash
tsume web [--host 127.0.0.1] [--port 8400] [--reload] [--allow-remote]
```

See below.

## History & the web dashboard

Every successful lookup — from the CLI or the web UI — is recorded to a local
SQLite database at `~/.tsume_history.db`. There's no separate opt-in: it's
written automatically so the dashboard has real data to show without extra
setup, and it never leaves your machine.

`tsume web` (requires the `web` extra) starts a local FastAPI server with six views:

- **History** (default view) — a searchable, filterable, sortable log of every past lookup, with an expandable row showing the full raw provider response.
- **Trends** — aggregate stats: verdict/severity distribution, lookups by provider and IOC type, and a list of IOCs where providers disagreed on the verdict.
- **Live Lookup** — the same hash/ip/url/domain/cve/family lookups as the CLI, run from a form in the browser, streamed in per-provider over SSE as each one completes rather than waiting for the slowest one.
- **Monitor** — add/list/enable/disable/remove monitor rules and trigger a manual run, plus a recent-alerts feed; the browser equivalent of `tsume monitor`.
- **ATT&CK** — search techniques by ID or groups/software by name, same as `tsume attack`.
- **Tools** — file-upload workflows for YARA scanning, PDF/email IOC extraction, and Triage cloud detonation.

The dashboard is entirely local — no CDN assets, no external API calls beyond
the same provider APIs the CLI already talks to, and no data leaves your
machine except those provider requests. Design direction lives in
[PRODUCT.md](PRODUCT.md) and [DESIGN.md](DESIGN.md) (a security-tool "Operator's
Console" aesthetic: dark graphite surface, a single steel-cyan accent, verdict
colors reserved exclusively for verdict meaning).

**The dashboard has no authentication.** It defaults to binding `127.0.0.1`
only, which is safe — nothing off this machine can reach it. `tsume web`
refuses to bind to any other address unless you pass `--allow-remote`, because
several views can trigger real actions with no login prompt: Triage cloud
submissions against your API quota, YARA scans, and monitor rule changes. Only
use `--allow-remote` on a network you trust.

## Development

```bash
ruff check .
mypy --strict src/tsume/core
pytest
```

## Skipped providers

Researched against malwoverview's actual usage and ruled out entirely (not that they're bad
services — they just don't fit any of Tsume's command shapes):

- **BGPView** — the `bgpview.io` domain no longer resolves; the service appears to be shut down entirely.

## Roadmap

- **v1**: VirusTotal, AbuseIPDB, URLhaus lookups; validation + SSRF-guard core; tests + CI.
- **v2**: 15 hash/ip/url/domain providers, `cve` command (NIST, VulnCheck), `family` command (Malpedia), persistent SQLite history, disk-backed response cache, batch processing, YARA scanning, HTML/PDF reports, and a `tsume web` dashboard with per-provider SSE streaming.
- **v3**: MITRE ATT&CK mapping, IOC extraction from PDFs/emails, rule-based scheduled monitoring with alerting, and an expanded Triage integration (multi-IOC search, cloud detonation). A TUI dashboard was considered and deliberately dropped — the CLI and web dashboard already cover the scriptable and visual use cases, so a third UI wasn't worth the maintenance cost. LLM enrichment (summarizing multi-provider results) remains unbuilt by choice.
- **Not yet verified against a live key**: Malpedia's family-lookup response shape — blocked on manual account approval, not a code issue.
