ThreatLens · ZEIT3118 Capstone

ThreatLens Documentation

ThreatLens is a prototype Cyber Threat Intelligence Indicator Quality Checker. It extracts candidate indicators of compromise (IoCs) from CTI reports and feeds, organises them into a structured repository, and checks them for quality issues — before handing the reader an auditable quality_score for every indicator instead of a raw, unfiltered dump.

Capstone project for ZEIT3118, UNSW Canberra (ADFA), Semester 2 2026, for client Dr. Clemens Sauerwein (University of Innsbruck).

The problem

Threat reports are full of candidate indicators — IPs, domains, hashes, URLs, file paths — but not everything that looks like an indicator is one, and not everything that is one is reported cleanly. The client's brief names three quality problems explicitly, and ThreatLens exists to demonstrably address them:

1. Duplication — the same underlying intelligence stored more than once, whether as a repeated document or an indicator recorded under multiple representations. (Repeated observation of the same indicator across genuinely independent sources is not duplication — it's corroborating evidence, recorded as multiple sightings.)
2. Missing context — indicators with no supporting information about what they mean or where they came from.
3. Likely false positives — values that look like IoCs but aren't: technique idioms, private/non-routable IPs, a report's own self-referencing URL, code fragments mistaken for domains.

Whatever survives that screening is scored against the quality framework adapted from the project's research paper, producing a 0–100 quality_score for every stored indicator, alongside — but never conflated with — its 0–100 confidence of being malicious. Four of the paper's five dimensions are scored: timeliness, accuracy, completeness and relevance. The fifth, consistency, needs an external feed and is deferred (see Quality Checker).

How the pieces fit together

CTI report (PDF/text)
      │
      ▼
Extraction Engine ──▶ candidate IoCs + context, verified by one engine per
      │                 run (Qwen LLM by default, or python heuristics)
      │
      ├──▶ CLI (threatlens) ──▶ inspect one report: summary or JSON,
      │                          no database
      ▼
Quality Checker ──▶ structural rejection → STIX conformance →
      │              false-positive screen → dimension scoring
      ▼
Database (SQLite via SQLAlchemy + Alembic)
      │        sources, raw_documents, indicators, sightings,
      │        quality_flags, allowlist, scoring_config
      ▼
Dashboard / Web UI ──▶ browse and triage the data; upload, ingest, configure

Reports reach the database through one ingest pipeline: extract, quality checks, persist, rescore. The dashboard's Ingest page, the threatlens-ingest command and the batch runner behind the evaluation runs all use it. The threatlens CLI stops after extraction: it shows what the engine made of a single report, and its JSON output can be fed to the quality checker's own command line to inspect that stage as well.

Getting started

1. Install. pip install "threatlens-cti[dashboard]", or pip install -e ".[dashboard]" from a checkout (add dev for the tests). The package carries the dashboard's pages, these documentation pages and the database migrations, and installs the threatlens, threatlens-ingest, threatlens-calibrate and threatlens-dashboard commands. The llm engine also needs an Ollama server with the verifier model pulled.
2. Set up. Run threatlens init. It asks for the data folder, Ollama URL (tested live), models (chosen from those the server has pulled), dashboard address and optional quality weights, writes them to threatlens.conf, creates the database, and can calibrate the LLM settings for your hardware (details). The file lives in the repo root in a checkout and in the per-user config folder when pip-installed (where files live); re-run the wizard, use the dashboard's Configuration page, or edit the file to change it later.
3. Calibrate, if skipped during setup: threatlens-calibrate benchmarks the Ollama server and saves the [llm] settings (details). A database can likewise be created or upgraded on its own with python -m threatlens.db.migrate (details).
4. Run. threatlens-dashboard for the web UI (these pages are at /docs/), threatlens-ingest report.pdf to ingest from a terminal, or threatlens report.pdf to inspect one report.

Until setup and calibration are done, the commands and the dashboard print a warning naming the missing step (and, in a terminal, offer to run threatlens init) and carry on with built-in defaults (see the setup check).

Project components

Deliberately out of scope

No threat actor / campaign / kill-chain graph modelling, no inter-indicator relationships, no intelligence sharing between organisations, no external enrichment (VirusTotal, AbuseIPDB). These are OpenCTI territory and are noted as future work rather than built.

---

Status key Pages marked Written are the full explanation for that component. Pages marked Stub are placeholders — a short, accurate summary and pointers into the source — pending the full write-up.

ThreatLens is free software, licensed under the GNU Affero General Public License v3.0 or later (AGPL-3.0-or-later). Copyright © 2026 Tyler Passey, Ben King and Ethan McLachlan.