Metadata-Version: 2.4
Name: llm-model-security-scanner
Version: 1.1.0
Summary: Comprehensive enterprise security scanner & proof verification engine for open-source LLMs and Hugging Face repositories
Author: Mesut Güngör
License-Expression: Apache-2.0
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.0.0
Requires-Dist: safetensors>=0.4.0
Requires-Dist: huggingface_hub>=0.20.0
Requires-Dist: requests>=2.28.0
Requires-Dist: rich>=13.0.0
Requires-Dist: click>=8.0.0
Provides-Extra: mcp
Requires-Dist: mcp>=1.0.0; extra == "mcp"
Dynamic: license-file

# 🛡️ LLM Model Security Scanner (`llm-scan`)

[![License: Apache-2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE)
[![Python Version](https://img.shields.io/badge/python-3.9%20%7C%203.10%20%7C%203.11%20%7C%203.12-brightgreen.svg)](pyproject.toml)
[![VS Code Extension](https://img.shields.io/badge/VS_Code_Extension-v1.0.0-007ACC?logo=visual-studio-code&logoColor=white)](https://github.com/mesutgungor/vscode-llm-model-security-scanner)
[![MCP Server](https://img.shields.io/badge/MCP-Server%20Ready-00D26A?logo=anthropic)](#-model-context-protocol-mcp-server-for-coding-agents)
[![Tests Passing](https://img.shields.io/badge/tests-38%2F38%20passing-success.svg)](tests/)
[![Security Standards](https://img.shields.io/badge/Compliance-OWASP%20%7C%20MITRE%20ATLAS%20%7C%20NIST%20AI%20RMF%20%7C%20EU%20AI%20Act-purple.svg)](#-regulatory--security-framework-crosswalk)
[![SARIF v2.1.0](https://img.shields.io/badge/SARIF-GitHub%20Security%20Tab-orange.svg)](#-cicd-github-actions-sarif-integration)

> **Enterprise-grade, zero-execution static analysis scanner & cryptographic Proof-of-Finding verification engine for open-source Large Language Models (LLMs) distributed across Hugging Face, ModelScope, and local storage.**

<p align="center">
  <img src="https://raw.githubusercontent.com/mesutgungor/llm-model-security-scanner/main/assets/demo.gif" alt="llm-scan CLI in action" width="850">
</p>

`llm-scan` inspects model serialization formats, weights, configuration manifests, tokenizer dictionaries, chat templates, and custom architecture scripts **before** they are loaded into runtime environments like PyTorch, Hugging Face Transformers, vLLM, Ollama, or llama.cpp.

---

## 📌 Table of Contents

- [Why LLM Model Security?](#-why-llm-model-security)
- [Core Architecture & Zero-Execution Safety](#-core-architecture--zero-execution-safety)
- [Key Features](#-key-features)
- [💻 Visual Studio Code Extension](#-visual-studio-code-extension)
- [🤖 Model Context Protocol (MCP) Server for Coding Agents](#-model-context-protocol-mcp-server-for-coding-agents)
- [Installation & Quickstart](#-installation--quickstart)
- [CLI Command Reference](#-cli-command-reference)
  - [1. Scan Local Models](#1-scan-local-models)
  - [2. Remote Hub Auditing (Zero Weight Download)](#2-remote-hub-auditing-zero-weight-download)
  - [3. Proof-of-Finding Engine (`llm-scan verify`)](#3-proof-of-finding-engine-llm-scan-verify)
  - [4. Output Formats (Terminal, SARIF, Markdown, JSON, HTML)](#4-output-formats)
  - [5. CI/CD Gating](#5-cicd-quality-gate)
- [Proof-of-Finding Verification Engine](#-proof-of-finding-verification-engine)
- [Complete 42-Rule Security Catalog](#-complete-42-rule-security-catalog)
- [Regulatory & Security Framework Crosswalk](#-regulatory--security-framework-crosswalk)
- [Validated Real-World Models](#-validated-real-world-models)
- [Python SDK Reference](#-python-sdk-reference)
- [CI/CD GitHub Actions Integration](#-cicd-github-actions-sarif-integration)
- [Threat Model Lifecycle](#-threat-model-lifecycle)
- [Running the Test Suite](#-running-the-test-suite)
- [Author & License](#-author--license)

---

## 🚨 Why LLM Model Security?

Deploying untrusted third-party foundation models introduces severe attack surfaces into enterprise infrastructure:

1. **Pickle Remote Code Execution (RCE):** Legacy PyTorch `.bin`, `.pt`, and `.ckpt` files are serialized via Python's `pickle` engine. Deserialization allows attackers to execute arbitrary system commands (`os.system`, `subprocess`, socket reverse shells) the instant `torch.load()` is executed.
2. **`trust_remote_code=True` Exploits:** Repositories declaring custom model classes in `config.json` (`auto_map`) run arbitrary unvetted Python code on the host machine.
3. **Jinja2 SSTI in Chat Templates (CVE-2024-34359):** Malicious `tokenizer_config.json` chat templates can escape the Jinja sandbox via Python reflection (`__class__.__mro__[1].__subclasses__()`) to achieve arbitrary code execution.
4. **Format Denial of Service (CVE-2023-44469):** SafeTensors 8-byte header allocations can be crafted into memory allocation bombs, crashing host inference servers.
5. **ONNX External Data Traversal (CVE-2024-27318):** Crafty ONNX files can write or read outside repository boundaries via path traversal.
6. **Token Delimiter Hijacking & Trojan Source (CVE-2021-42574):** Token ID collisions and unescaped role tags (`<|im_start|>`) break prompt boundary isolation, enabling prompt injection. Unicode bidirectional overrides smuggle invisible instructions into vocabularies.
7. **LoRA Adapter Scaling Sabotage & Stealth Layers:** Malicious fine-tuned adapters deliberately strip base model safety alignments by setting astronomical scaling factors ($\alpha \gg r$) or injecting Trojan gate weights.
8. **Credential & Secret Leaks:** Model authors frequently leave hardcoded Hugging Face write tokens, OpenAI API keys, AWS credentials, and SSH private keys in files.

---

## 🔒 Core Architecture & Zero-Execution Safety

Traditional scanners often import models into Python, which can accidentally trigger the very exploits they seek to find. 

**`llm-scan` enforces a strict Zero-Execution Architecture:**
- **No `torch.load()` Execution:** Pickles and PyTorch archives are disassembled into opcodes via Python's standard `pickletools` without instantiating objects or running `__reduce__`.
- **AST Parsing for Python Code:** Custom modeling files (`modeling_*.py`, `configuration_*.py`) are analyzed purely as Abstract Syntax Trees via Python's native `ast` compiler.
- **Memory-Mapped Binary Streaming:** SafeTensors and GGUF files are checked via 8-byte stream headers and `mmap` slices without allocating memory or evaluating data arrays.
- **Remote Hub Inspection:** Scans remote Hugging Face repositories in seconds over the Hugging Face REST API without downloading gigabytes of model weights.

```
                              ┌──────────────────────────────────────────────┐
                              │     Model Target (Local Folder / Hub ID)     │
                              └──────────────────────┬───────────────────────┘
                                                     │
               ┌─────────────────────────────────────┴─────────────────────────────────────┐
               ▼                                                                           ▼
   [Remote Hub Inspection]                                                     [Local Model Filesystem]
   - Repository Metadata                                                       - Zero-Execution Disassembly
   - Typosquatting Check                                                       - SafeTensors / GGUF Stream
   - Fast Weightless Audit                                                     - AST Static Analysis
               │                                                                           │
               └─────────────────────────────────────┬─────────────────────────────────────┘
                                                     │
                                                     ▼
                                      ┌─────────────────────────────┐
                                      │   13 Specialized Scanners   │
                                      │    (42 Registered Rules)    │
                                      └──────────────┬──────────────┘
                                                     │
                                                     ▼
                                      ┌─────────────────────────────┐
                                      │  Finding Verification Engine │
                                      │ (Cryptographic SHA-256 Proof)│
                                      └──────────────┬──────────────┘
                                                     │
                         ┌───────────────────────────┼───────────────────────────┐
                         ▼                           ▼                           ▼
                 [Rich Terminal]             [SARIF / JSON]            [Interactive HTML / MD]
              Color Tables & Proof Cards  GitHub Security Tab Upload     Client Proof Dossiers
```

---

## ✨ Key Features

- **13 Specialized Scanners & 42 Production Rules:** Comprehensive coverage across all major model types.
- **Deterministic Proof-of-Finding Engine:** Generates mathematical, byte-level evidence to prove vulnerabilities to model authors and security teams.
- **Interactive HTML Proof Certificates:** Self-contained, zero-dependency HTML certificates with embedded syntax highlighters and SHA-256 fingerprints.
- **Compliance Crosswalk Built-in:** Instant mapping to **OWASP Top 10 for LLMs (2025)**, **MITRE ATLAS**, **NIST AI RMF (AI 100-1)**, and **EU AI Act (Regulation 2024/1689)**.
- **Remote Hugging Face Hub Auditing:** Scan public or private repositories directly using the Hugging Face API without downloading 10GB–100GB weights.
- **Native CI/CD Integration:** Produces standard **SARIF v2.1.0** reports directly compatible with GitHub Code Scanning & Advanced Security.
- **Wide Format Support:** SafeTensors, PyTorch `.bin`/`.pt`, GGUF, GGML, ONNX, PEFT/LoRA adapters, Hugging Face Tokenizers, and Chat Templates.

---

## 💻 Visual Studio Code Extension

Prefer an in-editor IDE experience? Install the official **LLM Model Security Scanner for Visual Studio Code**:

👉 **[vscode-llm-model-security-scanner](https://github.com/mesutgungor/vscode-llm-model-security-scanner)**

<p align="center">
  <img src="https://raw.githubusercontent.com/mesutgungor/vscode-llm-model-security-scanner/main/media/preview.png" alt="LLM Model Security Scanner VS Code Extension" width="850">
</p>

- **🔴 Real-Time As-You-Write Linter:** Flags insecure calls (`os.system`, `eval()`, `zlib.decompress`, credential leaks) as you write Python code and manifests with red & yellow squiggly underlines.
- **🛡️ Remote Import Guardian:** As soon as you write `AutoModelForCausalLM.from_pretrained("org/model")`, `pipeline(..., model="org/model")`, or `SentenceTransformer(...)`, the extension automatically audits the Hugging Face repository over the API weightless.
- **🔍 Inline CodeLens Badges:** Displays instant status above model loading lines (`$(pass) Clean (0/100)` or `$(alert) [HIGH_RISK] [View Proof]`).
- **💡 1-Click Quick Fixes (`⌘.`):** Explain vulnerabilities, open documentation anchors, or generate Proof Dossiers.
- **🔬 Interactive Proof Webview:** View cryptographic Proof-of-Finding certificates and AST traces directly inside a native VS Code tab.

---

## 🤖 Model Context Protocol (MCP) Server for Coding Agents

Autonomous coding agents (**Claude Code**, **Cursor Agent**, **Windsurf**, **Devin**) often write code and execute terminal commands on your behalf. If an agent recommends or imports an unvetted or typosquatted Hugging Face model, malicious code or pickle payloads execute on your machine automatically.

The **LLM Security MCP Server** acts as an **autonomous pre-flight guardrail**, allowing coding agents to vet models before writing code or running inference:

### Configure in Claude Desktop / Claude Code / Cursor:
Add to your `claude_desktop_config.json` or Cursor MCP settings:

```json
{
  "mcpServers": {
    "llm-security": {
      "command": "llm-scan",
      "args": ["mcp"]
    }
  }
}
```

*(Or via Python directly: `{"command": "python3", "args": ["-m", "llm_security_scanner.mcp_server"]}`)*

### Tools Exposed to Agents:
| Tool Name | Parameters | Purpose |
| :--- | :--- | :--- |
| `scan_huggingface_model` | `repo_id`, `hf_token?` | Audits remote Hub manifests & templates weightless before the agent imports it. |
| `audit_local_model` | `target_path`, `deep?` | Zero-execution scan on local files (`.safetensors`, `.gguf`, `.py`, `.bin`). |
| `verify_model_proof` | `target`, `is_remote?` | Produces verifiable cryptographic Proof Dossiers with SHA-256 fingerprints. |
| `explain_security_rule` | `rule_id` | Explains CVE, CWE, and OWASP/MITRE/EU AI Act compliance mappings for any rule. |

### Autonomous Agent Gating Workflow:
When a developer prompts: *"Build a sentiment app with a Hugging Face model"*:
1. The agent queries `scan_huggingface_model("candidate-org/model")`.
2. If `safe_to_load: false`, the agent rejects the candidate and automatically selects a verified secure model.
3. If `safe_to_load: true`, the agent proceeds with generating the implementation.

---

## 📦 Installation & Quickstart

### Prerequisites
- Python $\ge$ 3.9
- Operating System: macOS, Linux, or Windows (WSL)

### Install from PyPI (Recommended)
```bash
# Standard installation
pip install llm-model-security-scanner

# With MCP Server support for coding agents
pip install "llm-model-security-scanner[mcp]"
```

### Install from Source
```bash
git clone https://github.com/mesutgungor/llm-model-security-scanner.git
cd llm-model-security-scanner
pip install -e .
```

Verify the installation:
```bash
llm-scan --help
```

---

## 🛠️ CLI Command Reference

### 1. Scan Local Models

Scan a complete model directory:
```bash
llm-scan scan ./path-to-model-directory
```

Scan a single model file:
```bash
llm-scan scan ./models/pytorch_model.bin
```

Enable deep weight sampling (checks for NaN/Infinity DoS poisoning and sabotaged layers):
```bash
llm-scan scan ./models/my-model --deep
```

Run in Fast Mode (skips tensor arrays, audits metadata, code, templates, and configurations):
```bash
llm-scan scan ./models/my-model --fast
```

---

### 2. Remote Hub Auditing (Zero Weight Download)

Scan remote repositories directly on Hugging Face without downloading weight files:
```bash
# Public repository
llm-scan scan meta-llama/Llama-3-8B-Instruct --remote

# Private / Gated repository (requires HF token)
llm-scan scan my-org/custom-model --remote --token hf_yourTokenHere
```

---

### 3. Proof-of-Finding Engine (`llm-scan verify`)

Generate verifiable proof dossiers containing cryptographic file hashes, exact code locations, and non-destructive reproduction steps:

```bash
# Verify local model and render terminal proof cards
llm-scan verify ./path-to-model

# Verify remote Hugging Face model and export self-contained HTML proof certificate
llm-scan verify NousResearch/Hermes-3-Llama-3.1-8B --remote --format html --output hermes-certificate.html

# Export Markdown proof dossier for issue tracking & PRs
llm-scan verify THUDM/chatglm-6b --remote --format markdown --output chatglm-proof-dossier.md

# Export machine-readable JSON proof dossier for security orchestrators
llm-scan verify meta-llama/Llama-3-8B --remote --format json --output proofs.json
```

---

### 4. Output Formats

`llm-scan` supports flexible output formats:

| Format Flag | Target Use Case | Example Command |
| :--- | :--- | :--- |
| `terminal` *(default)* | Human interactive inspection | `llm-scan scan ./model` |
| `sarif` | GitHub Advanced Security / IDEs | `llm-scan scan ./model --format sarif -o results.sarif` |
| `markdown` | GitHub PR comments / Documentation | `llm-scan scan ./model --format markdown -o report.md` |
| `json` | Automation pipelines / SIEM / SOAR | `llm-scan scan ./model --format json -o report.json` |
| `html` *(verify)* | Executive & Consumer Proof Certificates | `llm-scan verify <target> --format html -o cert.html` |

---

### 5. CI/CD Quality Gate

Enforce security thresholds in automated deployment pipelines. If any finding meets or exceeds `--max-severity`, `llm-scan` exits with non-zero code `1`:

```bash
llm-scan scan ./models/production-candidate --max-severity HIGH
```

Severity Levels: `LOW` | `MEDIUM` | `HIGH` | `CRITICAL`

---

### 6. Utility Commands

```bash
# Print all 42 registered security detection rules
llm-scan list-rules

# View the full regulatory & framework crosswalk (OWASP, MITRE ATLAS, NIST, EU AI Act)
llm-scan compliance

# View the 5-phase end-to-end LLM security lifecycle threat model
llm-scan threat-model
```

---

## 🔬 Proof-of-Finding Verification Engine

Security scanners frequently encounter pushback when alerting on model repositories: *"Is this really exploitable?"* or *"How do I reproduce this?"*

`llm-scan verify` solves this problem by generating **undeniable, verifiable proof artifacts**:

1. **Cryptographic SHA-256 Fingerprint:** Proves the exact file bytes scanned.
2. **Context Code Window:** Extracts the exact offending lines and surrounding context.
3. **AST Syntax Tree / Opcode Trace:** Provides the compiler-level structure of the exploit.
4. **Step-by-Step Reproduction Guide:** Clear, non-destructive reproduction steps for engineers.
5. **Interactive HTML Proof Certificate:** Standalone, zero-dependency HTML file featuring:
   - Severity badges & status indicators.
   - Interactive verification modal.
   - One-click copy for reproduction steps and SHA-256 hashes.
   - Regulatory framework tags.

---

## 📋 Complete 42-Rule Security Catalog

| Rule ID | Severity | Category | Description | CWE / Reference |
| :--- | :---: | :--- | :--- | :--- |
| <a id="sec-pck-001"></a>**`SEC-PCK-001`** | **CRITICAL** | Deserialization Exploit | Arbitrary Code Execution via dangerous pickle opcodes (`GLOBAL`, `REDUCE`, `BUILD`) | CWE-502 |
| <a id="sec-pck-002"></a>**`SEC-PCK-002`** | **HIGH** | Deserialization Exploit | Suspicious or unapproved global object in pickle stream (`os`, `subprocess`, `posix`) | CWE-502 |
| <a id="sec-pck-003"></a>**`SEC-PCK-003`** | **LOW** | Format Integrity | Incomplete or malformed pickle bytecode stream | CWE-20 |
| <a id="sec-pck-info-001"></a>**`SEC-PCK-INFO-001`** | **LOW** | Format Integrity | Legacy PyTorch pickle serialization format detected (migrate to SafeTensors) | CWE-502 |
| <a id="sec-zip-001"></a>**`SEC-ZIP-001`** | **CRITICAL** | Deserialization Exploit | ZipSlip path traversal in PyTorch archive (`../` file escape) | CWE-22 |
| <a id="sec-zip-002"></a>**`SEC-ZIP-002`** | **LOW** | Format Integrity | Corrupted PyTorch ZIP container structure | CWE-20 |
| <a id="sec-rce-001"></a>**`SEC-RCE-001`** | **HIGH** | Remote Code Execution | Suspicious module imports in architecture scripts (`subprocess`, `socket`, `urllib`) | CWE-94 |
| <a id="sec-rce-002"></a>**`SEC-RCE-002`** | **CRITICAL** | Remote Code Execution | Dynamic code execution via `eval()`, `exec()`, `__import__()` | CWE-95 |
| <a id="sec-rce-003"></a>**`SEC-RCE-003`** | **CRITICAL** | Remote Code Execution | OS command execution via `system()`, `popen()`, `spawn()` | CWE-78 |
| <a id="sec-rce-map-001"></a>**`SEC-RCE-MAP-001`** | **HIGH** | Remote Code Execution | Auto-map declaration in `config.json` requiring `trust_remote_code=True` | CWE-94 |
| <a id="sec-rce-map-002"></a>**`SEC-RCE-MAP-002`** | **MEDIUM** | Remote Code Execution | Custom modeling parameter or custom Triton/CUDA kernel declaration | CWE-94 |
| <a id="sec-obf-001"></a>**`SEC-OBF-001`** | **HIGH** | Remote Code Execution | Obfuscated payload decoding (`base64.b64decode`, `zlib.decompress`) | CWE-116 |
| <a id="sec-exf-001"></a>**`SEC-EXF-001`** | **MEDIUM** | Secret Leak | Environment variable access (`os.environ`) in model scripts | CWE-200 |
| <a id="sec-tmpl-ssti-001"></a>**`SEC-TMPL-SSTI-001`** | **CRITICAL** | Chat Template SSTI | Jinja2 reflection escape / Transformers SSTI vulnerability (**CVE-2024-34359**) | CWE-1336 / CVE-2024-34359 |
| <a id="sec-tmpl-ssti-002"></a>**`SEC-TMPL-SSTI-002`** | **CRITICAL** | Chat Template SSTI | Jinja2 sandbox escape / global breakout attempt (`lipsum.__globals__`, `cycler`) | CWE-1336 |
| <a id="sec-st-001"></a>**`SEC-ST-001`** | **HIGH** | Format Integrity | Truncated SafeTensors file (< 8 bytes) | CWE-125 |
| <a id="sec-st-002"></a>**`SEC-ST-002`** | **HIGH** | Format Integrity | Invalid zero or negative SafeTensors header size | CWE-125 |
| <a id="sec-st-003"></a>**`SEC-ST-003`** | **CRITICAL** | Format Integrity | SafeTensors header allocation memory bomb (**CVE-2023-44469**) | CWE-400 / CVE-2023-44469 |
| <a id="sec-st-004"></a>**`SEC-ST-004`** | **HIGH** | Format Integrity | Malformed SafeTensors JSON header | CWE-20 |
| <a id="sec-st-006"></a>**`SEC-ST-006`** | **CRITICAL** | Format Integrity | SafeTensors out-of-bounds or inverted tensor offsets | CWE-125 |
| <a id="sec-st-007"></a>**`SEC-ST-007`** | **CRITICAL** | Format Integrity | Tensor data offsets exceed physical file boundary | CWE-125 |
| <a id="sec-st-meta-001"></a>**`SEC-ST-META-001`** | **HIGH** | Suspicious Artifact | Suspicious executable pattern in SafeTensors `__metadata__` | CWE-94 |
| <a id="sec-gguf-001"></a>**`SEC-GGUF-001`** | **HIGH** | Format Integrity | Truncated GGUF binary file | CWE-125 |
| <a id="sec-gguf-002"></a>**`SEC-GGUF-002`** | **HIGH** | Format Integrity | Invalid GGUF magic header bytes | CWE-20 |
| <a id="sec-gguf-004"></a>**`SEC-GGUF-004`** | **CRITICAL** | Format Integrity | Anomalous GGUF tensor count triggering C++ integer overflow | CWE-190 |
| <a id="sec-gguf-005"></a>**`SEC-GGUF-005`** | **HIGH** | Format Integrity | Anomalous GGUF metadata key-value count (DoS) | CWE-400 |
| <a id="sec-onnx-001"></a>**`SEC-ONNX-001`** | **CRITICAL** | Deserialization Exploit | ONNX external data path traversal (**CVE-2024-27318**) | CWE-22 / CVE-2024-27318 |
| <a id="sec-onnx-002"></a>**`SEC-ONNX-002`** | **MEDIUM** | Remote Code Execution | ONNX custom domain operator declaration | CWE-94 |
| <a id="sec-tok-001"></a>**`SEC-TOK-001`** | **MEDIUM** | Special Tokens | Missing End-of-Sequence (`eos_token`) configuration causing unbounded generation DoS | CWE-400 |
| <a id="sec-tok-002"></a>**`SEC-TOK-002`** | **HIGH** | Special Tokens | Empty string EOS token configured | CWE-20 |
| <a id="sec-tok-003"></a>**`SEC-TOK-003`** | **HIGH** | Special Tokens | Token ID collision enabling role delimiter hijacking | CWE-20 |
| <a id="sec-tok-004"></a>**`SEC-TOK-004`** | **HIGH** | Special Tokens | Critical delimiter marked non-special (prompt boundary injection) | CWE-20 |
| <a id="sec-tok-005"></a>**`SEC-TOK-005`** | **LOW** | Special Tokens | BOS and EOS tokens are identical | CWE-20 |
| <a id="sec-tok-006"></a>**`SEC-TOK-006`** | **HIGH** | Unicode Smuggling | Unicode Bidirectional (Bidi) Trojan Override (**CVE-2021-42574**) | CWE-94 / CVE-2021-42574 |
| <a id="sec-tok-007"></a>**`SEC-TOK-007`** | **MEDIUM** | Unicode Smuggling | Zero-width invisible character smuggling in vocabulary | CWE-116 |
| <a id="sec-gen-001"></a>**`SEC-GEN-001`** | **HIGH** | Generation DoS | Extreme token generation limit in `max_new_tokens` (KV cache exhaustion) | CWE-400 |
| <a id="sec-gen-002"></a>**`SEC-GEN-002`** | **MEDIUM** | Generation DoS | Invalid sampling temperature (division by zero / inference crash) | CWE-20 |
| <a id="sec-gen-003"></a>**`SEC-GEN-003`** | **MEDIUM** | Generation DoS | Missing `eos_token_id` in generation manifest | CWE-400 |
| <a id="sec-peft-001"></a>**`SEC-PEFT-001`** | **HIGH** | Adapter Poisoning | Suspicious remote base model path in adapter manifest | CWE-494 |
| <a id="sec-peft-002"></a>**`SEC-PEFT-002`** | **HIGH** | Adapter Poisoning | LoRA scaling factor sabotage (alignment stripping / jailbreak) | CWE-693 |
| <a id="sec-peft-003"></a>**`SEC-PEFT-003`** | **MEDIUM** | Adapter Poisoning | LoRA adapter targets safety / classification layer | CWE-693 |
| <a id="sec-peft-004"></a>**`SEC-PEFT-004`** | **HIGH** | Adapter Poisoning | Unrecognized or custom PEFT method declaration | CWE-94 |
| <a id="sec-wgt-001"></a>**`SEC-WGT-001`** | **HIGH** | Weight Poisoning | Suspicious tensor name matching trigger / backdoor keywords | CWE-506 |
| <a id="sec-wgt-002"></a>**`SEC-WGT-002`** | **CRITICAL** | Weight Poisoning | Poisoned weights containing NaN or Infinity values (Inference DoS) | CWE-400 |
| <a id="sec-wgt-003"></a>**`SEC-WGT-003`** | **HIGH** | Weight Poisoning | Sabotaged / all-zero layer weights | CWE-400 |
| <a id="sec-wgt-004"></a>**`SEC-WGT-004`** | **MEDIUM** | Weight Poisoning | Extreme weight magnitude outlier spike (Trojan gate trigger) | CWE-506 |
| <a id="sec-wgt-005"></a>**`SEC-WGT-005`** | **CRITICAL** | Weight Poisoning | Stealth out-of-bounds layer injected (hidden Trojan layer) | CWE-506 |
| <a id="sec-art-001"></a>**`SEC-ART-001`** | **HIGH** | Suspicious Artifact | Standalone executable script or native binary in model directory | CWE-507 |
| <a id="sec-art-002"></a>**`SEC-ART-002`** | **CRITICAL** | Suspicious Artifact | Disguised Linux ELF, Windows PE, or macOS Mach-O binary disguised as `.bin` | CWE-507 |
| <a id="sec-art-003"></a>**`SEC-ART-003`** | **MEDIUM** | Suspicious Artifact | Abnormally high Shannon entropy in plaintext file | CWE-507 |
| <a id="sec-sym-001"></a>**`SEC-SYM-001`** | **CRITICAL** | Suspicious Artifact | Malicious symlink traversal pointing outside model repository | CWE-59 |
| <a id="sec-sec-001"></a>**`SEC-SEC-001`** | **CRITICAL** | Secret Leak | Exposed Hugging Face token, OpenAI API key, AWS secret key, or private SSH key | CWE-798 |
| <a id="sec-hub-001"></a>**`SEC-HUB-001`** | **HIGH** | Supply Chain | Repository only distributes unsafe pickle weights without SafeTensors | CWE-502 |
| <a id="sec-hub-002"></a>**`SEC-HUB-002`** | **HIGH** | Supply Chain | Standalone executable script file in Hub repository | CWE-507 |
| <a id="sec-hub-typo-001"></a>**`SEC-HUB-TYPO-001`** | **HIGH** | Supply Chain | Author typosquatting popular publishers on Hugging Face | Supply Chain |

---

## 🏛️ Regulatory & Security Framework Crosswalk

Every rule in `llm-scan` is systematically mapped to the four principal AI safety and cybersecurity governance frameworks:

| Scanner Rule | Threat Description | OWASP Top 10 for LLMs (2025) | MITRE ATLAS | NIST AI RMF (100-1) | EU AI Act (2024/1689) |
| :--- | :--- | :--- | :--- | :--- | :--- |
| <a id="sec-pck-001"></a>**`SEC-PCK-001`** | Deserialization RCE | **LLM03: Supply Chain** | **AML.T0010: Supply Chain** | **MANAGE-2.4 (Vetting)** | **Art. 15 / Art. 55 (Systemic)** |
| <a id="sec-zip-001"></a>**`SEC-ZIP-001`** | ZipSlip Path Traversal | **LLM03: Supply Chain** | **AML.T0010: Supply Chain** | **MANAGE-2.4 (Vetting)** | **Art. 15 (Cybersecurity)** |
| <a id="sec-rce-001"></a>**`SEC-RCE-001`** | Dangerous Module Imports | **LLM03: Supply Chain** | **AML.T0010: Supply Chain** | **MANAGE-2.4 (Vetting)** | **Art. 15 (Cybersecurity)** |
| <a id="sec-rce-002"></a>**`SEC-RCE-002`** | Dynamic Eval/Exec | **LLM03: Supply Chain** | **AML.T0048: Code Execution** | **MANAGE-2.4 (Vetting)** | **Art. 15 (Cybersecurity)** |
| <a id="sec-rce-003"></a>**`SEC-RCE-003`** | OS Command Execution | **LLM03: Supply Chain** | **AML.T0048: Code Execution** | **MANAGE-2.4 (Vetting)** | **Art. 15 (Cybersecurity)** |
| <a id="sec-rce-map"></a>**`SEC-RCE-MAP`** | `auto_map` Remote Code | **LLM03: Supply Chain** | **AML.T0010: Supply Chain** | **MANAGE-2.4 (Vetting)** | **Art. 53 (GPAI Docs)** |
| <a id="sec-tmpl-ssti"></a>**`SEC-TMPL-SSTI`** | Jinja2 SSTI (CVE-2024-34359)| **LLM03 / LLM01** | **AML.T0048: Code Execution** | **MEASURE-2.6 (Testing)** | **Art. 15 (Cybersecurity)** |
| <a id="sec-st-003"></a>**`SEC-ST-003`** | SafeTensors Memory Bomb | **LLM10: Unbounded Consumption** | **AML.T0029: Denial of Service**| **MEASURE-2.6 (Testing)** | **Art. 15 (Robustness)** |
| <a id="sec-onnx-001"></a>**`SEC-ONNX-001`** | ONNX Path Traversal | **LLM03: Supply Chain** | **AML.T0010: Supply Chain** | **MANAGE-2.4 (Vetting)** | **Art. 15 (Cybersecurity)** |
| <a id="sec-wgt-002"></a>**`SEC-WGT-002`** | Poisoned NaN/Inf Weights | **LLM04: Model Poisoning** | **AML.T0029: Denial of Service**| **MEASURE-2.6 (Testing)** | **Art. 15 / Art. 55 (Systemic)** |
| <a id="sec-wgt-003"></a>**`SEC-WGT-003`** | Sabotaged / Zero Weights | **LLM04: Model Poisoning** | **AML.T0018: Backdoor ML** | **MEASURE-2.6 (Testing)** | **Art. 15 (Robustness)** |
| <a id="sec-wgt-005"></a>**`SEC-WGT-005`** | Stealth Injected Layer | **LLM04: Model Poisoning** | **AML.T0018: Backdoor ML** | **MEASURE-2.6 (Testing)** | **Art. 15 (Cybersecurity)** |
| <a id="sec-tok-003"></a>**`SEC-TOK-003`** | Token ID Delimiter Hijack | **LLM01: Prompt Injection** | **AML.T0051: LLM Injection** | **MEASURE-2.6 (Testing)** | **Art. 15 (Cybersecurity)** |
| <a id="sec-tok-004"></a>**`SEC-TOK-004`** | Delimiter Marked Non-Special| **LLM01: Prompt Injection** | **AML.T0051: LLM Injection** | **MEASURE-2.6 (Testing)** | **Art. 15 (Cybersecurity)** |
| <a id="sec-tok-006"></a>**`SEC-TOK-006`** | Unicode Bidi Trojan Source | **LLM01: Prompt Injection** | **AML.T0051: LLM Injection** | **MEASURE-2.6 (Testing)** | **Art. 15 (Robustness)** |
| <a id="sec-gen-001"></a>**`SEC-GEN-001`** | Extreme Generation KV DoS | **LLM10: Unbounded Consumption** | **AML.T0029: Denial of Service**| **MEASURE-2.6 (Testing)** | **Art. 15 (Robustness)** |
| <a id="sec-peft-001"></a>**`SEC-PEFT-001`** | Suspicious LoRA Base Model | **LLM03: Supply Chain** | **AML.T0010: Supply Chain** | **MANAGE-2.4 (Vetting)** | **Art. 15 (Cybersecurity)** |
| <a id="sec-peft-002"></a>**`SEC-PEFT-002`** | LoRA Scaling Sabotage | **LLM04: Model Poisoning** | **AML.T0018: Backdoor ML** | **MEASURE-2.6 (Testing)** | **Art. 15 (Robustness)** |
| <a id="sec-art-002"></a>**`SEC-ART-002`** | Disguised ELF/PE Binaries | **LLM03: Supply Chain** | **AML.T0010: Supply Chain** | **MANAGE-2.4 (Vetting)** | **Art. 15 (Cybersecurity)** |
| <a id="sec-sec-001"></a>**`SEC-SEC-001`** | Leaked Secrets / API Keys | **LLM02: Sensitive Data Leak** | **AML.T0040: ML IP Theft** | **GOVERN-1.2 & MANAGE** | **Art. 15 & Art. 53 (Protection)** |

---

## 🧪 Validated Real-World Models

`llm-scan` has been validated against production foundation models on the Hugging Face Hub:

| Model Repository | Verdict | Risk Score | Key Findings & Verification |
| :--- | :---: | :---: | :--- |
| **`Qwen/Qwen2.5-0.5B-Instruct`** | `SAFE` | **0 / 100** | Zero findings. Native SafeTensors format, clean tokenizer, no arbitrary remote code. |
| **`gpt2`** | `MEDIUM_RISK` | **9 / 100** | Contains legacy PyTorch pickle weights (`pytorch_model.bin`). Flagged for modernization. |
| **`tiiuae/falcon-7b`** | `HIGH_RISK` | **45 / 100** | Declares `auto_map` in `config.json` requiring `trust_remote_code=True`. Verified via AST proof. |
| **`THUDM/chatglm-6b`** | `HIGH_RISK` | **92 / 100** | 5 high-severity findings: base64 decompression at `quantization.py:L30`, process spawning, `auto_map`. |
| **`NousResearch/Hermes-3-Llama-3.1-8B`** | `HIGH_RISK` | **45 / 100** | Role delimiter `<|im_start|>` configured as non-special token (ID 128039) in `tokenizer.json`. |

---

## 🐍 Python SDK Reference

Integrate `llm-scan` directly into your data pipelines, model ingestion workflows, or PyTorch inference servers:

```python
from llm_security_scanner import ModelSecurityScanner, ScannerConfig, SecurityVerdict

# Configure the security scanning engine
scanner = ModelSecurityScanner(
    config=ScannerConfig(
        scan_pickles=True,
        scan_remote_code=True,
        scan_chat_templates=True,
        scan_safetensors=True,
        scan_special_tokens=True,
        deep_weight_sampling=True,
    )
)

# Audit a local model directory or file
report = scanner.scan("./models/llama-3-8b")

print(f"Model: {report.model_path}")
print(f"Security Verdict: {report.verdict.value}")
print(f"Risk Score: {report.risk_score} / 100")

# Enforce security gates before model loading
if report.verdict in (SecurityVerdict.HIGH_RISK, SecurityVerdict.CRITICAL):
    print("🚨 Refusing to load model due to critical security findings:")
    for finding in report.findings:
        print(f" - [{finding.severity.value}] {finding.title} ({finding.file_path})")
        print(f"   Remediation: {finding.remediation}\n")
    raise SystemExit("Pipeline aborted: Model failed security gate.")

# Safe to proceed with torch.load or Transformers
print("✅ Security gate passed: Model verified safe for production loading.")
```

---

## 🔄 CI/CD GitHub Actions (SARIF Integration)

Integrate `llm-scan` into your GitHub CI/CD workflows. Security alerts will automatically appear in your repository's **GitHub Security > Code Scanning Alerts** tab:

```yaml
name: LLM Model Security Audit

on:
  pull_request:
    paths:
      - 'models/**'
  push:
    branches: [main]

jobs:
  model-security-audit:
    name: Model Security Scan & SARIF Upload
    runs-on: ubuntu-latest

    steps:
      - name: Checkout Repository
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'
          cache: 'pip'

      - name: Install Scanner
        run: |
          pip install --upgrade pip
          pip install -e .

      - name: Execute Model Security Scan
        run: |
          llm-scan scan ./models/candidate-model \
            --format sarif \
            --output results.sarif \
            --max-severity HIGH

      - name: Upload SARIF to GitHub Security Tab
        uses: github/codeql-action/upload-sarif@v3
        if: always()
        with:
          sarif_file: results.sarif
```

---

## 🌐 Threat Model Lifecycle

`llm-scan` is architected around the full 5-stage Machine Learning Security Lifecycle:

```
  Phase 1: Training Data Collection
  ├── Web scraping cache poisoning & Clean-label Trojan injections
  └── PII / Secret leakage in pre-training corpora

  Phase 2: Training & Model Architecture
  ├── Malicious custom CUDA / Triton kernels
  ├── Sleeper Agent backdoor weights & Trojan gates
  └── LoRA safety-alignment stripping

  Phase 3: Serialization & Distribution (Core Target of llm-scan)
  ├── Weaponized pickle bytecode RCE (torch.load)
  ├── trust_remote_code=True auto_map dynamic execution
  ├── Jinja2 Chat Template SSTI (CVE-2024-34359)
  ├── SafeTensors & GGUF header memory allocation bombs
  ├── ONNX external data path traversal (CVE-2024-27318)
  ├── Tokenizer role delimiter hijacking & Unicode Bidi smuggling
  └── Disguised ELF / PE binaries & hardcoded credentials

  Phase 4: Deployment & Infrastructure
  ├── Inference engine memory corruption (vLLM, llama.cpp)
  └── GPU memory exhaustion via oversized token generation configs

  Phase 5: Post-Deployment & Inference
  ├── Direct & Indirect Prompt Injections
  └── Jailbreaking & Autonomous Agent tool abuse (SSRF)
```

Run `llm-scan threat-model` in your terminal to view details on every lifecycle phase.

---

## 🧪 Running the Test Suite

The test suite covers all 13 scanner modules, the verification engine, and SARIF export:

```bash
# Run all unit and integration tests
python3 -m unittest discover -s tests -v
```

Expected output:
```
Ran 35 tests in 0.030s

OK
```

All 35 tests execute without executing dangerous bytecode or downloading external files.

---

## 👤 Author & Contact

**Mesut Güngör**  
- GitHub: [@mesutgungor](https://github.com/mesutgungor)
- Repository: [https://github.com/mesutgungor/llm_model_security](https://github.com/mesutgungor/llm_model_security)

---

## 📄 License

This project is licensed under the **Apache License 2.0**. See the [LICENSE](LICENSE) file for complete terms and copyright notices.
