Metadata-Version: 2.3
Name: modelaudit
Version: 0.2.1
Summary: Static scanning library for detecting malicious code, backdoors, and other security risks in ML model files
Project-URL: Repository, https://github.com/promptfoo/modelaudit
Project-URL: Homepage, https://github.com/promptfoo/modelaudit
Author-email: Ian Webster <ian@promptfoo.dev>, Michael D'Angelo <michael@promptfoo.dev>
License: MIT
Keywords: ai,ml,model-scanning,pickle,pytorch,security,tensorflow
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Security
Requires-Python: >=3.9
Requires-Dist: click>=8.1.7
Requires-Dist: cyclonedx-python-lib>=10.2.0
Requires-Dist: defusedxml>=0.7.1
Requires-Dist: huggingface-hub>=0.23.0
Requires-Dist: numpy>=1.19.0
Requires-Dist: python-dotenv>=1.0.0
Requires-Dist: requests>=2.28.0
Requires-Dist: scipy>=1.7.0
Requires-Dist: yaspin>=2.5.0
Provides-Extra: all
Requires-Dist: coremltools>=8.0; extra == 'all'
Requires-Dist: dill<1.0,>=0.3.0; extra == 'all'
Requires-Dist: fsspec>=2025.5.1; extra == 'all'
Requires-Dist: gcsfs>=2025.5.1; extra == 'all'
Requires-Dist: h5py<4.0,>=3.1; extra == 'all'
Requires-Dist: huggingface-hub>=0.23.0; extra == 'all'
Requires-Dist: joblib<2.0,>=1.0.0; extra == 'all'
Requires-Dist: mlflow>=2.12.0; extra == 'all'
Requires-Dist: msgpack<2.0,>=1.0.0; extra == 'all'
Requires-Dist: onnx<2.0,>=1.12.0; extra == 'all'
Requires-Dist: pyyaml<7.0,>=6.0; extra == 'all'
Requires-Dist: s3fs>=2025.5.1; extra == 'all'
Requires-Dist: safetensors>=0.4.0; extra == 'all'
Requires-Dist: scikit-learn<2.0,>=1.0.0; extra == 'all'
Requires-Dist: tensorflow<3.0,>=2.13.0; extra == 'all'
Requires-Dist: tensorrt>=8.6.0; extra == 'all'
Requires-Dist: tflite>=2.18.0; extra == 'all'
Requires-Dist: torch<3.0,>=2.6.0; extra == 'all'
Provides-Extra: all-ci
Requires-Dist: coremltools>=8.0; extra == 'all-ci'
Requires-Dist: dill<1.0,>=0.3.0; extra == 'all-ci'
Requires-Dist: fsspec>=2025.5.1; extra == 'all-ci'
Requires-Dist: gcsfs>=2025.5.1; extra == 'all-ci'
Requires-Dist: h5py<4.0,>=3.1; extra == 'all-ci'
Requires-Dist: huggingface-hub>=0.23.0; extra == 'all-ci'
Requires-Dist: joblib<2.0,>=1.0.0; extra == 'all-ci'
Requires-Dist: mlflow>=2.12.0; extra == 'all-ci'
Requires-Dist: msgpack<2.0,>=1.0.0; extra == 'all-ci'
Requires-Dist: onnx<2.0,>=1.12.0; extra == 'all-ci'
Requires-Dist: pyyaml<7.0,>=6.0; extra == 'all-ci'
Requires-Dist: s3fs>=2025.5.1; extra == 'all-ci'
Requires-Dist: safetensors>=0.4.0; extra == 'all-ci'
Requires-Dist: scikit-learn<2.0,>=1.0.0; extra == 'all-ci'
Requires-Dist: tensorflow<3.0,>=2.13.0; extra == 'all-ci'
Requires-Dist: tflite>=2.18.0; extra == 'all-ci'
Requires-Dist: torch<3.0,>=2.6.0; extra == 'all-ci'
Provides-Extra: cloud
Requires-Dist: fsspec>=2025.5.1; extra == 'cloud'
Requires-Dist: gcsfs>=2025.5.1; extra == 'cloud'
Requires-Dist: s3fs>=2025.5.1; extra == 'cloud'
Provides-Extra: coreml
Requires-Dist: coremltools>=8.0; extra == 'coreml'
Provides-Extra: dill
Requires-Dist: dill<1.0,>=0.3.0; extra == 'dill'
Provides-Extra: flax
Requires-Dist: msgpack<2.0,>=1.0.0; extra == 'flax'
Provides-Extra: h5
Requires-Dist: h5py<4.0,>=3.1; extra == 'h5'
Provides-Extra: huggingface
Requires-Dist: huggingface-hub>=0.23.0; extra == 'huggingface'
Provides-Extra: joblib
Requires-Dist: joblib<2.0,>=1.0.0; extra == 'joblib'
Requires-Dist: scikit-learn<2.0,>=1.0.0; extra == 'joblib'
Provides-Extra: mlflow
Requires-Dist: mlflow>=2.12.0; extra == 'mlflow'
Provides-Extra: numpy1
Requires-Dist: coremltools>=8.0; extra == 'numpy1'
Requires-Dist: dill<1.0,>=0.3.0; extra == 'numpy1'
Requires-Dist: fsspec>=2025.5.1; extra == 'numpy1'
Requires-Dist: gcsfs>=2025.5.1; extra == 'numpy1'
Requires-Dist: h5py<4.0,>=3.1; extra == 'numpy1'
Requires-Dist: huggingface-hub>=0.23.0; extra == 'numpy1'
Requires-Dist: joblib<2.0,>=1.0.0; extra == 'numpy1'
Requires-Dist: msgpack<2.0,>=1.0.0; extra == 'numpy1'
Requires-Dist: numpy<2.0,>=1.19.0; extra == 'numpy1'
Requires-Dist: onnx<2.0,>=1.12.0; extra == 'numpy1'
Requires-Dist: pyyaml<7.0,>=6.0; extra == 'numpy1'
Requires-Dist: s3fs>=2025.5.1; extra == 'numpy1'
Requires-Dist: safetensors>=0.4.0; extra == 'numpy1'
Requires-Dist: scikit-learn<2.0,>=1.0.0; extra == 'numpy1'
Requires-Dist: tensorflow<3.0,>=2.13.0; extra == 'numpy1'
Requires-Dist: tensorrt>=8.6.0; extra == 'numpy1'
Requires-Dist: tflite>=2.18.0; extra == 'numpy1'
Requires-Dist: torch<3.0,>=2.6.0; extra == 'numpy1'
Provides-Extra: onnx
Requires-Dist: onnx<2.0,>=1.12.0; extra == 'onnx'
Provides-Extra: pytorch
Requires-Dist: torch<3.0,>=2.6.0; extra == 'pytorch'
Provides-Extra: safetensors
Requires-Dist: safetensors>=0.4.0; extra == 'safetensors'
Provides-Extra: tensorflow
Requires-Dist: tensorflow<3.0,>=2.13.0; extra == 'tensorflow'
Provides-Extra: tensorrt
Requires-Dist: tensorrt>=8.6.0; extra == 'tensorrt'
Provides-Extra: tflite
Requires-Dist: tflite>=2.18.0; extra == 'tflite'
Provides-Extra: yaml
Requires-Dist: pyyaml<7.0,>=6.0; extra == 'yaml'
Description-Content-Type: text/markdown

# ModelAudit

**Secure your AI models before deployment.** Detects malicious code, backdoors, and security vulnerabilities in ML model files.

[![PyPI version](https://badge.fury.io/py/modelaudit.svg)](https://pypi.org/project/modelaudit/)
[![Python versions](https://img.shields.io/pypi/pyversions/modelaudit.svg)](https://pypi.org/project/modelaudit/)
[![Code Style: ruff](https://img.shields.io/badge/code%20style-ruff-005cd7.svg)](https://github.com/astral-sh/ruff)
[![License](https://img.shields.io/github/license/promptfoo/promptfoo)](https://github.com/promptfoo/promptfoo/blob/main/LICENSE)

<img width="989" alt="image" src="https://www.promptfoo.dev/img/docs/modelaudit/modelaudit-result.png" />

📖 **[Full Documentation](https://www.promptfoo.dev/docs/model-audit/)** | 🎯 **[Usage Examples](https://www.promptfoo.dev/docs/model-audit/usage/)** | 🔍 **[Supported Formats](https://www.promptfoo.dev/docs/model-audit/scanners/)**

## 🚀 Quick Start

**Install and scan in 30 seconds:**

```bash
# Install ModelAudit with all ML framework support
pip install modelaudit[all]

# Scan a model file
modelaudit model.pkl

# Scan a directory
modelaudit ./models/

# Export results for CI/CD
modelaudit model.pkl --format json --output results.json
```

**Example output:**

```bash
$ modelaudit suspicious_model.pkl

✓ Scanning suspicious_model.pkl
Files scanned: 1 | Issues found: 2 critical, 1 warning

1. suspicious_model.pkl (pos 28): [CRITICAL] Malicious code execution attempt
   Why: Contains os.system() call that could run arbitrary commands

2. suspicious_model.pkl (pos 52): [WARNING] Dangerous pickle deserialization
   Why: Could execute code when the model loads

✗ Security issues found - DO NOT deploy this model
```

## 🛡️ What Problems It Solves

### **Prevents Code Execution Attacks**

Stops malicious models that run arbitrary commands when loaded (common in PyTorch .pt files)

### **Detects Model Backdoors**

Identifies trojaned models with hidden functionality or suspicious weight patterns

### **Ensures Supply Chain Security**

Validates model integrity and prevents tampering in your ML pipeline

### **Enforces License Compliance**

Checks for license violations that could expose your company to legal risk

## 📊 Supported Model Formats

ModelAudit scans **all major ML model formats** with specialized security analysis for each:

| Format          | Extensions                            | Risk Level | Notes                                        |
| --------------- | ------------------------------------- | ---------- | -------------------------------------------- |
| **PyTorch**     | `.pt`, `.pth`, `.ckpt`, `.bin`        | 🔴 HIGH    | Contains pickle serialization - always scan  |
| **Pickle**      | `.pkl`, `.pickle`, `.dill`            | 🔴 HIGH    | Avoid in production - convert to SafeTensors |
| **Joblib**      | `.joblib`                             | 🔴 HIGH    | Can contain pickled objects                  |
| **SafeTensors** | `.safetensors`                        | 🟢 SAFE    | Preferred secure format                      |
| **GGUF/GGML**   | `.gguf`, `.ggml`                      | 🟢 SAFE    | LLM standard, binary format                  |
| **ONNX**        | `.onnx`                               | 🟢 SAFE    | Industry standard, good interoperability     |
| **TensorFlow**  | `.pb`, SavedModel                     | 🟠 MEDIUM  | Scan for dangerous operations                |
| **Keras**       | `.h5`, `.keras`, `.hdf5`              | 🟠 MEDIUM  | Check for executable layers                  |
| **JAX/Flax**    | `.msgpack`, `.flax`, `.orbax`, `.jax` | 🟡 LOW     | Validate transforms                          |

Plus 10+ additional formats including ExecuTorch, TensorFlow Lite, Core ML, and more.

[View complete format documentation →](https://www.promptfoo.dev/docs/model-audit/scanners/)

## 🎯 Common Use Cases

### **Pre-Deployment Security Checks**

```bash
modelaudit production_model.safetensors --format json --output security_report.json
```

### **CI/CD Pipeline Integration**

ModelAudit automatically detects CI environments and adjusts output accordingly:

```bash
# Recommended: Use JSON format for machine-readable output
modelaudit models/ --format json --output results.json

# Text output automatically adapts to CI (no spinners, plain text)
modelaudit models/ --timeout 300

# Disable colors explicitly with NO_COLOR environment variable
NO_COLOR=1 modelaudit models/
```

**CI-Friendly Features:**

- 🚫 Spinners automatically disabled when output is piped or in CI
- 🎨 Colors disabled when `NO_COLOR` environment variable is set
- 📊 JSON output recommended for parsing in CI pipelines
- 🔍 Exit codes: 0 (clean), 1 (issues found), 2 (errors)

### **Third-Party Model Validation**

```bash
# Scan models from HuggingFace, PyTorch Hub, or cloud storage
modelaudit https://huggingface.co/gpt2
modelaudit https://pytorch.org/hub/pytorch_vision_resnet/
modelaudit s3://my-bucket/downloaded-model.pt
modelaudit https://company.jfrog.io/artifactory/repo/model.pt \
    --jfrog-api-token YOUR_TOKEN
```

### **Compliance & Audit Reporting**

```bash
modelaudit model_package.zip --sbom compliance_report.json --verbose
```

[View advanced usage examples →](https://www.promptfoo.dev/docs/model-audit/usage/)

### Static Scanning vs. Promptfoo Redteaming

ModelAudit performs **static** analysis only. It examines model files for risky patterns
without ever loading or executing them. Promptfoo's redteaming module is
**dynamic**—it loads the model (locally or via API) and sends crafted prompts to
probe runtime behavior. Use ModelAudit first to verify the model file itself,
then run redteaming if you need to test how the model responds when invoked.

## ⚙️ Installation Options

**Basic installation (recommended for most users):**

### Quick Install Decision Guide

**🚀 Just want everything to work?**

```bash
pip install modelaudit[all]
```

**💡 Know what formats you need?**

```bash
# Basic installation (pickle, joblib, numpy, zip/tar archives)
pip install modelaudit

# Add only what you need
pip install modelaudit[tensorflow]  # TensorFlow SavedModel (.pb)
pip install modelaudit[pytorch]     # PyTorch models (.pt, .pth)
pip install modelaudit[h5]          # Keras/H5 models (.h5, .keras)
pip install modelaudit[onnx]        # ONNX models (.onnx)
pip install modelaudit[safetensors] # SafeTensors (.safetensors)

# Multiple formats
pip install modelaudit[tensorflow,pytorch,h5]
```

**☁️ Need cloud storage support?**

```bash
pip install modelaudit[cloud]  # S3, GCS, and Azure support
```

**⚠️ Having NumPy compatibility issues?**

```bash
# Some ML frameworks require NumPy < 2.0
pip install modelaudit[numpy1]

# Check what's working
modelaudit doctor --show-failed
```

**Docker installation:**

```bash
docker pull ghcr.io/promptfoo/modelaudit:latest
docker run --rm -v $(pwd):/data ghcr.io/promptfoo/modelaudit:latest model.pkl
```

### 📦 Dependency Reference

<details>
<summary><b>View all available extras and what they include</b></summary>

| Extra           | Includes                      | Use When                                |
| --------------- | ----------------------------- | --------------------------------------- |
| `[tensorflow]`  | TensorFlow framework          | Scanning `.pb` SavedModel files         |
| `[pytorch]`     | PyTorch framework             | Scanning `.pt`, `.pth` files            |
| `[h5]`          | h5py library                  | Scanning `.h5`, `.keras`, `.hdf5` files |
| `[onnx]`        | ONNX runtime                  | Scanning `.onnx` model files            |
| `[safetensors]` | SafeTensors library           | Scanning `.safetensors` files           |
| `[flax]`        | msgpack for JAX/Flax          | Scanning `.msgpack`, `.flax` files      |
| `[cloud]`       | fsspec, s3fs, gcsfs           | Scanning from S3, GCS, Azure            |
| `[mlflow]`      | MLflow library                | Scanning MLflow model registry          |
| `[all]`         | All ML frameworks             | Maximum compatibility                   |
| `[numpy1]`      | All ML frameworks + NumPy<2.0 | When facing NumPy conflicts             |

</details>

## 📋 Output Formats

**Human-readable output (default):**

```bash
$ modelaudit model.pkl

✓ Scanning model.pkl
Files scanned: 1 | Issues found: 1 critical

1. model.pkl (pos 28): [CRITICAL] Malicious code execution attempt
   Why: Contains os.system() call that could run arbitrary commands
```

**JSON output for automation:**

```json
{
  "files_scanned": 1,
  "issues": [
    {
      "message": "Malicious code execution attempt",
      "severity": "critical",
      "location": "model.pkl (pos 28)"
    }
  ]
}
```

## 🔧 Getting Help

- **Documentation**: [promptfoo.dev/docs/model-audit/](https://www.promptfoo.dev/docs/model-audit/)
- **Troubleshooting**: [promptfoo.dev/docs/model-audit/troubleshooting/](https://www.promptfoo.dev/docs/model-audit/troubleshooting/)
- **Issues**: [github.com/promptfoo/modelaudit/issues](https://github.com/promptfoo/modelaudit/issues)

### 🔍 Troubleshooting Common Issues

**Scanner not working?**

```bash
# Check which scanners are available
modelaudit doctor --show-failed
```

**NumPy compatibility errors?**

```bash
# Option 1: Use the numpy1 compatibility mode
pip install modelaudit[numpy1]

# Option 2: Manually downgrade NumPy
pip install "numpy<2.0" --force-reinstall
pip install --force-reinstall tensorflow torch h5py  # Reinstall ML frameworks
```

**Missing scanner for your format?**

```bash
# ModelAudit will tell you exactly what to install
modelaudit your-model.onnx
# Output: "onnx not installed, cannot scan ONNX files. Install with 'pip install modelaudit[onnx]'"
```

## 📝 License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
