Metadata-Version: 2.4
Name: vlm-occlusion
Version: 0.1.0
Summary: Find which image regions influence a VLM claim through grid-based occlusion testing.
Author-email: "Eduardo J. Barrios" <edujbarrios@outlook.com>
License-Expression: MPL-2.0
Project-URL: Repository, https://github.com/edujbarrios/vlm-occlusion
Project-URL: Issues, https://github.com/edujbarrios/vlm-occlusion/issues
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: Pillow>=9.0
Provides-Extra: dev
Requires-Dist: build>=1.0; extra == "dev"
Requires-Dist: pytest>=7.0; extra == "dev"
Dynamic: license-file

# vlm-occlusion

Find which image regions influence a Vision-Language Model claim through black-box occlusion testing.

## Installation

```bash
pip install vlm-occlusion
```

## Usage

```python
from vlm_occlusion import analyze_occlusion

result = analyze_occlusion(
    image="example.png",
    predict_fn=describe_image,
    score_fn=claim_score,
    grid_size=(4, 4),
    occlusion="blur",
    output_path="heatmap.png",
)

print(result["most_influential_region"])
print(result["importance_scores"])
```

`predict_fn` receives a Pillow image and returns text or a number. `score_fn` turns that result into a numerical score for the claim you are testing. The package compares the original score with scores produced after occluding each grid cell.

It is model-agnostic and includes no VLM, model adapter, or network client. It measures black-box sensitivity to controlled perturbations; it does not inspect attention or reveal a model's internal reasoning.

## Example

The included deterministic example tests the claim:

> There is a red desk lamp in the upper-left area.

The mock predictor measures visible red evidence; no model, API key, or network connection is used. The 4×4 analysis identifies the left-side cell containing most of the lamp as the strongest influence.

| Input image | Most influential cell occluded | Importance heatmap |
|---|---|---|
| ![A detailed robotics workbench with a red lamp in the upper-left](https://raw.githubusercontent.com/edujbarrios/vlm-occlusion/main/docs/images/robotics_workbench_input.png) | ![The same workbench with the strongest lamp region occluded in black](https://raw.githubusercontent.com/edujbarrios/vlm-occlusion/main/docs/images/robotics_workbench_occluded.png) | ![Occlusion importance heatmap highlighting the strongest lamp region](https://raw.githubusercontent.com/edujbarrios/vlm-occlusion/main/docs/images/robotics_workbench_heatmap.png) |

The scene is an original generated illustration created for this repository. The occluded image and heatmap are produced locally by the example.

```bash
python examples/generate_example_images.py
python examples/basic_usage.py
```

The README assets are saved under `docs/images/`.

## Features

- Model-agnostic callback API
- Grid-based black, white, mean, and blur occlusion
- Normalized regional importance scores
- Pillow-only visualization
- No network access required

## Limitations

Occlusion sensitivity is an approximation. A perturbation may change a response for reasons unrelated to visual grounding, and results can vary with grid size and occlusion strategy. This package does not expose internal model attention.

## Issues

Report issues at https://github.com/edujbarrios/vlm-occlusion/issues.

## Author

Eduardo J. Barrios — [edujbarrios@outlook.com](mailto:edujbarrios@outlook.com)

## License

Mozilla Public License 2.0
