Metadata-Version: 2.4
Name: object-remove
Version: 0.2.0
Summary: From-scratch reference implementation of a tiered on-device object-removal pipeline, CPU and GPU
Author: shalaga44
License-Expression: MIT
Project-URL: Homepage, https://github.com/shalaga44/object_remove
Project-URL: Issues, https://github.com/shalaga44/object_remove/issues
Keywords: image-processing,inpainting,object-removal,computer-vision
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Image Processing
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=2.0
Requires-Dist: scipy>=1.10
Requires-Dist: Pillow>=9.0
Requires-Dist: torch>=2.2
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == "dev"
Provides-Extra: web
Requires-Dist: Flask>=2.2; extra == "web"
Dynamic: license-file

# object_remove

A from scratch implementation of an on device object removal pipeline (see
[`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) for the full design): a
polygon/scanline mask layer, **four fill engines** of increasing
sophistication, **two interchangeable blend backends**, GPU acceleration
(CUDA + Apple MPS) that engages automatically when available, and a
locked/queued session persistence layer.

Everything here is implemented from primitives (FAST corners, BRIEF descriptors,
Hamming matching, a quad-tree patch-grid solver, PatchMatch, Laplacian/Poisson
blending) — no OpenCV, no learned weights.

## Quick start

```bash
pip install object-remove
```

```python
import numpy as np
from object_remove import RemoveObjectJob, EngineTier

result = RemoveObjectJob(image_rgb).run(mask, tier=EngineTier.AUTO,
                                        progress=lambda stage, frac: ...)
removed = result.image          # float32 RGB in [0, 1]
print(result.tier_used)         # which engine AUTO picked
```

`image_rgb` is any `(H, W, 3)` array (uint8 or float); `mask` is `(H, W)` with
nonzero marking the object to remove. Every engine and blender defaults to
`device="auto"`: use a GPU (CUDA on NVIDIA, MPS on Apple Silicon) when one is
available, otherwise the plain NumPy path — no separate install, no manual
opt in. Pass `"cuda"`/`"mps"`/`"cpu"` to force a specific torch backend, or
`device=None` to force plain NumPy regardless of hardware — see
[GPU acceleration](#gpu-acceleration) below.

**New to the package? Start with [`docs/USAGE.md`](docs/USAGE.md)** — it walks
through loading an image, building a mask three different ways (brush stroke,
polygon selection, tap-to-auto-select), refining it, running the removal, and
undo/session history, with runnable snippets for each.

To run from a source checkout instead of the installed package:

```bash
pip install -r requirements.txt          # numpy, scipy, Pillow, torch
PYTHONPATH=. python examples/demo.py     # writes before/after PNGs to examples/out/
PYTHONPATH=. python -m pytest -q         # tests
```

## Module layout

| Stage | Concern | This repo |
|---|---|---|
| A — mask | selection, brush, auto select | `object_remove/mask/` |
| B/D — scale/tiles | pyramid + tile scheduling | `object_remove/pyramid/` |
| C0 — legacy CPU fill | segment aware patch comparator | `fillengines/patch_comparator.py` |
| C1 — general solver | quad-tree patch-grid solve | `fillengines/primary/` |
| C2 — texture synthesis | self similar rigid shift | `fillengines/self_similar_shift.py` |
| C3 — patch match | multi scale PatchMatch | `fillengines/patchmatch/` |
| E — blend | multi band + Poisson | `object_remove/render/` |
| F — session | undo / persistence | `object_remove/session/` |
| — GPU runtime | device resolution | `object_remove/runtime/device.py` |
| — orchestration | tier selection, progress, cancellation | `pipeline/remove_object_job.py` |

## The four fill engines

- **Tier 1 comparator — `SegmentAwarePatchFillEngine`** — greedy onion peel
  exemplar fill scored by segment aware SSD with byte mask validity. Cheap,
  correct on small holes; the low end / tiny hole fallback.
- **Tier 1 primary — `PrimaryFillEngine`** — the general purpose solver: a
  quad-tree patch search (`fillengines/primary/patch_search_tree.py`) over a
  **sparse grid of patch centres** (not a dense per-pixel field), solved in
  onion peel order with adaptive, blur-tested scale selection instead of a
  fixed pyramid formula.
- **Tier 2 — `SelfSimilarShiftEngine`** — the fast default for repetitive texture.
  FAST corners + 256 bit BRIEF + Hamming matching find a *rigid self similar
  offset*; a 1D DP seam cuts the copied block in cleanly, run for x then y via
  a transpose trick.
- **Tier 3 — `PatchMatchEngine`** — multi scale, multi group PatchMatch inpainting
  (random init → propagation → random search, then patch voting reconstruction,
  coarse to fine). Available for explicit selection.

`EngineTier.AUTO` sends truly tiny holes to tier 1 comparator, otherwise
tries the cheap tier 2 rigid shift first and falls back to tier 1 primary
when the shift's boundary continuity score is poor.

## GPU acceleration

`torch` ships as a regular dependency — one `pip install object-remove` gets
both CPU and GPU support, nothing extra to install. Every fill engine and
both blenders accept a `device` kwarg, defaulting to `"auto"`: use a GPU
(CUDA on NVIDIA, MPS on Apple Silicon) when one is available, otherwise the
plain NumPy path. Explicit `"cuda"`/`"mps"`/`"cpu"` force that exact torch
backend (and raise clearly if it isn't available); `device=None` forces
plain NumPy regardless of hardware.

```python
from object_remove.render.multiband_blender import MultiBandBlender
from object_remove.fillengines import PrimaryFillEngine

blender = MultiBandBlender(bands=3)          # device="auto" by default
engine = PrimaryFillEngine(device=None)      # force plain NumPy
```

Notes on the GPU path:

- The multi-band blender is a straight port (pure array math). The Poisson
  blender solves the identical linear system via matrix-free conjugate
  gradient instead of an exact sparse solve. The PatchMatch solver uses
  red-black checkerboard propagation instead of raster order, so GPU output
  legitimately differs pixel for pixel from the CPU path (same quality bar,
  different but equally valid traversal order).
- The torch path runs in **float32** throughout (Apple MPS has weak/no
  float64 support) — a deliberate precision tradeoff vs. the plain NumPy
  path's float64.

## The two blend backends

- **`MultiBandBlender`** — Laplacian pyramid blend with a fixed **4 tap
  kernel**, `[0.15439, 0.15133, 0.14252, 0.12896]`, and the mask feathered
  *per pyramid level*.
- **`PoissonBlend` / `PoissonBlend2`** — gradient domain blend (`sigma`,
  `multiplier`, `algo_n` 1|2 variant), solved as a sparse Poisson system.
  Poisson is the safe default for single image inpainting because it reads
  only the (valid) hole boundary; the compositor repairs the background under
  the hole so no backend ever samples the object being removed.

## Session persistence

`SessionManager` gives each edit a session id and an RLE mask history. Directory
removal is dispatched onto a background queue and run **under a lock** — never
an inline delete from the caller. A separate `EmptySessionSweeper` GCs orphaned
empty session dirs left by aborted/crashed edits.

## Beyond object removal

A handful of adjacent features share the same primitives as object removal
rather than each reimplementing their own:

- **`object_remove/cv/`** — reusable classic CV building blocks: ORB
  (FAST + BRIEF + Hamming, also used by tier 2), Harris/Shi-Tomasi corners,
  Canny edges, a distance transform, a separable max filter / non-max
  suppression, a generic sigma-configurable Gaussian/box blur, nearest-opaque
  fill, frequency separation.
- **`object_remove/effects/`** — `background_blur` (selection-distance
  driven falloff, since no depth model ships) and
  `FourPointPerspectiveCorrector` (4-point homography + inverse warp).
- **`fillengines/clone_stamp.py`** — manual clone stamp: a user-specified
  source offset instead of a searched one, composing with `PatchCompositor`
  like any other engine.
- **`fillengines/wire_removal.py`** — wire/cable removal: a *derived*
  (Hessian ridge traced), not painted, mask acquisition model, feeding the
  same fill engines used for object removal.

## Status & scope

Implemented and tested: the full pipeline across all stages, CPU (NumPy) and
optional GPU (PyTorch: CUDA + Apple MPS) backends for every fill engine and
blender, plus the adjacent features above. Auto select is explicitly an
*interface + CPU fallback*: a real segmentation model can be dropped in
behind `AutoSelectModel`, which ships a working color flood fallback so tap
to select runs end to end without one.

```
object_remove/
  mask/        brush_rasterizer, polygon_selection, selection_enhancer,
               connected_components, auto_select
  pyramid/     image_pyramid, scale_scheduler, tile_scheduler, torch_pyramid
  fillengines/ patch_comparator (T1 comparator), primary/ (T1 general solver),
               self_similar_shift (T2), patchmatch/ (T3),
               clone_stamp, wire_removal
               each fill engine has a *_gpu.py / torch_*.py sibling module
  render/      multiband_blender(+_gpu), poisson_blender(+_gpu), compositor,
               fused_patch_blend
  cv/          orb, harris, canny, distance_transform, max_finder,
               symmetric_convolution, box_filter, nearest_opaque,
               frequency_separation
  effects/     background_blur, perspective_correction
  runtime/     device (GPU device resolution)
  session/     undo_session_manager, empty_session_sweeper
  pipeline/    remove_object_job
```
