Metadata-Version: 2.4
Name: smart-bbox
Version: 0.1.22
Summary: Official SMART pipeline for tight 3D bounding boxes
Author: Chanhyeok Park
License-Expression: CC-BY-NC-SA-4.0
Project-URL: Homepage, https://github.com/chpark1111/SMART
Project-URL: Repository, https://github.com/chpark1111/SMART
Project-URL: Paper, https://arxiv.org/abs/2304.04336
Keywords: 3d,bounding-boxes,shape-abstraction,mesh,mcts,c++
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: C++
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Scientific/Engineering :: Image Processing
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy
Requires-Dist: PyYAML
Provides-Extra: pipeline
Requires-Dist: scipy; extra == "pipeline"
Requires-Dist: scikit-learn; extra == "pipeline"
Requires-Dist: trimesh; extra == "pipeline"
Requires-Dist: torch; extra == "pipeline"
Requires-Dist: tensorboard; extra == "pipeline"
Requires-Dist: tqdm; extra == "pipeline"
Requires-Dist: typing_extensions; extra == "pipeline"
Requires-Dist: coacd; extra == "pipeline"
Requires-Dist: rtree; extra == "pipeline"
Requires-Dist: matplotlib; extra == "pipeline"
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: cibuildwheel>=2.22; extra == "dev"
Requires-Dist: twine; extra == "dev"
Requires-Dist: wheel; extra == "dev"
Dynamic: license-file
Dynamic: requires-python

# SMART: Split, Merge, and Refine

Official implementation of **Split, Merge, and Refine: Fitting Tight Bounding
Boxes via Over-Segmentation and Iterative Search**, 3DV 2024.

![SMART teaser](https://raw.githubusercontent.com/chpark1111/SMART/main/docs/teaser.png)

[Paper](https://arxiv.org/pdf/2304.04336) |
[arXiv](https://arxiv.org/abs/2304.04336) |
[Quickstart](https://github.com/chpark1111/SMART/blob/main/docs/QUICKSTART.md) |
[Pipeline](https://github.com/chpark1111/SMART/blob/main/docs/PIPELINE.md) |
[Package Docs](https://github.com/chpark1111/SMART/blob/main/docs/PYTHON_PACKAGE.md) |
[All Docs](#documentation)

[Chanhyeok Park](https://chpark1111.github.io/) and
[Minhyuk Sung](https://mhsung.github.io/)

SMART fits a compact set of tight 3D bounding boxes to a mesh without human
supervision. The package uses a Python CLI/API with a native C++ SMART core and
the fixed vendored Manifold backend.

## Highlights

- Paper-style pipeline: normalize mesh, tetrahedralize, pre-segment, merge,
  refine, run MCTS, render, and evaluate.
- Native C++ backend for the SMART core through `smart._cpp` and
  `smart-cpp-native`.
- Fixed vendored Manifold source is kept unchanged for exact geometry
  operations.
- Python API and CLI for reproducible experiments and package use.
- ShapeNet airplane/chair/table reproduction configs are included.

## Installation

Install the package:

```bash
python -m pip install "smart-bbox[pipeline]"
```

Install from source:

```bash
git clone https://github.com/chpark1111/SMART.git
cd SMART
python -m pip install -e ".[pipeline]"
```

Verify the install:

```bash
smart --config configs/smoke_5.yaml doctor
smart-cpp-native --help
```

For the complete install and reproduction path, see
[`docs/QUICKSTART.md`](https://github.com/chpark1111/SMART/blob/main/docs/QUICKSTART.md).

## Documentation

Main user docs:

- [Quickstart](https://github.com/chpark1111/SMART/blob/main/docs/QUICKSTART.md): install, verify, prepare tools/data, and run
  a small reproduction.
- [Pipeline](https://github.com/chpark1111/SMART/blob/main/docs/PIPELINE.md): stage order, config control, rendering, failure
  handling, and parameter overrides.
- [Python Package](https://github.com/chpark1111/SMART/blob/main/docs/PYTHON_PACKAGE.md): CLI, Python API, native executable,
  packaged configs, and library usage.
- [Tetra Failure Playbook](https://github.com/chpark1111/SMART/blob/main/docs/TETRA_FAILURE_PLAYBOOK.md): why Mesh2Tet/fTetWild
  fails, how SMART records failures, and which repair knobs to try.
- [Repository Structure](https://github.com/chpark1111/SMART/blob/main/docs/REPOSITORY_STRUCTURE.md): public release layout
  versus ignored local data, runs, external tools, and experiments.

Maintainer and research docs:

- [Release Guide](https://github.com/chpark1111/SMART/blob/main/docs/RELEASE.md): local release preflight, wheel checks, tags,
  and PyPI publishing.
- [Release Notes 0.1.0](https://github.com/chpark1111/SMART/blob/main/docs/RELEASE_NOTES_0.1.0.md): current release scope and
  verification notes.
- [Learned Router Release Snapshot](https://github.com/chpark1111/SMART/blob/main/docs/LEARNED_ROUTER_RELEASE.md): current
  packaged learned SMART+Agent path, accuracy/speed gates, and Transformer comparison.
- [Learned Geometry Router](https://github.com/chpark1111/SMART/blob/main/docs/LEARNED_ROUTER.md): packaged DeepSets
  refine router, hard-state gates, exact-call reduction, and quality reinvestment experiments.
- [Research Plan](https://github.com/chpark1111/SMART/blob/main/docs/RESEARCH_PLAN.md): RL/deep learning priors, MCTS upgrade,
  memory/table-based search, and promotion rules.

## Local Example

If this checkout already has local ShapeNet meshes under `data/`, create a
3-per-category example set and run it:

```bash
bash examples/prepare_sample_shapes.sh
bash examples/run_example_3x3.sh
```

The example meshes are copied to `examples/sample_shapes/`, which is ignored by
git and excluded from packages.

## Data

SMART expects ShapeNet-style mesh folders:

```text
data/shapenet_airplane/<model_id>/model.obj
data/shapenet_chair/<model_id>/model.obj
data/shapenet_table/<model_id>/model.obj
```

Paper category synsets:

```text
airplane  02691156
chair     03001627
table     04379243
```

Prepare zipped category archives:

```bash
python scripts/prepare_shapenet_samples.py \
  --archive-dir /path/to/shapenet_zips \
  --output-root data/expanded \
  --categories airplane chair table \
  --limit 100000 \
  --normalize preserve
```

SMART writes normalized meshes under `runs/<profile>/normalized/`; it does not
modify the downloaded meshes in `data/`.

## Build External Tools

Full reproduction from raw meshes requires Mesh2Tet tools, CoACD, and the fixed
Manifold runtime. In a source checkout:

```bash
smart --config configs/smoke_5.yaml build-tools
```

After installing from a wheel, run the same setup in a writable project/cache
directory:

```bash
export SMART_TOOLS_ROOT="$PWD/.smart-tools"
smart --config smoke_5.yaml build-tools
```

`pip install` installs the SMART Python package and the bundled native SMART
C++ extension/executable from wheels. It intentionally does not clone and
compile Mesh2Tet/fTetWild/ManifoldPlus during installation. Those external
builds are large, platform-specific, and can require local compiler/system
packages, so SMART exposes them as an explicit `smart build-tools` step
instead. That one command prepares ManifoldPlus, fTetWild, the CoACD Python CLI
runtime, the fixed Manifold runtime, and the local `smart._cpp`/
`smart-cpp-native` build for a source checkout.
It is idempotent: if CoACD already probes successfully, SMART skips the slow
editable install; if source editable installation fails, SMART tries the PyPI
CoACD runtime and only fails the command when no working `coacd` CLI is found.

Use `smart --config configs/smoke_5.yaml build-cpp` only when you need to
rebuild the SMART C++ extension/executable without rebuilding external tools.

Prebuilt binaries can also be supplied:

```bash
export SMART_MANIFOLDPLUS_BIN=/path/to/ManifoldPlus/build/manifold
export SMART_FTETWILD_BIN=/path/to/fTetWild/build/FloatTetwild_bin
export SMART_COACD_BIN=/path/to/coacd
export SMART_MANIFOLD_PYTHON=/path/to/smart/vendor/manifold/build/bindings/python
```

## Tetrahedralization Failures

Mesh2Tet can fail on noisy ShapeNet meshes because the input OBJ may be
non-watertight, self-intersecting, degenerate, or split into awkward components.
SMART handles this per mesh, not as a fatal dataset error:

- logs each ManifoldPlus/fTetWild attempt under `runs/<profile>/logs/tetra/`;
- retries with finer settings, coarser settings, `--coarsen`, and robust winding
  number settings;
- validates that `tetra.msh` and `tetra.msh__sf.obj` exist and are usable;
- records failed attempts in the tetra manifest, then skips downstream stages
  for that mesh while continuing the rest of the dataset.

Before tetrahedralization, SMART runs conservative mesh cleanup. The tetra stage
also classifies failures and queues targeted repair retries automatically:

| Detected failure | Likely cause | Automatic SMART response |
| --- | --- | --- |
| `surface is not watertight` | holes or open mesh boundaries | retry with a temporary `fill_holes=true` repaired input |
| fTetWild/ManifoldPlus timeout or crash | self-intersection, very thin parts, degenerate faces, non-manifold edges | retry with conservative repaired input and robust/coarser parameter attempts |
| `tetra element count below minimum` | tetra parameters too fine/coarse or damaged repair output | keep fine/coarse retry schedule and record the failed parameters |
| ManifoldPlus repaired surface is too large | repair exploded the face count and fTetWild is likely to hit the timeout | opt-in `tetra.max_manifold_faces_for_ftetwild` guard skips that repaired surface before fTetWild |
| disconnected components | true multi-part shape or small detached fragments | only use `keep_largest_component=true` if explicitly enabled, because it can delete real parts |

Repaired inputs are written under `runs/<profile>/logs/tetra/...`; SMART never
mutates the original `data/` OBJ. More destructive rescue, such as
`keep_largest_component=true`, is available in config but is off by default
because it can remove real disconnected shape parts. A failed mesh is therefore
usually recoverable by either enabling a stronger repair variant or
loosening/coarsening the tetra parameters, but SMART will not silently corrupt
the shape just to force success.

See [`docs/TETRA_FAILURE_PLAYBOOK.md`](https://github.com/chpark1111/SMART/blob/main/docs/TETRA_FAILURE_PLAYBOOK.md) for
debug commands and stronger repair options.

## Run SMART

Smoke run through the Python pipeline:

```bash
smart --config configs/smoke_5.yaml run
smart --config configs/smoke_5.yaml summary
```

Run one mesh through the native C++ executable:

```bash
smart-cpp-native run-pipeline \
  --input data/shapenet_airplane/<model_id>/model.obj \
  --work_dir runs/native_one/<model_id> \
  --manifoldplus_bin external/mesh2tet/ManifoldPlus/build/manifold \
  --ftetwild_bin external/mesh2tet/fTetWild/build/FloatTetwild_bin \
  --coacd_bin external/CoACD/python/package/bin/coacd \
  --epsilon 0.002 \
  --edge_length 0.1 \
  --refine_max_step 2000 \
  --mcts_iter 3000
```

Run a native batch:

```bash
smart-cpp-native run-batch \
  --data_root data \
  --categories shapenet_airplane,shapenet_chair,shapenet_table \
  --limit_per_category 1 \
  --output_root runs/native_batch \
  --manifoldplus_bin external/mesh2tet/ManifoldPlus/build/manifold \
  --ftetwild_bin external/mesh2tet/fTetWild/build/FloatTetwild_bin \
  --coacd_bin external/CoACD/python/package/bin/coacd \
  --jobs auto \
  --reuse_existing \
  --resume_success

smart-cpp-native batch-summary \
  --manifest runs/native_batch/native_pipeline.jsonl
```

## Evaluate And Render

Evaluate bbox outputs with the paper metrics:

```bash
smart --config configs/smoke_5.yaml evaluate
```

Render final boxes:

```bash
smart --config configs/smoke_5.yaml render \
  --set render.transparent=true \
  --set render.joint_mesh=false
```

The default renderer is the packaged software preview renderer so macOS does
not launch Blender during normal runs. The adapted paper Blender renderer is
still packaged under `smart/legacy/renderer` and can be enabled explicitly:

```bash
smart --config configs/smoke_5.yaml --set render.backend=blender render
```

## Python API

```python
import smart

cfg = smart.load_config("configs/smoke_5.yaml")
records = smart.run(cfg)
print(records[-1])
```

Package/API details are in [`docs/PYTHON_PACKAGE.md`](https://github.com/chpark1111/SMART/blob/main/docs/PYTHON_PACKAGE.md).

Research acceleration hooks are also exposed from `smart.cpp`. The packaged
DeepSets refine router ranks candidate edits cheaply, then exact-scores the
selected subset with native SMART/Manifold before applying an action.  The
accepted reward therefore remains exact; the speed gain comes from fewer exact
geometry calls.

Installed wheels expose the packaged learned-router status and checkpoint:

```bash
smart learned-release-readiness --json
smart learned-release-readiness --fail-if-not-ready
smart learned-router-summary --json
smart assets --kind policies --json
```

From Python:

```python
import smart

print(smart.learned_router_profile_summary()["validation_snapshot"])
print(smart.asset_path("policies", "default"))
```

The learned refine helper's `profile="auto"` now resolves to the v9
production-candidate router for multibox states while preserving exact native
refine for one-box states:

```python
import smart.cpp as sc

engine = sc.NativeSmartEngine(...)
result = sc.run_builtin_deepset_policy_refine(
    engine,
    max_steps=4,
    profile="auto",
)
```

For the current fallback-free MCTS/portfolio replacement research profile, use
the packaged v13 DAgger policy and one of the explicit v13 profiles:

```python
import smart.cpp as sc

engine = sc.NativeSmartEngine(...)

# Quality-safe: 2000/2000 strict replay-ready states matched the exact
# candidate portfolio, with fewer exact checks but roughly neutral wall time.
result = sc.run_builtin_deepset_policy_refine(
    engine,
    max_steps=4,
    policy="mcts_replacement_v13",
    profile="mcts_replacement_v13_quality_safe",
)

# Faster held-out profile: val/test zero-regret with stronger runtime gain,
# but not yet all-split zero-regret.
fast_result = sc.run_builtin_deepset_policy_refine(
    engine,
    max_steps=4,
    policy="mcts_replacement_v13",
    profile="mcts_replacement_v13_heldout_fast",
)
```

Local validation for this profile:

```text
full token split 1015 states: 0 losses, 30.5% fewer exact calls, 1.204x vs exact oracle
1000 replay states:        0 losses, 30.7% fewer exact calls, 1.203x vs exact oracle
held-out test 264 states:  0 losses, 38.7% fewer exact calls, 1.361x vs exact oracle
target50 v11 runtime:      300 states, 0 losses, 30.5% fewer exact calls, 1.115x vs exact oracle
```

Large GPU Transformer teacher status:

```text
d512/l8/e16 MPS teacher: 2298 audited candidate-set steps
guide regret -> transformer regret: 3.1165 -> 0.0121
positive top-1: 2273/2298, exact-best top-1: 2004/2298
live C++ path: native proxy/top-K remains faster than per-state PyTorch/MPS inference
C++ DeepSets runtime candidate: 300 states, 0 losses, 30.5% fewer exact calls, 1.115x wall-time speedup
```

The Transformer path is therefore used for teacher scoring, hard-state mining,
and distillation research. The release runtime keeps C++ native routing plus
exact SMART/Manifold validation.

Current learned replacement status:

```text
mcts_replacement_v13_quality_safe:
  strict replay-ready split: 2000/2000 zero-regret against the exact candidate portfolio
  exact-call reduction:      13.6%
  wall-time effect:          roughly neutral on the local benchmark
  runtime:                   C++ DeepSets scorer + exact SMART/Manifold validator

mcts_replacement_v13_heldout_fast:
  val/test:                  zero-regret
  exact-call reduction:      26.5%
  wall-time effect:          about 1.07x over all splits
  caveat:                    4 train losses, so not the quality-safe profile
```

This is a learned candidate-ranking replacement for MCTS/exhaustive candidate
portfolio search, not a replacement for exact geometry scoring.  The stronger
research goal, `geometry_model_only` without exact top-K selection, remains a
research gate and is not the public default.

The portfolio and MCTS-prior helpers remain available for research sweeps:
`smart.cpp.run_builtin_deepset_portfolio_refine(...)` and
`smart.cpp.run_builtin_deepset_prior_mcts(...)`.  The packaged
`configs/learned_auto_safe.yaml` profile is a quick local validation preset:

```bash
smart --config configs/learned_auto_safe.yaml run
```

The same router can be enabled in the normal pipeline:

```bash
smart --config configs/smoke_5.yaml refine \
  --set refine.learned_router.enabled=true \
  --set refine.learned_router.profile=auto
```

See [`docs/PYTHON_PACKAGE.md`](https://github.com/chpark1111/SMART/blob/main/docs/PYTHON_PACKAGE.md)
for profile details.

The variable-length macro-skill controller is a release-candidate opt-in
post-refinement path. It proposes reusable multi-step fitting skills and accepts
only exact SMART/Manifold non-worse updates:

Config profile:

```bash
smart --config configs/learned_macro_safe.yaml run
```

Stage-source validated top-3 program-gate profile:

```bash
smart --config configs/learned_macro_program_gate_top3.yaml run
```

This tighter opt-in profile keeps the exact SMART/Manifold validator, disables
macro-memory reranking, and evaluates only the top three mined 3D macro-skill
programs.  The current release gate passes both post-refine and post-MCTS
stage-source replay (`456/456` accepted for each) while reducing exact
macro-skill attempts by `81.25%` versus the 16-skill portfolio.

Learned MCTS-replacement default-agent profile:

```bash
smart run
smart agent-run
smart run --agent
```

Equivalent explicit config:

```bash
smart --config configs/learned_default.yaml run
smart --config learned_default run
```

With no explicit `--config`, `smart run` now uses this learned default-agent
profile. It skips MCTS, feeds refine output into the packaged macrohash skill
bank, ranks a 16-skill exact portfolio with the learned JSON MLP selector,
tries the learned top-3 first, and falls back to the full exact portfolio when
the learned attempts are not confidently separated. The guarded candidate has
passed the current 510-state MCTS-replacement gate with zero losses versus the
exact fallback portfolio and `26.3%` fewer exact skill attempts. The paper
reproduction profiles remain available by passing an explicit config such as
`--config configs/paper_like.yaml`.

Use `smart run --category chair --mesh <mesh_id>` or
`smart agent-run --category chair --mesh <mesh_id>` to run the learned
default-agent path on a subset. If `--config` is provided, `run` respects that
config; `agent-run` additionally applies the learned agent overlay.
Use `smart --config <your_config.yaml> run --agent` to keep your data/config but
replace the MCTS stage with the guarded learned agent overlay.

Python API:

```python
import smart

records = smart.run_agent(
    category="chair",
    meshes=["11b7c86fc42306ec7e7e25239e7b8f85"],
)

records = smart.run()
records = smart.run("configs/smoke_5.yaml", agent=True)
```

The underlying explicit guarded profile is also available:

```bash
smart --config configs/learned_macro_mcts_replacement_guarded.yaml run
```

Pipeline stage usage:

```bash
smart --config configs/smoke_5.yaml \
  --set macro_skill.input_stage=mcts \
  --set macro_skill.quality_preset=balanced \
  macro_skill
```

The stage first looks for regular `mcts`/`refine` stage outputs and then falls
back to C++ native one-shot outputs under
`runs/<profile>/native_pipeline/<category>/<mesh>/{mcts,refine}_bboxs_steps0`.
This means `smart native-pipeline` results can be polished by `macro_skill`
without manually copying `bbox_params.json`.

To render the macro-skill output in the same pipeline:

```bash
smart --config configs/smoke_5.yaml \
  --set stages.macro_skill=true \
  --set render.input_stage=macro_skill \
  run
```

If no macro skill improves the exact SMART score, the stage restores and exports
the input bbox state so downstream rendering and evaluation still have a valid
output.

Prepared-state CLI usage:

```bash
smart macro-skill \
  --msh runs/example/tetra/airplane/0001/tetra.msh \
  --bbox-metadata runs/example/mcts/airplane/0001/bbox_params.json \
  --category airplane \
  --quality-preset balanced \
  --output runs/example/macro_skill/airplane/0001/result.json \
  --output-bbox-dir runs/example/macro_skill/airplane/0001/bboxs
```

The default macro-skill executor is the compact C++ path. Use
`--no-native-executor` only for ablations against the Python skill loop.
Check the packaged release gate with:

```bash
smart learned-release-readiness --json
smart learned-release-readiness --fail-if-not-ready
smart macro-skill-summary --json
```

Use `--quality-preset efficient` to spend the higher exact budget only on the
currently validated chair-like scheduler bucket. Use the learned Pareto family
`--quality-preset learned_fast`, `learned_efficient`, or `learned_quality` to
let a packaged state-conditioned ridge gate decide where to spend higher exact
budget from native geometry features. Use `--quality-preset quality` to spend
all top-k exact skill attempts for stronger quality polishing. See
[`docs/LEARNED_ROUTER.md`](https://github.com/chpark1111/SMART/blob/main/docs/LEARNED_ROUTER.md)
for the current benchmark evidence and safety contract.

## Repository Layout

```text
smart/        Python package, CLI/API, configs, pipeline wrappers
cpp/          Native C++ SMART core and smart-cpp-native executable
configs/      Source-checkout YAML profiles
examples/     Public shell examples; local sample meshes are ignored
scripts/      Supported data prep, release, and reproduction utilities
tests/        Package/native/release tests
docs/         User docs, paper assets, and release notes
experiments/  Ignored local research configs, scripts, assets, and tests
data/         Local ShapeNet data only; not packaged
runs/         Local outputs only; not packaged
external/     Downloaded Mesh2Tet/CoACD tools; not packaged
past_codes/   Original research archive; reference only
```

See [`docs/REPOSITORY_STRUCTURE.md`](https://github.com/chpark1111/SMART/blob/main/docs/REPOSITORY_STRUCTURE.md) for more
detail.

## Configs

Recommended public configs:

- `configs/smoke_5.yaml`: fast local smoke test.
- `configs/example_3x3.yaml`: 3 meshes per paper category from local example data.
- `configs/demo.yaml`: small demo profile.
- `configs/paper_like.yaml`: paper-style parameters.
- `configs/expanded_full.yaml`: larger local ShapeNet layout.

Experimental RL, pruning, and acceleration profiles are local-only under
`experiments/configs/`; they are ignored by git and excluded from release
packages.

Research directions for learned policy/value agents, MCTS priors,
local-minimum escape policies, and memory/table-based search are tracked in
[`docs/RESEARCH_PLAN.md`](https://github.com/chpark1111/SMART/blob/main/docs/RESEARCH_PLAN.md).

## Compatibility Notes

`pymesh.py` is a compatibility shim, not the external PyMesh package. It keeps
legacy SMART code that imports `pymesh` working by forwarding to
`smart.pymesh_compat`. New code should import `smart.pymesh_compat` directly.

## Release

Build and validate release artifacts:

```bash
smart-release-preflight \
  --dist-dir /private/tmp/smart_release_check \
  --venv-dir /private/tmp/smart_release_venv \
  --recreate-venv \
  --run-asan-smoke
```

The preflight also checks the learned-router, macro-skill, and learned
default-agent release contract with
`smart learned-release-readiness --fail-if-not-ready --require-default-ready`,
both from the source checkout and from the installed wheel.

Release notes and publishing steps are in [`docs/RELEASE.md`](https://github.com/chpark1111/SMART/blob/main/docs/RELEASE.md).

## Citation

```bibtex
@inproceedings{park2024smart,
  title = {Split, Merge, and Refine: Fitting Tight Bounding Boxes via Over-Segmentation and Iterative Search},
  author = {Park, Chanhyeok and Sung, Minhyuk},
  booktitle = {International Conference on 3D Vision (3DV)},
  year = {2024}
}
```

## License

This project is released for non-commercial research under
CC BY-NC-SA 4.0. See [`LICENSE`](LICENSE).
