Metadata-Version: 2.4
Name: cars_edge_detection_plugin
Version: 0.1.0
Summary: Cars plugin to use the MoGe2 model for edge detection
Author-email: CNES <cars@cnes.fr>
Project-URL: Documentation, https://cars.readthedocs.io/en/latest/
Keywords: cars,3D,DEM,pandora,photogrammetry,moge2
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: End Users/Desktop
Classifier: Intended Audience :: Science/Research
Classifier: Environment :: Console
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3.10
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
License-File: AUTHORS.md
Requires-Dist: numpy
Requires-Dist: rasterio>=1.3.0
Requires-Dist: huggingface-hub
Requires-Dist: torch
Requires-Dist: torchvision
Requires-Dist: click
Requires-Dist: opencv-python
Requires-Dist: scipy
Requires-Dist: matplotlib
Requires-Dist: moderngl
Requires-Dist: trimesh
Requires-Dist: pillow
Requires-Dist: gradio
Requires-Dist: xarray
Requires-Dist: json_checker
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: pre-commit; extra == "dev"
Requires-Dist: isort>=5.8.0; extra == "dev"
Requires-Dist: black>=24.1a1; extra == "dev"
Requires-Dist: flake8>=3.9.1; extra == "dev"
Requires-Dist: flake8-pyproject; extra == "dev"
Requires-Dist: flake8-comprehensions>=3.4.0; extra == "dev"
Requires-Dist: flake8-bugbear<24.0.0,>=21.4.3; extra == "dev"
Requires-Dist: jupyter_contrib_nbextensions; extra == "dev"
Requires-Dist: pylint<3.0.0,>=2.8.12; extra == "dev"
Requires-Dist: setuptools_scm; extra == "dev"
Requires-Dist: virtualenv; extra == "dev"
Requires-Dist: configupdater; extra == "dev"
Requires-Dist: build; extra == "dev"
Provides-Extra: cars
Requires-Dist: cars<1.1,>=1.0a0; extra == "cars"
Dynamic: license-file

# Edge Detection plugin for CARS

This plugin enables the use of the MoGe2 model in CARS, for higher accuracy building reconstruction.

- [Context](#contexte)
- [Installation](#installation)
- [Using the new pipeline](#using-the-new-pipeline) 

## Context

This plugin is made to be used in conjunction with CARS, the stereo-reconstruction software. 
More information can be found over at [CARS's GitHub page](https://github.com/CNES/cars).

## Installation

First clone this repository, using :

```bash
$ git clone --recurse-submodules git@gitlab.cnes.fr:dali/cars-park/cars-plugins/cars-edge-detection-plugin.git
$ cd cars-edge-detection-plugin   
``` 

> **Note:** `--recurse-submodules` is required. This project vendors [MoGe](https://github.com/microsoft/MoGe) as a submodule, and MoGe itself vendors `utils3d` and `pipeline` as nested submodules. A plain `git clone` will leave those directories empty, causing build failures.

If you already cloned the repository without submodules, initialize them before installing:

```bash
$ git submodule update --init --recursive
```

You can then create a virtual environment and install the plugin, which will install CARS automatically :

```bash
$ python3 -m venv venv
$ source venv/bin/activate
$ make install   
``` 

Or install the plugin in your own environment, if it already has CARS :

```bash
$ source your/own/env/activate
$ pip install .
```

Once installed, don't forget to download a MoGe2 model, for example Ruicheng/moge-2-vitl-normal, using this command :

```bash
$ cars-download-moge2 --model vitl-normal
```

Or via any other means if you don't have a direct access to the internet. The plugin will attempt to download the vitl-normal model on first import, providing at least the default model.
If working from an environment such as the TREX cluster, an option is to directly use wget to fetch the model file, 
then move it to its proper place for the plugin to recognize it :

```bash
# fetch the model
$ wget https://huggingface.co/Ruicheng/moge-2-vitl-normal/resolve/main/model.pt

# move the model to the right place
# it should be under cars_edge_detection_plugin/applications/depth_map_generation/models with the proper name for each model :
#  - moge-2-vitl-normal.pt
#  - moge-2-vitb-normal.pt
#  - moge-2-vits-normal.pt
$ mkdir [your/plugin/installation/path/]cars_edge_detection_plugin/applications/depth_map_generation/models
$ mv ./model.pt [your/plugin/installation/path/]cars_edge_detection_plugin/applications/depth_map_generation/models/moge-2-vitl-normal.pt
```

## Using the new pipeline

Though this pipeline is intended to be used within CARS's meta pipeline, it can still be used as a stand-alone pipeline by providing the right configuration.

Once your configuration file is ready, you can launch the pipeline using CARS : 

```bash
$ cars configfile.yaml
```

### Configuration

The edge detection pipeline can be enabled by setting the pipeline parameter in the global advanced section of the CARS configuration.

A minimal example configuration is shown below:

```yaml
input:
  sensors:
    one: # sensor image path
    two: # sensor image path
pipeline: edge_detection
output:
  directory: outresults
```

The pipeline operates on image pairs. By default, edge detection is only computed where required by downstream applications, meaning on the left images only.

Additional options specific to the edge detection pipeline can be configured under the edge_detection section. 
For example, edge detection can also be applied to right images, and the MoGe2-based depth map generation application can be configured as follows:

```yaml
input: ...
advanced: ...
output: ...
edge_detection:
  advanced:
    save_intermediate_data: false
    right_image_edge_detection: true
  applications:
    depth_map_generation:
      method: moge2
      model: Ruicheng/moge-2-vitl-normal
      save_intermediate_data: true
      edge_threshold: 0.7
```

The ``model`` parameter can reference either a local MoGe2 checkpoint or a Hugging Face model identifier.

If ``save_intermediate_data`` is set to false, only the edge map will be created in the output folder. 
Else, all by-products (depth map, normal map, tile_id) will be saved in the ``dump_dir`` folder.
