Metadata-Version: 2.4
Name: arcope
Version: 0.1.9
Summary: Integration package bridging ARC crop parameter retrieval with SCOPE radiative transfer simulations
Author-email: Feng Yin <ucfafyi@ucl.ac.uk>
License-Expression: MIT
Project-URL: Homepage, https://github.com/MarcYin/ARCOPE
Project-URL: Documentation, https://marcyin.github.io/ARCOPE/
Project-URL: Repository, https://github.com/MarcYin/ARCOPE
Project-URL: Issues, https://github.com/MarcYin/ARCOPE/issues
Project-URL: Changelog, https://github.com/MarcYin/ARCOPE/releases
Keywords: remote sensing,crop monitoring,radiative transfer,sentinel-2,SIF,thermal,SCOPE,ARC,fluorescence,biophysical retrieval
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Atmospheric Science
Classifier: Topic :: Scientific/Engineering :: GIS
Classifier: Topic :: Scientific/Engineering :: Physics
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Typing :: Typed
Requires-Python: <3.15,>=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.24
Requires-Dist: xarray>=2023.1
Requires-Dist: scipy>=1.11
Requires-Dist: pandas>=2.0
Provides-Extra: scope
Requires-Dist: scope-rtm>=0.4.4; extra == "scope"
Requires-Dist: torch>=2.2; extra == "scope"
Provides-Extra: weather
Requires-Dist: cdsapi>=0.6; extra == "weather"
Provides-Extra: optim
Requires-Dist: scope-rtm>=0.4.4; extra == "optim"
Requires-Dist: torch>=2.2; extra == "optim"
Provides-Extra: all
Requires-Dist: arcope[optim,scope,weather]; extra == "all"
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-cov>=4.1; extra == "dev"
Requires-Dist: build>=1.2; extra == "dev"
Requires-Dist: twine>=5.0; extra == "dev"
Provides-Extra: docs
Requires-Dist: mkdocs>=1.6; extra == "docs"
Requires-Dist: mkdocs-material>=9.7; extra == "docs"
Requires-Dist: matplotlib>=3.7; extra == "docs"
Dynamic: license-file

# ARC-SCOPE

**Bridge ARC crop-parameter retrieval with SCOPE radiative-transfer simulations.**

[![PyPI version](https://img.shields.io/pypi/v/arcope.svg)](https://pypi.org/project/arcope/)
[![Python versions](https://img.shields.io/pypi/pyversions/arcope.svg)](https://pypi.org/project/arcope/)
[![Tests](https://github.com/MarcYin/ARCOPE/actions/workflows/tests.yml/badge.svg)](https://github.com/MarcYin/ARCOPE/actions/workflows/tests.yml)
[![Docs](https://github.com/MarcYin/ARCOPE/actions/workflows/docs-pages.yml/badge.svg)](https://marcyin.github.io/ARCOPE/)
[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)

ARC-SCOPE is an integration package that connects the **ARC** (Automated Retrieval of Crop biophysical parameters) system with the **SCOPE** (Soil-Canopy Observation of Photosynthesis and Energy fluxes) radiative-transfer model.  It converts Sentinel-2-derived biophysical parameters into SCOPE-ready inputs, fetches meteorological forcing data, orchestrates the full simulation pipeline, and supports parameter optimisation against observed SIF, thermal, or flux data.

- **PyPI**: `pip install arcope` · [pypi.org/project/arcope](https://pypi.org/project/arcope/) (imports as `arc_scope`)
- **Documentation**: [marcyin.github.io/ARCOPE](https://marcyin.github.io/ARCOPE/)
- **Python**: 3.9, 3.10, 3.11, 3.12, 3.13, 3.14 — CI-verified across the full range
- **Source**: [github.com/MarcYin/ARCOPE](https://github.com/MarcYin/ARCOPE)

The repo has two example tracks:

- a **real ARC retrieval -> SCOPE reflectance/SIF/thermal example** for the heavy runtime
- a **core-dependency showcase** for fast onboarding without ARC or SCOPE

## Architecture

```
                         ARC-SCOPE Data Flow
 ================================================================

  GeoJSON + Dates ──> ARC Retrieval
                         |
                         v
                    post_bio_tensor ──> Bridge ──> post_bio_da
                    scale_data              |      post_bio_scale_da
                                            |
                    ERA5 / Local ───> Weather Provider ──> weather_ds
                                            |
                    Field centroid ──> Observation Geometry ──> obs_ds
                                            |
                                            v
                                   prepare_scope_dataset()
                                            |
                                            v
                                     SCOPE Simulation
                                     (PyTorch runner)
                                            |
                                            v
                                  Reflectance / SIF / LST
                                  Energy-balance fluxes
                                            |
                                            v
                                   [Optional] Optimisation
                                   Tune fqe, rss, rbs, ...
                                   against observations
```

## Installation

Requires Python **3.9 – 3.14**.

| What you want | Install command |
|---|---|
| Core bridge + pipeline | `pip install arcope` |
| + ARC satellite retrieval (GDAL, JAX) | `pip install arcope && pip install git+https://github.com/MarcYin/ARC` |
| + SCOPE radiative transfer (PyTorch) | `pip install "arcope[scope]"` |
| + ERA5 weather downloads | `pip install "arcope[weather]"` |
| Everything | `pip install "arcope[all]"` |
| Development (tests + build tools) | `pip install -e ".[dev]"` |

The **core package** only needs `numpy`, `xarray`, `scipy`, and `pandas` — the heavy ARC, SCOPE, and ERA5 dependencies are opt-in via extras. This means you can run the bridge and the core-dependency showcase with a minimal install.

See [docs/installation.md](docs/installation.md) for detailed instructions including GDAL setup and ERA5 credential configuration.

## Quick Start

### 1. Full run example (ARC + SCOPE)

Run the real end-to-end example from the bundled Belgium test field in 2021:

```bash
pip install "arcope[all]"
scope fetch-upstream --dest ./upstream/SCOPE
python3 -m arc_scope.experiments.dual_workflow \
  --start-date 2021-05-15 \
  --end-date 2021-10-01 \
  --growth-season-length 60 \
  --weather-provider local \
  --weather-file ./src/arc_scope/data/showcase_weather.csv \
  --scope-root-path ./upstream/SCOPE \
  --workflow reflectance \
  --workflow fluorescence \
  --workflow thermal \
  --simulation-subset-size 8 \
  --scope-chunk-size 512 \
  --dtype float32 \
  --output-dir ./full-run-output
```

This run performs one real ARC retrieval, prepares SCOPE inputs from weather and observation geometry, runs the validated `reflectance`, `fluorescence`, and `thermal` workflows, and writes a markdown report plus an extensive figure suite and browser explorer.

See [docs/full-run-example.md](docs/full-run-example.md) for the artifact bundle and figure inventory.

### 2. Core showcase (core dependencies only)

Run the primary in-repo showcase to assemble SCOPE-shaped inputs, inspect forcing diagnostics, and fit a proxy fluorescence response without requiring ARC, SCOPE, or ERA5 credentials:

```bash
pip install arcope
python3 -m arc_scope.experiments.showcase --output-dir ./showcase-output
```

If you are working from a repo checkout, `examples/05_showcase_experiment.py` wraps the same packaged entry point.

See [docs/showcase-experiment.md](docs/showcase-experiment.md) for the full walkthrough and generated artifacts.

### 3. Bridge conversion (standalone, from NPZ)

Convert saved ARC outputs to SCOPE-ready xarray DataArrays without needing ARC or SCOPE installed:

```python
from arc_scope.bridge import arc_npz_to_scope_inputs

post_bio_da, post_bio_scale_da = arc_npz_to_scope_inputs(
    "arc_output.npz", year=2021
)
print(post_bio_da.dims)   # ('y', 'x', 'band', 'time')
print(post_bio_da.coords["band"].values)  # ['N', 'cab', 'cm', ...]
```

### 4. Full pipeline (requires ARC + SCOPE)

Run the complete workflow from a GeoJSON field boundary to SCOPE outputs:

```python
from arc_scope.pipeline import ArcScopePipeline, PipelineConfig

config = PipelineConfig(
    geojson_path="field.geojson",
    start_date="2021-05-15",
    end_date="2021-10-01",
    crop_type="wheat",
    start_of_season=170,
    year=2021,
    scope_workflow="reflectance",
)
pipeline = ArcScopePipeline(config)
result = pipeline.run()
```

### 5. Parameter optimisation

Optimise SCOPE parameters (e.g., fluorescence quantum efficiency) against
observations. The high-level pipeline can run the optimisation branch directly
when `optim_config` provides observations and target variables:

```python
from arc_scope.pipeline import ArcScopePipeline, PipelineConfig

config = PipelineConfig(
    geojson_path="field.geojson",
    start_date="2021-05-15",
    end_date="2021-10-01",
    crop_type="wheat",
    start_of_season=170,
    year=2021,
    scope_workflow="fluorescence",
    scope_root_path="./upstream/SCOPE",
    scope_chunk_size=512,
    optim_config={
        "enabled": True,
        "batch_size": 256,
        "observations_path": "observed_f740.nc",
        "target_variables": ["F740"],
        "parameters": [
            {"name": "fqe", "initial": 0.01, "lower": 0.001, "upper": 0.1, "transform": "log"}
        ],
        "optimizer": {"type": "scipy", "method": "L-BFGS-B", "use_autograd_jac": "required"},
    },
)

result = ArcScopePipeline(config).run()
print(result.optimization_result.parameters_optimized)
```

The [Optimization Guide](docs/optimization-guide.md) includes proxy examples
for SIF, thermal, and energy-balance fitting plus a real ARC -> upstream SCOPE
fluorescence run where `fqe` recovers a known `F740` twin-observation target.

## Module Overview

| Module | Purpose | Key classes/functions |
|--------|---------|----------------------|
| `arc_scope.bridge` | Convert ARC arrays to SCOPE format | `arc_arrays_to_scope_inputs`, `arc_npz_to_scope_inputs`, `validate_soil_params` |
| `arc_scope.weather` | Fetch meteorological forcing data | `WeatherProvider`, `ERA5Provider`, `LocalProvider` |
| `arc_scope.pipeline` | End-to-end orchestration | `PipelineConfig`, `ArcScopePipeline` |
| `arc_scope.optim` | Parameter optimisation | `ParameterSpec`, `ParameterSet`, `ScopeObjective`, `ScipyOptimizer`, `TorchOptimizer` |
| `arc_scope.experiments` | Reproducible showcase experiments | `run_full_experiment`, `write_full_run_artifacts`, `run_showcase_experiment`, `write_showcase_artifacts` |
| `arc_scope.utils` | Geometry, I/O, type aliases | `solar_position`, `load_geojson_bounds`, `BBox`, `PathLike` |
| `arc_scope.data` | Bundled test data | `TEST_FIELD_GEOJSON`, `SHOWCASE_WEATHER_CSV` |

## Configuration Reference

`PipelineConfig` is a dataclass that controls the entire pipeline:

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `geojson_path` | `PathLike` | *required* | Path to field boundary GeoJSON |
| `start_date` | `str` | *required* | Start date (`YYYY-MM-DD`) |
| `end_date` | `str` | *required* | End date (`YYYY-MM-DD`) |
| `crop_type` | `str` | *required* | Crop identifier (e.g., `"wheat"`) |
| `start_of_season` | `int` | *required* | Growth season start (day of year) |
| `year` | `int` | *required* | Calendar year |
| `num_samples` | `int` | `100000` | ARC archetype samples |
| `growth_season_length` | `int` | `45` | Season length in days |
| `weather_provider` | `str` | `"era5"` | `"era5"` or `"local"` |
| `scope_workflow` | `str` | `"reflectance"` | Simulation workflow |
| `device` | `str` | `"cpu"` | PyTorch device |
| `output_dir` | `PathLike` | `"./output"` | Output directory |

## SCOPE Workflows

| Workflow | `calc_fluor` | `calc_planck` | Description |
|----------|:---:|:---:|-------------|
| `reflectance` | 0 | 0 | Directional reflectance only (fastest) |
| `fluorescence` | 1 | 0 | Reflectance + SIF (F685, F740) |
| `thermal` | 0 | 1 | Reflectance + thermal emission (LST) |
| `energy-balance` | 1 | 1 | Full energy balance with SIF + thermal + fluxes |

## Examples

Working examples are in the [`examples/`](examples/) directory:

- **[01_bridge_conversion.py](examples/01_bridge_conversion.py)** -- Convert synthetic ARC arrays to SCOPE format (no external dependencies)
- **[02_reflectance_simulation.py](examples/02_reflectance_simulation.py)** -- Prepare and inspect a SCOPE-ready dataset
- **[03_full_pipeline.py](examples/03_full_pipeline.py)** -- Complete pipeline configuration and step-by-step execution
- **[04_optimization_demo.py](examples/04_optimization_demo.py)** -- Parameter optimisation workflow with transforms and injection
- **[05_showcase_experiment.py](examples/05_showcase_experiment.py)** -- Core-only showcase experiment with local weather, geometry, radiation partitioning, and proxy calibration
- **[06_dual_workflow_full_run.py](examples/06_dual_workflow_full_run.py)** -- Real ARC retrieval plus a documented SCOPE reflectance run with artifact generation
- **[07_optimization_examples.py](examples/07_optimization_examples.py)** -- Generate reproducible optimisation output tables and SVG fit plots for the optimisation guide

## Development

```bash
git clone https://github.com/MarcYin/ARCOPE.git
cd ARCOPE
pip install -e ".[dev]"
python -m pytest --tb=short -q
```

CI runs the full test suite against Python 3.9, 3.10, 3.11, 3.12, 3.13, and 3.14 on every push.

## Releases

- **Current**: see the [PyPI page](https://pypi.org/project/arcope/) and the [GitHub releases](https://github.com/MarcYin/ARCOPE/releases).
- **Changelog**: [CHANGELOG.md](CHANGELOG.md).
- **Maintainers**: release steps are documented in [RELEASING.md](RELEASING.md). Publishes via GitHub Actions with PyPI OIDC trusted publishing — no long-lived tokens.

## License

MIT -- see [LICENSE](LICENSE) for details.
