Metadata-Version: 2.4
Name: cloudyview
Version: 0.2.0
Summary: 3D cloud field visualization toolkit with radiative transfer
Author-email: "Thomas D. DeWitt" <thomas.dewitt@utah.edu>
License: MIT
Project-URL: Homepage, https://github.com/thomasdewitt/cloudyview
Project-URL: Documentation, https://github.com/thomasdewitt/cloudyview
Project-URL: Repository, https://github.com/thomasdewitt/cloudyview.git
Project-URL: Issues, https://github.com/thomasdewitt/cloudyview/issues
Keywords: clouds,visualization,radiative-transfer,atmospheric-science
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Atmospheric Science
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.20
Requires-Dist: matplotlib>=3.3
Requires-Dist: xarray>=0.18
Requires-Dist: netCDF4>=1.5
Requires-Dist: wgpu>=0.30
Provides-Extra: behold
Requires-Dist: mitsuba>=3.0.0; extra == "behold"
Requires-Dist: drjit>=0.3.0; extra == "behold"
Provides-Extra: dev
Requires-Dist: pytest>=6.0; extra == "dev"
Requires-Dist: imageio>=2.28; extra == "dev"
Requires-Dist: pillow>=9.0; extra == "dev"
Requires-Dist: scaleinvariance>=0.14.0; extra == "dev"
Dynamic: license-file

# CloudyView

A toolkit for 3D cloud field visualization with optical depth calculations and Monte Carlo radiative transfer.

## Interactive rendering in browser (recommended)

Visit [thomasddewitt.com/soar](https://thomasddewitt.com/soar), open your NetCDF file containing cloud condensate, adjust quality and time of day, and render images and videos. Soar uses the `witness` renderer, described below.

![Witness render of a STEAM cloud field, looking into the sun](https://raw.githubusercontent.com/thomasdewitt/cloudyview/main/docs/witness_small_c002_s0100_into_sun.png)

## Programmatic rendering: Overview

For rendering with more fine-grained control, `cloudyview` may be installed to give three terminal commands for renders in three increasing levels of cost:

- **Glimpse** (`glimpse`): Quick optical depth calculation + 2D visualization from above, like a satellite image
- **Witness** (`witness`): Fast volumetric ray marching with visually tuned heuristics, on GPU
- **Behold** (`behold`): Physically accurate Monte Carlo path tracing with Mitsuba 3. Very, very expensive for clouds.

The source code for Soar is in `web/soar`.

## Quick Start: Terminal Commands

For `witness` and `behold`, it is recommended to first choose your view in [Soar](https://thomasddewitt.com/soar), and use the "Render this view in terminal" command, which gives a command to copy-paste.

All scripts require a NetCDF file with cloud ice/water mixing ratio variables.

### Glimpse: Quick 2D View

```bash
glimpse example_cloud.nc
```

Generates a matplotlib visualization of column optical depth.

Options:

- `--output`, `-o`: Output directory (default: current directory)
- `--label-dirs`: Label N/S/W/E sections of the domain
- `--label`: Overlay camera marker and field-of-view on the top view
- `--camera-position X Y Z`, `--camera-azimuth`, `--camera-elevation`, `--fov`:
  Camera used by `--label` (same conventions as witness/behold)

### Witness: Fast Volumetric Rendering

```bash
witness example_cloud.nc
```

Generates a volumetric ray-marched render with multi-scattering, procedural sky, and ocean surface.

`--size W H` sets the image size (default 600×400). `--quality` renders as
the app's capture at that tier (`minimal`, `low`, `medium`, `high`, `max`).

Options:

| Argument                  | Default      | Description                                                                   |
| ------------------------- | ------------ | ----------------------------------------------------------------------------- |
| `--output`, `-o`          | `.`          | Output directory                                                              |
| `--camera-position X Y Z` | `0 0 -0.999` | Camera position in relative coords (±1.0 = domain edge)                       |
| `--camera-azimuth`        | `0`          | Camera azimuth in degrees (0=N, 90=E, 180=S, 270=W)                           |
| `--camera-elevation`      | `35`         | Camera elevation in degrees (angle above horizon)                             |
| `--fov`                   | `100`        | Horizontal field of view in degrees                                           |
| `--sun-azimuth`           | `20`         | Sun azimuth in degrees                                                        |
| `--sun-elevation`         | `55`         | Sun elevation in degrees                                                      |
| `--exposure`              | `4.0`        | Tone-map exposure (soar's "render in terminal" writes its metered value here) |
| `--gamma`                 | `1.66`       | Tone-map gamma                                                                |
| `--white-point`           | `15.0`       | Extended-Reinhard white point: the exposed radiance mapping to 1.0            |
| `--contrast`              | `1.0`        | Display contrast about mid-grey, applied after the gamma encode               |
| `--haze`                  | `1.0`        | Aerosol amount, as soar's haze slider: -0.22 (clearest) to 2.5                |
| `--haze-height-dependent` | off          | Thin the haze with height on a 2500 m scale height                            |
| `--lod`                   | `0.5`        | Angular level of detail; smaller is finer and slower                          |
| `--periodic`              | off          | Wrap the domain in x and y, as soar does for LES fields                       |
| `--nest-group GROUP`      | —            | NetCDF group in the same file holding a finer field to render as a nest       |

The image controls are the same knobs the browser app exposes, so a soar
view reproduces exactly in the terminal. All three commands also accept the
dataset-selection overrides described under
[Input Data Requirements](#input-data-requirements) (`--group`,
`--liquid-water-var`, `--x-dim`, ...); see `witness --help` for the full
list.

### Soar: Render a Recorded Flight

```bash
soar cloudyview_track_20260901.json
```

Re-renders a flight track recorded in the browser app (R to record, then
"Save the track") into an H.264 mp4 next to the track. Every frame is
rendered with the quality tier's converged accumulation — the still the app
reaches when the camera stops — so the video has none of the in-flight
motion speckle, and the frame timing is the track's, whatever each frame
cost. Everything about the picture (field, sun, exposure, tone map, haze,
LOD, quality tier, size, the night city) comes from the track's header, so
the command needs only the track; the flags override one thing at a time.

| Argument             | Default              | Description                                                       |
| -------------------- | -------------------- | ----------------------------------------------------------------- |
| `--output`, `-o`     | track name + `.mp4`  | Video path                                                        |
| `--field FILE.nc`    | the header's path    | The cloud field, when it is not where the header says             |
| `--size W H`         | the capture size     | Video size (odd sizes round down to even)                         |
| `--fps`              | `60`                 | Frame rate                                                        |
| `--quality`          | the header's tier    | Render at another soar tier                                       |
| `--crf`              | `18`                 | x264 quality, lower is better                                     |
| `--nest-group GROUP` | —                    | As for `witness`                                                  |

Needs a GPU and the `ffmpeg` binary on PATH; there is no fallback encoder.
The browser renders the same track in the tab ("Render it to video") — this
is for a GPU under your own control, a render that survives the tab, and a
terminal to watch it from.

### Behold: Photorealistic Rendering

```bash
behold example_cloud.nc --cpu
```

Generates photorealistic path-traced render using Mitsuba 3.

Arguments:

- `--cpu` or `--gpu`: Backend selection (required)
- `quality`: `min`, `low`, `medium` (default), `high`, `max`, or `custom`

Quality tiers:

- `min`: 150×100, spp=1, max_depth=2, rr_depth=1
- `low`: 300×200, spp=32, max_depth=4, rr_depth=2
- `medium`: 600×400, spp=512, max_depth=8, rr_depth=4 (default)
- `high`: 960×640, spp=1024, max_depth=8, rr_depth=4
- `max`: 1200×800, spp=2048, max_depth=96, rr_depth=64 (untruncated path depths)
- `custom`: User-specified spp, resolution, and/or max_depth/rr_depth
  (unspecified depths default to `max`'s)

Clouds bury most of a path tracer's budget in dense multiple scattering, so
`min` through `high` truncate paths early (`max_depth=8` or less) and accept
the darkening bias; `max` keeps the deep budgets and is not tractable for
most cloud fields.

Options:

| Argument                  | Default      | Description                                                           |
| ------------------------- | ------------ | --------------------------------------------------------------------- |
| `--output`, `-o`          | `.`          | Output directory for renders                                          |
| `--ice FILE`              | —            | Separate NetCDF file with the ice variable (SAM LPT split-file style) |
| `--spp N`                 | varies       | Samples per pixel (for custom quality)                                |
| `--size W H`              | varies       | Image dimensions in pixels (for custom quality)                       |
| `--max-depth N`           | varies       | Maximum ray bounce depth (for custom quality override)                |
| `--rr-depth N`            | varies       | Russian roulette depth (for custom quality override)                  |
| `--camera-position X Y Z` | `0 0 -0.999` | Camera position in relative coords (±1.0 = domain edge)               |
| `--camera-azimuth`        | `0`          | Camera view azimuth in degrees (0=N, 90=E, 180=S, 270=W)              |
| `--camera-elevation`      | `35`         | Camera view elevation in degrees (angle above horizon)                |
| `--fov`                   | `100`        | Camera field of view in degrees                                       |
| `--sun-azimuth`           | `20`         | Sun azimuth in degrees (0=N, 90=E, 180=S, 270=W)                      |
| `--sun-elevation`         | `55`         | Sun elevation in degrees (angle above horizon)                        |

## Installation for programmatic rendering

```bash
pip install cloudyview            # glimpse and witness
pip install 'cloudyview[behold]'  # + Mitsuba 3 for path-traced renders
```

`witness` needs a GPU (WebGPU via wgpu-py; Metal on a Mac, Vulkan or D3D12
elsewhere). The soar shader and the ocean and city tiles ship inside the
package (`cloudyview/soar/`); the web app reaches the same files through
symlinks under `web/soar/`, so the browser and the Python host compile one
shader.

### Development install

```bash
git clone https://github.com/thomasdewitt/cloudyview
cd cloudyview
uv sync
```

or `pip install -e ".[dev]"`.

## Library usage

CloudyView is library-first: 3D cloud volume in, image array out. The CLIs
below are thin wrappers over these functions.

```python
import cloudyview as cv

# Load — one file with both variables, or split files (SAM LPT-style
# output writes one variable per file: ..._QC_*.nc, ..._QI_*.nc)
field = cv.load("cloud.nc")                          # autodetect qc + qi
field = cv.load("..._QC_0000000600.nc",
                ice="..._QI_0000000600.nc")          # split liquid/ice files
field = cv.load("cloud.nc", liquid_water_var="QC")   # explicit overrides

field.lwc, field.iwc      # (nx, ny, nz) float32 g/kg; iwc may be None
field.x, field.y, field.z # 1D coords, meters

cam = cv.Camera(position=(0, -0.8, -0.95),  # relative coords, ±1 = domain edge
                azimuth=0,                  # met bearing: 0=N, 90=E
                elevation=35,               # degrees above horizon
                fov=100)                    # horizontal field of view

albedo = cv.glimpse(field)                        # (ny, nx) two-stream visual albedo
img = cv.witness(field, camera=cam, size=(600, 400))   # (H, W, 3) in [0, 1]
img = cv.behold(field, camera=cam, quality="high")     # (H, W, 3), Mitsuba 3

cv.save_image(img, "render.png")
```

Notes:

- Render functions return arrays and never write files; use `cv.save_image`
  or matplotlib for output.
- `cv.witness(...)` drives soar's WGSL ray marcher through wgpu.
  `cv.behold(..., gpu=True)` selects Mitsuba's CUDA variant.
- Library code raises exceptions and is quiet by default
  (`verbose=True` restores the CLI-style diagnostics).
- The functions `cv.glimpse` / `cv.witness` / `cv.behold` shadow the
  same-named submodules on the package namespace; the modules remain
  importable directly (e.g. `from cloudyview.witness import NestedLevel`).
- A finer field nested inside a coarser one renders as
  `cv.witness(field, camera, nest=fine_field)`.
- `cloudyview.image.read_metadata("still.png")` returns what soar embedded
  in a still: the source file, camera, sun and the command that reproduces
  it.

## Input Data Requirements

NetCDF files must contain:

- **Liquid water variable** (required): One of `qc`, `QC`, `ql`, `QL`, `LWC`, `clw`, `cloud_liquid_water_mixing_ratio`, `liquid_water_content`, `q_liquid`, or `lwc`
- **Ice water variable** (optional): One of `qi`, `QI`, `qice`, `QICE`, `IWC`, `cli`, `cloud_ice_mixing_ratio`, `ice_water_content`, `q_ice`, or `iwc`
- **Spatial dimensions**: Must be 3D, with physical x/y/z coordinates
- **Temporal dimension**: One timestep is rendered; pick it with `--timestep N` on a multi-step file

When autodetection is not enough — non-standard names, variables inside
NetCDF groups, unusual dimension names — all three commands accept the same
override flags: `--group`, `--liquid-water-var`, `--liquid-water-group`,
`--ice-water-var`, `--ice-water-group`, `--coords-group`,
`--x-coord`/`--y-coord`/`--z-coord`, and `--x-dim`/`--y-dim`/`--z-dim`.
Physical x/y/z coordinates are always required (index axes are not enough:
optical depth and rendering depend on grid spacing). See `witness --help`
for the rules and example patterns.

## Coordinate System

CloudyView uses the **meteorological convention**:

- **East** = +x direction
- **North** = +y direction
- **Up** = +z direction

Azimuth angles use the meteorological convention (clockwise from North):

- 0° = North, 90° = East, 180° = South, 270° = West

Elevation angles are measured from the horizon:

- 0° = horizon, 90° = zenith, -90° = nadir

## Usage Examples

### Quick Visualization Pipeline

```bash
# 1. Quick 2D overview
glimpse my_cloud_data.nc

# 2. Fast volumetric render
witness my_cloud_data.nc

# 3. Photorealistic render
behold my_cloud_data.nc high --gpu --output ./renders
```

### Witness Examples

```bash
# Default render
witness cloud.nc

# Soar's max capture tier, custom camera
witness cloud.nc --quality max --camera-position 0 -0.9 -0.99 --camera-azimuth 0 --camera-elevation 35

# Custom size (default 600x400)
witness cloud.nc --size 1200 800
```

### Behold Examples

```bash
# Fast CPU preview
behold cloud.nc --cpu

# Balanced GPU render
behold cloud.nc --gpu

# Production quality render
behold cloud.nc max --gpu --output ./final_renders

# Quick GPU preview
behold cloud.nc low --gpu

# Custom quality: 1024x768 at 256 spp with max_depth=64
behold cloud.nc custom --gpu --size 1024 768 --spp 256 --max-depth 64 --rr-depth 32
```

## Where things are

- `cloudyview/` is the Python package: loading (`field.py`, `io.py`), the
  three renderers (`glimpse.py`, `witness.py`, `behold.py`) and `track.py`,
  which renders a recorded flight (`soar TRACK.json`).
- `cloudyview/soar/` is the renderer that `witness` and the browser share:
  the WGSL shader (`shader/`), its surface tiles, and the Python host that
  drives it through wgpu.
- `web/soar/` is the browser app.
- `docs/architecture.md` describes how the pieces fit.

## Features

### Visualizations

- **Glimpse**: 2D column optical depth visualization
- **Witness**: Fast volumetric ray marching with multi-scattering
- **Behold**: Photorealistic Monte Carlo path tracing with Mitsuba 3

### Data Handling

- Automatic variable name inference (tries common naming conventions)
- Comprehensive validation (checks dimensions, units and coordinates)
- Support for both liquid and ice water phases
- Works with different coordinate systems
- Variable vertical spacing supported

### Radiative Transfer (Behold)

- Full volumetric path tracing with Mitsuba 3
- Physically-based Preetham sunsky model
- Accurate Mie scattering from pre-computed phase functions
- Configurable camera and sun positions via CLI flags
- Multiple quality levels from instant preview to production
- Progressive rendering with checkpoints
- Ocean surface with realistic reflections

### Configuration

- Built-in defaults with CLI overrides
- Relative coordinate positioning (±1.0 = domain edge)
- Customizable rendering parameters via CLI flags

## Dependencies

### Core Dependencies

- `numpy>=1.20`: Array operations
- `matplotlib>=3.3`: 2D plotting
- `xarray>=0.18`: NetCDF file handling with labeled arrays
- `netCDF4>=1.5`: NetCDF4 file support
- `wgpu>=0.30`: WebGPU for witness (needs a GPU at run time)

### Optional Dependencies

- **For behold (photorealistic rendering)**: `mitsuba>=3.0.0`, `drjit>=0.3.0` — `pip install 'cloudyview[behold]'`

## Mie Phase Function Tables

Behold uses pre-computed Mie scattering phase function tables for accurate cloud scattering.

### Liquid Water Droplets

- **Mie0_normalized.txt**: Forward scattering peak (strongly forward-scattered component)
- **MiePF3_normalized.txt**: Rest of phase function distribution (sideways and backward scattering, wavelength-dependent RGB values)

These tables are from the PhD thesis:

**Antoine Bouthors (2008)** - "Real-time realistic rendering of clouds"
PhD Thesis, Université Grenoble I - Joseph Fourier
https://theses.hal.science/tel-00319974

The dual-table approach enables efficient importance sampling by separately handling the extremely strong forward peak and the much weaker sideways/backward scattering regions. The tables are blended with equal weighting (0.5) for optimal sampling.

### Ice Crystals

- **IceMie0_normalized.txt**: Forward scattering peak for ice crystals (RGB wavelength-dependent)
- **IceMiePF3_normalized.txt**: Rest of phase function distribution for ice crystals (RGB wavelength-dependent)

These tables are derived from:

**Baum, B.A., Yang, P., Heymsfield, A.J., Bansemer, A., Cole, B.H., Merrelli, A., Schmitt, C., Wang, C. (2014)** - "Ice cloud single-scattering property models with the full phase matrix at wavelengths from 0.2 to 100 µm"
*Journal of Quantitative Spectroscopy and Radiative Transfer*, 146, 123-139.
https://www.sciencedirect.com/science/article/pii/S0022407314000867#ab0010

Full ice scattering database downloaded from: http://download.ssec.wisc.edu/files/polarization_models/

**Note**: The complete ice scattering data and processing scripts are located in `data/ice_models/`, which contains a script to reformat the original data and extract the phase functions needed for rendering.

## Author

Thomas D. DeWitt (https://github.com/thomasdewitt/)

## License

MIT License - See LICENSE file for details
