Metadata-Version: 2.4
Name: nc2cog
Version: 0.1.3
Summary: Convert netCDF files to Cloud-Optimized GeoTIFF format with advanced compression
Author: Sam Chen
License: MIT
Project-URL: Homepage, https://github.com/cyberpsyche/nc2cog
Project-URL: Repository, https://github.com/cyberpsyche/nc2cog
Keywords: netcdf,cog,geotiff,gdal,geospatial
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: GIS
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: GDAL>=3.8.0
Requires-Dist: click>=8.1.0
Requires-Dist: PyYAML>=6.0
Requires-Dist: netCDF4>=1.6.0
Requires-Dist: numpy>=1.21.0
Dynamic: license-file

# nc2cog - netCDF to Cloud-Optimized GeoTIFF Converter

[中文](README_zh.md)

Convert netCDF files to Cloud-Optimized GeoTIFF format with advanced compression and performance settings.

## 🚀 Features

- **Format Conversion**: Convert netCDF files to Cloud-Optimized GeoTIFF (COG)
- **Batch Processing**: Process entire directories of netCDF files
- **Advanced Compression**: Support for deflate, lzw, and jpeg compression with configurable levels
- **Performance Optimization**: Configurable tile and block sizes for optimal performance
- **Pyramid Structure**: Customizable overview (pyramid) levels for multi-scale access
- **GDAL Optimized**: Eliminated GDAL warnings and optimized driver-specific parameters
- **Parallel Processing**: Multi-threaded conversion for faster processing
- **Resume Capability**: Resume interrupted conversions
- **Projection Transformation**: Reproject data from source to target coordinate systems during conversion
- **Flexible Output**: Output to a directory or directly to a specified `.tif` file
- **Rich Metadata**: 18 metadata fields automatically written to output COG (source, extent, resolution, min/max, unit, etc.)
- **Clean COG Layout**: Metadata written during conversion pipeline — no post-creation modifications that break COG layout optimization

## 📋 Requirements

- Python 3.7+
- GDAL library with Python bindings
- click library for CLI handling
- numpy for numerical operations

## 🛠️ Installation

```bash
# Clone the repository
git clone <repository-url>
cd nc2cog

# Install dependencies
pip install -r requirements.txt

# Install GDAL
# On macOS: brew install gdal
# On Ubuntu: sudo apt-get install gdal-bin libgdal-dev
# On Windows: Use OSGeo4W installer
```

## 🚀 Quick Start

### Single File Conversion

Convert a single netCDF file to COG TIFF:

```bash
nc2cog input.nc output/
```

Specify the output filename:

```bash
nc2cog input.nc output/my_custom_name.tif
```

### Batch Conversion

Convert all netCDF files in a directory:

```bash
nc2cog input_dir/ output/
```

### Multi-dimensional NetCDF Files

For netCDF files containing multiple variables and time dimensions (e.g., `PRE(time, lat, lon)`, `REF(time, lat, lon)`), the tool automatically detects the structure and converts each variable to a separate multi-band COG file, with time steps as bands.

```bash
# Auto-detect and convert all variables
nc2cog MPF_V4_20251113144500.nc output/
# Produces: output/PRE.tif (N bands), output/REF.tif (N bands)

# Convert only specific variables to a directory
nc2cog --variables PRE MPF_V4_20251113144500.nc output/

# Convert a single variable to a specific output file
nc2cog --variables PRE MPF_V4_20251113144500.nc output/MPF_v4_PRE.tif
# Produces: output/MPF_v4_PRE.tif (N bands, each band is a time step)
```

### Advanced Usage

With custom compression and performance settings:

```bash
nc2cog input.nc output/ \
  --compression deflate \
  --zlevel 9 \
  --tile-size 1024 \
  --block-size 512 \
  --resampling cubic \
  --overview-levels 2,4,8,16,32 \
  --metadata-source "My Satellite Data"
```

With parallel processing:

```bash
nc2cog input_dir/ output/ --threads 4
```

## ⚙️ Command Line Options

### Compression Options
- `--compression` [deflate|lzw|jpeg]: Choose compression algorithm
- `--zlevel` [1-9]: Set compression level for deflate (default: 6)

### Performance Options
- `--tile-size` INTEGER: Tile size for COG (default: 512)
- `--block-size` INTEGER: Block size for compression (default: 256)

### Pyramid/Overview Options
- `--resampling` [nearest|bilinear|cubic|...]: Resampling method for overviews (default: nearest)
- `--overview-levels` TEXT: Overview levels (comma-separated, default: 2,4,8,16)

### General Options
- `--overwrite`: Overwrite existing output files
- `--dry-run`: Show what would be processed without doing it
- `--verbose`, `-v`: Enable verbose logging
- `--resume`: Resume from last processed file
- `--threads` INTEGER: Number of parallel processing threads (default: 1)
- `--src-proj` TEXT: Source projection in EPSG format (e.g., EPSG:4326)
- `--dst-proj` TEXT: Target projection in EPSG format (e.g., EPSG:3857)
- `--variables` TEXT: Variables to convert in multi-dimensional NC files (comma-separated, e.g., `PRE,REF`). If omitted, all data variables are auto-detected.
- `--metadata-source` TEXT: Custom source name for COG metadata. If omitted, auto-detected from netCDF global attributes (`source`, `platform`, `institution`).
- `--version`, `-V`: Show version information and exit.

## 📁 Output Path Rules

The behavior of the `output_path` argument depends on its format:

| Output path ends with | Behavior |
|-----------------------|----------|
| `.tif` | Output to the specified file directly |
| `/` or no extension | Output to a directory (one file per variable for multi-dim) |

Examples:
```bash
# Directory output: creates output/PRE.tif and output/REF.tif
nc2cog MPF_V4_20251113144500.nc output/

# File output: creates the exact file specified
nc2cog --variables PRE MPF_V4_20251113144500.nc output/MPF_v4_PRE.tif
```

**Note**: File output (`.tif` ending) only works when converting a single variable. If multiple variables are specified with a `.tif` output path, only the first variable will be converted.

## 🔧 Configuration File

Create a `config.yaml` for complex setups:

```yaml
# Processing parameters
compression: "deflate"
zlevel: 6
tile_size: [512, 512]
block_size: [256, 256]

# Output options
overviews:
  resampling: "nearest"
  levels: [2, 4, 8, 16]

# Metadata options
metadata:
  source: ""        # Custom source name (auto-detected from netCDF if empty)
  offset: 0.0       # Data offset
  scale: 1.0        # Data scale factor
  unit: ""          # Data unit (auto-detected from netCDF if empty)

# Processing control
overwrite: false
skip_errors: true
```

Use with: `nc2cog --config config.yaml input.nc output/`

## 📊 GDAL Optimization

This tool eliminates common GDAL warnings by using driver-appropriate parameters:

- **GTiff driver**: Uses `BLOCKXSIZE`/`BLOCKYSIZE` instead of `TILEWIDTH`/`TILEHEIGHT`
- **COG driver**: Uses `BLOCKSIZE` instead of `BLOCKXSIZE`/`BLOCKYSIZE`
- **Overview handling**: Optimized to avoid `COPY_SRC_OVERVIEWS` conflicts
- **Clean COG Layout**: Metadata is written to the intermediate GeoTIFF **before** COG conversion. The GDAL COG driver automatically carries metadata from source to output, eliminating the need for post-creation file modifications that would break COG layout optimization (no "IFD has been rewritten" warnings).

## 📋 COG Metadata

Each output COG file contains 18 metadata fields written during the conversion pipeline:

| Field | Description | Example |
|-------|-------------|---------|
| `Coordinate System` | CRS with projection info | `WGS84 (EPSG:4326)` |
| `Band Count` | Number of bands (time steps) | `10` |
| `Data Type` | Pixel data type | `Float32` |
| `Resolution` | Pixel resolution in degrees | `0.01000000` |
| `Extent` | Bounding box (minLon, minLat, maxLon, maxLat) | `97.095, 37.295, 126.105, 53.405` |
| `Creation Time` | UTC timestamp of conversion | `2026-05-26 12:08:29` |
| `Source` | Data source name | `hfioi` |
| `Compression` | Compression algorithm | `DEFLATE` |
| `startX` / `startY` | Upper-left corner coordinates | `97.095000` / `53.405000` |
| `endX` / `endY` | Lower-right corner coordinates | `126.105000` / `37.295000` |
| `min` / `max` | Global data value range | `0.00` / `7.91` |
| `offset` / `scale` | Linear transformation parameters | `0.0000` / `1.0000` |
| `unit` | Data unit from netCDF variable | `mm` |
| `NODATA` | No-data value | `nan` |

### Metadata Source Configuration

The `Source` field is determined by the following priority:

1. **`--metadata-source` CLI parameter** (highest priority) — explicitly set source name
2. **netCDF global attributes** — auto-detected from `source`, `platform`, or `institution` attributes
3. **Empty string** (default fallback)

```bash
# Use custom source name
nc2cog --metadata-source "FY-4 Satellite" input.nc output/

# Auto-detect from netCDF attributes (default behavior)
nc2cog input.nc output/
```

### Viewing Metadata

```bash
# View all metadata fields
gdalinfo output.tif

# View metadata as JSON
gdalinfo -json output.tif | python -c "
import json, sys
d = json.load(sys.stdin)
m = d.get('metadata', {}).get('', {})
for k, v in sorted(m.items()):
    print(f'{k}: {v}')
"
```

## 🎯 Use Cases

### Climate/Meteorological Data
```bash
nc2cog climate_data.nc output/ \
  --compression deflate \
  --zlevel 9 \
  --resampling cubic \
  --overview-levels 2,4,8,16,32
```

### Oceanographic Data
```bash
nc2cog ocean_data.nc output/ \
  --compression lzw \
  --tile-size 1024 \
  --resampling bilinear \
  --overview-levels 2,4,8
```

### Large Dataset Processing
```bash
nc2cog large_dataset/ output/ \
  --threads 4 \
  --compression deflate \
  --zlevel 7 \
  --tile-size 512
```

### Projection Transformations
With the new projection support, you can transform coordinate systems during conversion:

Convert with reprojection from WGS84 to Web Mercator:
```bash
nc2cog --src-proj EPSG:4326 --dst-proj EPSG:3857 input.nc output/
```

Or specify only target projection (source will be detected automatically):
```bash
nc2cog --dst-proj EPSG:3857 input.nc output/
```

Combine with other conversion options:
```bash
nc2cog --src-proj EPSG:4326 --dst-proj EPSG:3857 \
  --compression deflate \
  --zlevel 9 \
  --tile-size 1024 \
  input.nc output/
```

## 🤝 Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests if applicable
5. Submit a pull request

## 📄 License

MIT License

## 🐛 Issues

Report issues on the GitHub repository.

## 📚 Documentation

For more detailed documentation:
- [User Guide](docs/user_guide.md)
- [GDAL Optimization Details](docs/gdal_optimization.md)
- [Usage Examples](docs/examples.md)
- [Installation Guide](INSTALLATION.md)
- [Project Structure](docs/project_structure.md)

## ℹ️ Additional Information

### Overview Level Generation

When using `--overview-levels`, note that GDAL intelligently determines which overview levels to actually create based on the source image size. For smaller images, GDAL may generate fewer overview levels than specified to avoid creating overly small and potentially useless overviews.

For example, with `--overview-levels 2,4,8,16,32` on a 1781x1572 image, GDAL may only generate levels 2 and 4 (890x786 and 445x393 pixels respectively) as the smaller levels would be too small to be useful.

---

Made with ❤️ for the geospatial community.
