nc2cog 0.1.5

Convert netCDF files to Cloud-Optimized GeoTIFF format with advanced compression

nc2cog - netCDF to Cloud-Optimized GeoTIFF Converter

中文

Convert netCDF files to Cloud-Optimized GeoTIFF format with advanced compression and performance settings.

🚀 Features

• Format Conversion: Convert netCDF files to Cloud-Optimized GeoTIFF (COG)
• Batch Processing: Process entire directories of netCDF files
• Advanced Compression: Support for deflate, lzw, and jpeg compression with configurable levels
• Performance Optimization: Configurable tile and block sizes for optimal performance
• Pyramid Structure: Customizable overview (pyramid) levels for multi-scale access
• GDAL Optimized: Eliminated GDAL warnings and optimized driver-specific parameters
• Parallel Processing: Multi-threaded conversion for faster processing
• Resume Capability: Resume interrupted conversions
• Projection Transformation: Reproject data from source to target coordinate systems during conversion
• Flexible Output: Output to a directory or directly to a specified .tif file
• Rich Metadata: 18 metadata fields automatically written to output COG (source, extent, resolution, min/max, unit, etc.)
• Clean COG Layout: Metadata written during conversion pipeline — no post-creation modifications that break COG layout optimization

📋 Requirements

• Python 3.9+

🛠️ Installation

From PyPI (Recommended)

pip install nc2cog

From Source

# Clone the repository
git clone https://github.com/cyberpsyche/nc2cog.git
cd nc2cog

# Install with uv
uv pip install .

# Or with pip
pip install .

System Dependencies

nc2cog requires GDAL with Python bindings:

• macOS: brew install gdal
• Ubuntu: sudo apt-get install gdal-bin libgdal-dev
• Windows: Use OSGeo4W installer

🚀 Quick Start

Single File Conversion

Convert a single netCDF file to COG TIFF:

nc2cog input.nc output/

Specify the output filename:

nc2cog input.nc output/my_custom_name.tif

Batch Conversion

Convert all netCDF files in a directory:

nc2cog input_dir/ output/

Multi-dimensional NetCDF Files

For netCDF files containing multiple variables and time dimensions (e.g., PRE(time, lat, lon), REF(time, lat, lon)), the tool automatically detects the structure and converts each variable to a separate multi-band COG file, with time steps as bands.

# Auto-detect and convert all variables
nc2cog MPF_V4_20251113144500.nc output/
# Produces: output/PRE.tif (N bands), output/REF.tif (N bands)

# Convert only specific variables to a directory
nc2cog --variables PRE MPF_V4_20251113144500.nc output/

# Convert a single variable to a specific output file
nc2cog --variables PRE MPF_V4_20251113144500.nc output/MPF_v4_PRE.tif
# Produces: output/MPF_v4_PRE.tif (N bands, each band is a time step)

Advanced Usage

With custom compression and performance settings:

nc2cog input.nc output/ \
  --compression deflate \
  --zlevel 9 \
  --tile-size 1024 \
  --block-size 512 \
  --resampling cubic \
  --overview-levels 2,4,8,16,32 \
  --metadata-source "My Satellite Data"

With parallel processing:

nc2cog input_dir/ output/ --threads 4

⚙️ Command Line Options

Compression Options

• --compression [deflate|lzw|jpeg]: Choose compression algorithm
• --zlevel [1-9]: Set compression level for deflate (default: 6)

Performance Options

• --tile-size INTEGER: Tile size for COG (default: 512)
• --block-size INTEGER: Block size for compression (default: 256)

Pyramid/Overview Options

• --resampling [nearest|bilinear|cubic|...]: Resampling method for overviews (default: nearest)
• --overview-levels TEXT: Overview levels (comma-separated, default: 2,4,8,16)

General Options

• --overwrite: Overwrite existing output files
• --dry-run: Show what would be processed without doing it
• --verbose, -v: Enable verbose logging
• --resume: Resume from last processed file
• --threads INTEGER: Number of parallel processing threads (default: 1)
• --src-proj TEXT: Source projection in EPSG format (e.g., EPSG:4326)
• --dst-proj TEXT: Target projection in EPSG format (e.g., EPSG:3857)
• --variables TEXT: Variables to convert in multi-dimensional NC files (comma-separated, e.g., PRE,REF). If omitted, all data variables are auto-detected.
• --metadata-source TEXT: Custom source name for COG metadata. If omitted, auto-detected from netCDF global attributes (source, platform, institution).
• --version, -V: Show version information and exit.

📁 Output Path Rules

The behavior of the output_path argument depends on its format:

Output path ends with	Behavior
.tif	Output to the specified file directly
/ or no extension	Output to a directory (one file per variable for multi-dim)

Examples:

# Directory output: creates output/PRE.tif and output/REF.tif
nc2cog MPF_V4_20251113144500.nc output/

# File output: creates the exact file specified
nc2cog --variables PRE MPF_V4_20251113144500.nc output/MPF_v4_PRE.tif

Note: File output (.tif ending) only works when converting a single variable. If multiple variables are specified with a .tif output path, only the first variable will be converted.

🔧 Configuration File

Create a config.yaml for complex setups:

# Processing parameters
compression: "deflate"
zlevel: 6
tile_size: [512, 512]
block_size: [256, 256]

# Output options
overviews:
  resampling: "nearest"
  levels: [2, 4, 8, 16]

# Metadata options
metadata:
  source: ""        # Custom source name (auto-detected from netCDF if empty)
  offset: 0.0       # Data offset
  scale: 1.0        # Data scale factor
  unit: ""          # Data unit (auto-detected from netCDF if empty)

# Processing control
overwrite: false
skip_errors: true

Use with: nc2cog --config config.yaml input.nc output/

📊 GDAL Optimization

This tool eliminates common GDAL warnings by using driver-appropriate parameters:

• GTiff driver: Uses BLOCKXSIZE/BLOCKYSIZE instead of TILEWIDTH/TILEHEIGHT
• COG driver: Uses BLOCKSIZE instead of BLOCKXSIZE/BLOCKYSIZE
• Overview handling: Optimized to avoid COPY_SRC_OVERVIEWS conflicts
• Clean COG Layout: Metadata is written to the intermediate GeoTIFF before COG conversion. The GDAL COG driver automatically carries metadata from source to output, eliminating the need for post-creation file modifications that would break COG layout optimization (no "IFD has been rewritten" warnings).

📋 COG Metadata

Each output COG file contains 18 metadata fields written during the conversion pipeline:

Field	Description	Example
Coordinate System	CRS with projection info	WGS84 (EPSG:4326)
Band Count	Number of bands (time steps)	10
Data Type	Pixel data type	Float32
Resolution	Pixel resolution in degrees	0.01000000
Extent	Bounding box (minLon, minLat, maxLon, maxLat)	97.095, 37.295, 126.105, 53.405
Creation Time	UTC timestamp of conversion	2026-05-26 12:08:29
Source	Data source name	hfioi
Compression	Compression algorithm	DEFLATE
startX / startY	Upper-left corner coordinates	97.095000 / 53.405000
endX / endY	Lower-right corner coordinates	126.105000 / 37.295000
min / max	Global data value range	0.00 / 7.91
offset / scale	Linear transformation parameters	0.0000 / 1.0000
unit	Data unit from netCDF variable	mm
NoData	No-data value	-9999.0

Metadata Source Configuration

The Source field is determined by the following priority:

1. --metadata-source CLI parameter (highest priority) — explicitly set source name
2. netCDF global attributes — auto-detected from source, platform, or institution attributes
3. Empty string (default fallback)

# Use custom source name
nc2cog --metadata-source "FY-4 Satellite" input.nc output/

# Auto-detect from netCDF attributes (default behavior)
nc2cog input.nc output/

Viewing Metadata

# View all metadata fields
gdalinfo output.tif

# View metadata as JSON
gdalinfo -json output.tif | python -c "
import json, sys
d = json.load(sys.stdin)
m = d.get('metadata', {}).get('', {})
for k, v in sorted(m.items()):
    print(f'{k}: {v}')
"

🎯 Use Cases

Climate/Meteorological Data

nc2cog climate_data.nc output/ \
  --compression deflate \
  --zlevel 9 \
  --resampling cubic \
  --overview-levels 2,4,8,16,32

Oceanographic Data

nc2cog ocean_data.nc output/ \
  --compression lzw \
  --tile-size 1024 \
  --resampling bilinear \
  --overview-levels 2,4,8

Large Dataset Processing

nc2cog large_dataset/ output/ \
  --threads 4 \
  --compression deflate \
  --zlevel 7 \
  --tile-size 512

Projection Transformations

With the new projection support, you can transform coordinate systems during conversion:

Convert with reprojection from WGS84 to Web Mercator:

nc2cog --src-proj EPSG:4326 --dst-proj EPSG:3857 input.nc output/

Or specify only target projection (source will be detected automatically):

nc2cog --dst-proj EPSG:3857 input.nc output/

Combine with other conversion options:

nc2cog --src-proj EPSG:4326 --dst-proj EPSG:3857 \
  --compression deflate \
  --zlevel 9 \
  --tile-size 1024 \
  input.nc output/

🤝 Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests if applicable
5. Submit a pull request

📄 License

MIT License

🐛 Issues

Report issues on the GitHub repository.

📚 Documentation

For more detailed documentation:

• User Guide
• GDAL Optimization Details
• Usage Examples
• Installation Guide
• Project Structure

ℹ️ Additional Information

Overview Level Generation

When using --overview-levels, note that GDAL intelligently determines which overview levels to actually create based on the source image size. For smaller images, GDAL may generate fewer overview levels than specified to avoid creating overly small and potentially useless overviews.

For example, with --overview-levels 2,4,8,16,32 on a 1781x1572 image, GDAL may only generate levels 2 and 4 (890x786 and 445x393 pixels respectively) as the smaller levels would be too small to be useful.

---

Made with ❤️ for the geospatial community.