Right-Sizing Workers for Raster Mosaic Jobs

To right-size a worker for a raster mosaic job, compute the decoded peak memory as width × height × bands × dtype_size, divide by a safe utilization fraction to get a memory ceiling, then pick the smallest instance tier that clears it — and if the ceiling is too high, switch to windowed reads so peak memory tracks the window instead of the whole scene. This page gives the exact formula, runnable Python, and a gdalinfo-based verification so you never OOM-kill an undersized worker or pay for an oversized one.

When to Use This Pattern#

Reach for explicit peak-memory sizing when any of these hold:

• You are provisioning a dedicated or spot worker pool for rasterio.merge, gdalwarp, or a tiled mosaic build.
• Jobs are OOM-killed (exit code 137) even though CPU sits idle — a classic under-RAM, over-CPU mismatch.
• You cannot explain why a given mosaic needs the instance tier it is running on.
• You are moving batch raster work onto cheaper, smaller spot instances and need to know how small you can safely go.
• Input scene dimensions, band count, or dtype vary across the batch and you want a per-job tier rather than one worst-case tier for all.

Complete Working Example#

The function below estimates full-scene peak memory from a raster’s metadata (no pixels read), derives a memory ceiling, checks it against a candidate instance, and — if the scene is too large — computes the largest block-aligned window that fits the budget. It reads only headers, so it is cheap enough to run as a gate before every mosaic task.

from dataclasses import dataclass
from typing import Optional

import numpy as np
import rasterio
from rasterio.io import DatasetReader

# Fraction of instance RAM a single read buffer may safely occupy.
# The remainder is headroom for GDAL's block cache, warp buffers, and the OS.
SAFE_UTILIZATION: float = 0.65


@dataclass
class SizingResult:
    peak_full_bytes: int          # decoded size of a full-scene read
    budget_bytes: int             # usable RAM on the candidate instance
    fits_full_read: bool          # can we read the whole scene at once?
    max_window_px: Optional[int]  # largest square window side if we cannot


def estimate_peak_bytes(width: int, height: int, bands: int, dtype: str) -> int:
    """Peak resident bytes for one full read: width * height * bands * dtype_size.

    GeoTIFFs are compressed on disk but decoded to a dense array in RAM, so the
    on-disk file size is irrelevant — only the decoded array size sets the ceiling.
    """
    dtype_size: int = np.dtype(dtype).itemsize
    return width * height * bands * dtype_size


def size_worker(src_path: str, instance_ram_gb: float) -> SizingResult:
    with rasterio.open(src_path) as src:  # type: DatasetReader
        peak: int = estimate_peak_bytes(src.width, src.height, src.count, src.dtypes[0])
        budget: int = int(instance_ram_gb * 1e9 * SAFE_UTILIZATION)
        fits: bool = peak <= budget

        max_window: Optional[int] = None
        if not fits:
            # Bytes per pixel across all bands.
            bpp: int = src.count * np.dtype(src.dtypes[0]).itemsize
            # Largest square window (in pixels per side) whose buffer fits the budget.
            max_window = int((budget / bpp) ** 0.5)
            # Snap down to the file's internal block size to avoid re-reading blocks.
            block_x: int = src.block_shapes[0][1]
            if block_x > 0:
                max_window = (max_window // block_x) * block_x
        return SizingResult(peak, budget, fits, max_window)


if __name__ == "__main__":
    # A 40k x 40k, 3-band uint16 scene on an 8 GB worker.
    result = size_worker("s3://imagery/scene_40k.tif", instance_ram_gb=8.0)
    print(f"full-scene peak : {result.peak_full_bytes / 1e9:.2f} GB")
    print(f"usable budget   : {result.budget_bytes / 1e9:.2f} GB")
    print(f"fits full read  : {result.fits_full_read}")
    if not result.fits_full_read:
        print(f"use windows <=  : {result.max_window_px} px per side")

For the 40000×40000 three-band uint16 example, the full-scene peak is 40000 × 40000 × 3 × 2 = 9.6 GB, which exceeds the 8 × 0.65 = 5.2 GB budget — so the function returns a maximum block-aligned window (roughly 25000 px per side snapped to the block grid, or smaller if you want more headroom for warp buffers). Choosing a 2048-px window keeps peak read memory near 2048 × 2048 × 3 × 2 ≈ 25 MB, letting the whole job run comfortably on the 8 GB tier instead of demanding a 128 GB one.

Parameter & Option Reference#

Verification & Testing#

Verify the metadata your estimate relies on with gdalinfo, then assert the arithmetic in a test.

Inspect the source scene’s dimensions, band count, dtype and internal block size:

gdalinfo s3://imagery/scene_40k.tif | grep -E "Size is|Band|Type|Block"
# Size is 40000, 40000
# Band 1 Block=512x512 Type=UInt16, ColorInterp=Gray

Confirm the estimator matches the hand calculation and that the windowed path caps memory:

import numpy as np


def test_peak_bytes_matches_hand_calc() -> None:
    # 40000 * 40000 * 3 bands * 2 bytes (uint16) = 9.6 GB
    peak = estimate_peak_bytes(40000, 40000, 3, "uint16")
    assert peak == 40000 * 40000 * 3 * 2
    assert peak == 9_600_000_000


def test_scene_does_not_fit_8gb() -> None:
    result = size_worker("s3://imagery/scene_40k.tif", instance_ram_gb=8.0)
    assert not result.fits_full_read
    assert result.max_window_px is not None
    # A windowed buffer at the returned side must fit the budget.
    bpp = 3 * np.dtype("uint16").itemsize
    assert result.max_window_px ** 2 * bpp <= result.budget_bytes

To validate empirically, run the mosaic under /usr/bin/time -v (Linux) and confirm Maximum resident set size stays under the instance RAM — if peak RSS approaches the physical limit, lower the window size or SAFE_UTILIZATION and rerun.

Common Pitfalls#

• Sizing from on-disk file size. A 400 MB compressed GeoTIFF can decode to 9.6 GB in RAM. Always size from decoded width × height × bands × dtype_size, never from the file’s byte size on object storage.
• Ignoring dtype width. Casting a uint8 pipeline to float32 for analysis quadruples peak memory silently. Check src.dtypes and keep the narrowest dtype that preserves your values, and never drop the nodata value when you do.
• Unaligned windows. Choosing a window size that is not a multiple of the file’s internal block (src.block_shapes) makes GDAL re-read overlapping blocks, inflating both I/O and egress. Snap window sides down to the block grid.
• Forgetting warp/merge overhead. rasterio.merge and gdalwarp allocate output and resampling buffers on top of the read buffer, so a budget that exactly equals the read size still OOMs. The 0.65 utilization fraction exists precisely to absorb this — do not raise it toward 1.0.

Related: This how-to backs the sizing decisions in cost optimization for spatial compute, feeds the checkpoint boundaries in checkpointing large raster mosaics, and complements how to structure a DAG for raster processing. See the rasterio windowed reading docs for the block-alignment details.

← Back to Cost Optimization for Spatial Compute