Containerizing GDAL Workers with Docker

A reliable GDAL worker container starts from a base that already ships a cohesive GDAL/PROJ/GEOS trio, exports PROJ_LIB and GDAL_DATA explicitly, installs your Python spatial stack against that same native library, and proves the drivers exist with gdalinfo --formats before the image is ever trusted. This page gives a multi-stage Dockerfile for a Prefect worker running rasterio and geopandas, explains why the OSGeo base usually beats hand-building on python-slim, and ends with a smoke test you can run in one command.

When to Use This Pattern#

• You run Prefect or Dagster workers that call GDAL, rasterio, geopandas, or pyproj and need identical behavior on every host.
• Your pipeline reprojects data and cannot tolerate the datum-grid drift that comes from a missing PROJ_LIB.
• You want a small final image but a full toolchain during the build (the classic multi-stage case).
• You are standardizing worker images ahead of the CI flow described in Deployment & CI/CD for Spatial Workers.

Complete Working Example#

Two viable bases exist. Building from python:3.11-slim and compiling GDAL yourself gives maximum control but costs a long compile and the burden of keeping GDAL, PROJ, and GEOS versions mutually compatible by hand. Starting from ghcr.io/osgeo/gdal hands you a tested native trio immediately; you layer Python on top. For nearly every worker, the OSGeo base wins — it removes the single most error-prone step, the native build. The Dockerfile below takes that route with a multi-stage layout so build-only tooling never bloats the runtime image.

The case for compiling on python-slim is narrow but real. You reach for it when you need a GDAL driver the OSGeo image omits, a version newer than any published tag, or a security posture that forbids inheriting a large upstream base you did not assemble. The cost is steep: a from-source GDAL build pulls dozens of -dev packages, takes many minutes even with caching, and forces you to match libproj and libgeos versions to the GDAL you chose or face link errors. Unless one of those forcing functions applies, the OSGeo base is the pragmatic default and the rest of this page assumes it. Whichever base you pick, the deciding rule is the same: the native trio must be internally consistent, and you must be able to prove it with a real GDAL call before the image is trusted.

The image is split into a builder stage that owns the compiler and a lean runtime stage that owns only what executes. This matters for GDAL workers specifically because build-essential and the -dev headers needed to compile rasterio against the system library add hundreds of megabytes that have no business shipping to production. The builder produces wheels; the runtime installs them with --no-index from a local directory and never sees a compiler.

# ---- Stage 1: builder -------------------------------------------------
# Pin the OSGeo base by tag; the delivery pipeline additionally locks the digest.
FROM ghcr.io/osgeo/gdal:ubuntu-small-3.8.4 AS builder

# System build tools live ONLY in this stage.
RUN apt-get update && apt-get install -y --no-install-recommends \
        python3-pip python3-dev build-essential \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /build
COPY requirements.txt .

# Build wheels against the base image's system GDAL so rasterio binds to it,
# not to a bundled copy from a manylinux wheel.
RUN pip3 wheel --no-cache-dir --wheel-dir /wheels \
        --no-binary rasterio --no-binary shapely \
        -r requirements.txt

# ---- Stage 2: runtime -------------------------------------------------
FROM ghcr.io/osgeo/gdal:ubuntu-small-3.8.4 AS runtime

# Make the native support-file locations explicit and permanent. If these are
# empty, EPSG transforms silently fall back to lower-accuracy paths.
ENV PROJ_LIB=/usr/share/proj \
    GDAL_DATA=/usr/share/gdal \
    PROJ_NETWORK=OFF \
    PYTHONUNBUFFERED=1

RUN apt-get update && apt-get install -y --no-install-recommends python3-pip \
    && rm -rf /var/lib/apt/lists/*

# Bake datum-shift grids into the image so an offline prod network still
# reprojects accurately. No runtime downloads.
RUN projsync --system-directory --all || true

# Install the pre-built wheels from the builder stage — no compiler needed here.
COPY --from=builder /wheels /wheels
COPY requirements.txt .
RUN pip3 install --no-cache-dir --no-index --find-links=/wheels -r requirements.txt \
    && rm -rf /wheels

# Run as a non-root worker.
RUN useradd --create-home --uid 10001 worker
USER worker
WORKDIR /app
COPY --chown=worker:worker . /app

# Fail fast at container start if the native stack is not what we expect.
HEALTHCHECK --interval=30s --timeout=10s \
    CMD python3 -c "from osgeo import gdal; gdal.UseExceptions(); gdal.Open" || exit 1

# Start the Prefect worker; the pool name comes from the deploy environment.
CMD ["prefect", "worker", "start", "--pool", "spatial-pool"]

Three lines in that Dockerfile do the heavy lifting and reward a closer look. The ENV PROJ_LIB=... GDAL_DATA=... block makes the native support-file locations explicit rather than relying on defaults that vary between base images; a worker that inherits an empty PROJ_LIB will import cleanly and then quietly reproject through a low-accuracy path. The projsync --system-directory --all line downloads every datum-shift grid once, at build time, so the finished image reprojects identically whether it runs on a networked laptop or an air-gapped production node. And the --no-binary rasterio flag in the builder forces rasterio to link the base image’s libgdal instead of dragging in a second GDAL bundled inside a manylinux wheel — the mismatch that produces the most baffling import-time crashes.

The matching requirements.txt keeps the Python layer thin and lets the native GDAL come from the base:

# requirements.txt (pinned to match GDAL 3.8.4 from the base image)
rasterio==1.3.9
geopandas==0.14.3
pyproj==3.6.1
shapely==2.0.3
prefect==2.14.21

Keeping the Image Small#

A GDAL worker image is never going to be tiny — the native trio and its grid set are large by nature — but it should carry nothing it does not run. Four habits keep it lean. First, prefer the ubuntu-small OSGeo variant over the full one, which trims rarely-used drivers; just confirm your pipeline’s drivers survive the cut with gdalinfo --formats. Second, keep the compiler and -dev headers in the builder stage only, as the multi-stage layout above does, so they never reach the runtime layer. Third, collapse apt-get update, install, and rm -rf /var/lib/apt/lists/* into a single RUN so the package index cache is not frozen into a layer. Fourth, pass --no-cache-dir to every pip invocation and delete the /wheels directory after install.

Be deliberate about grids: projsync --all can add hundreds of megabytes. If you only ever transform within a known region, sync just the grids for those areas instead of the full set, and let the environment parity startup guard assert that the ones you need are present. The goal is not the smallest possible image but the smallest image that still reprojects accurately offline.

Parameter & Option Reference#

Verification & Testing#

Never trust the image until real GDAL has run inside it. The smoke test below checks the version, enumerates writable raster drivers, confirms a specific driver your pipeline needs, and performs a reprojection whose result is asserted.

# Build the image.
docker build -t spatial-worker:smoke .

# 1. GDAL is present and the version matches the base tag.
docker run --rm spatial-worker:smoke gdalinfo --version

# 2. The drivers the pipeline depends on actually exist.
docker run --rm spatial-worker:smoke \
    sh -c "gdalinfo --formats | grep -E 'GTiff|COG|GPKG'"

# 3. PROJ_LIB resolves and a datum-shifted transform works.
docker run --rm spatial-worker:smoke python3 - <<'PY'
from pyproj import Transformer, CRS
t = Transformer.from_crs(CRS.from_epsg(4326), CRS.from_epsg(2056), always_xy=True)
easting, northing = t.transform(7.4474, 46.9480)  # Bern
assert 2_590_000 < easting < 2_610_000, f"bad easting {easting}"
print("reprojection OK:", round(easting), round(northing))
PY

For a Python-native check you can drop into CI, this assertion sketch fails the build on any missing driver:

from osgeo import gdal

def assert_drivers(required: list[str]) -> None:
    gdal.UseExceptions()
    available: set[str] = {
        gdal.GetDriver(i).ShortName for i in range(gdal.GetDriverCount())
    }
    missing = [d for d in required if d not in available]
    assert not missing, f"container missing GDAL drivers: {missing}"

assert_drivers(["GTiff", "COG", "GPKG"])
print("driver check passed")

Run the smoke test as the final step of your local build loop and again in CI against the exact image that will be published. The two runs should be identical because the image is; if they diverge, something outside the image — a mounted volume shadowing PROJ_LIB, a host environment variable leaking in — is the culprit, and that is worth finding before deploy. The Python driver check is the version to embed in the CI pipeline described in Deployment & CI/CD for Spatial Workers, since it returns a clean non-zero exit that fails the job on any missing driver.

Common Pitfalls#

• Leaving PROJ_LIB unset. The container imports fine and most transforms look right, but datum-grid shifts fall back to a coarse path and features drift by meters. Always export it and assert it — the same failure is dissected in Environment Parity for Spatial Pipelines.
• Installing rasterio from a manylinux wheel on top of a system GDAL. You end up with two GDAL copies and non-deterministic binding. Build it --no-binary against the base library, or install from the same conda channel as covered in pinning geospatial dependencies with conda-lock.
• Downloading PROJ grids at runtime. A pipeline that works in staging fails in an air-gapped prod network. Bake grids at build time with projsync --all.
• Using the ubuntu-small base without checking drivers. The slim OSGeo variants omit some drivers to save space; a PostGISRaster or netCDF job may fail only in production. Confirm with gdalinfo --formats in the smoke test.

Related: This image is the unit that the CI flow in Deployment & CI/CD for Spatial Workers builds and verifies, its dependencies are locked by pinning geospatial dependencies with conda-lock, and its startup checks connect to Security Boundaries for Spatial Data when the worker opens a PostGIS connection. The upstream OSGeo GDAL Docker images document every base variant.

← Back to Deployment & CI/CD for Spatial Workers