Testing Spatial Flows in CI With Synthetic Fixtures

Real geodata makes bad fixtures: it is large, often licensed, frequently personal, and nobody can say what the right answer is for any given operation on it. Synthetic fixtures invert every one of those properties. A raster generated from a formula has a known value at every pixel, so a warp can be checked rather than merely run; a polygon built from stated coordinates has a known area; and both are small enough that the whole suite runs on every commit.

When to Use This Pattern#

• CI is slow enough that people skip it, which usually means fixtures measured in hundreds of megabytes.
• Tests assert that operations complete rather than that they are correct, because nobody knows the right answer.
• Source data is licensed or personal, so committing a sample is not an option.
• Edge cases need reproducing — antimeridian crossings, nodata, invalid rings — which real data supplies only by luck.

Complete Working Example#

A fixture factory, generating rasters and vectors whose correct answers are known by construction.

# tests/fixtures/synthetic.py
from __future__ import annotations

from pathlib import Path

import numpy as np
import rasterio
from rasterio.transform import from_origin
from shapely.geometry import Polygon, box


def ramp_raster(path: Path, width=256, height=256, crs="EPSG:32630",
                origin=(500000.0, 5600000.0), res=10.0) -> Path:
    """A horizontal ramp: pixel value == column index. Any resampling is visible."""
    data = np.tile(np.arange(width, dtype="uint16"), (height, 1))
    with rasterio.open(
        path, "w", driver="GTiff", width=width, height=height, count=1,
        dtype="uint16", crs=crs, transform=from_origin(*origin, res, res),
        tiled=True, blockxsize=128, blockysize=128, compress="deflate", nodata=0,
    ) as dst:
        dst.write(data, 1)
        dst.build_overviews([2, 4], rasterio.enums.Resampling.average)
    return path


def checkerboard_raster(path: Path, size=256, cell=16, **kw) -> Path:
    """High-frequency content: any smoothing shows up as a changed variance."""
    ii, jj = np.indices((size, size)) // cell
    data = (((ii + jj) % 2) * 255).astype("uint8")
    return _write(path, data, dtype="uint8", **kw)


def square_km(centre=(500000.0, 5600000.0), crs="EPSG:32630") -> Polygon:
    """Exactly 1 km²  in a metric CRS — so ST_Area has a known correct answer."""
    x, y = centre
    return box(x - 500, y - 500, x + 500, y + 500)


def bowtie_polygon() -> Polygon:
    """Self-intersecting: invalid by construction, for the repair path."""
    return Polygon([(0, 0), (2, 2), (2, 0), (0, 2), (0, 0)])


def antimeridian_strip() -> Polygon:
    """Crosses 180°: the case every naive bounding box gets wrong."""
    return Polygon([(179.5, -10), (-179.5, -10), (-179.5, 10), (179.5, 10)])

The tests then assert values rather than absence of exceptions:

import numpy as np
import pytest
from rasterio.enums import Resampling


def test_nearest_neighbour_preserves_exact_values(tmp_path) -> None:
    src = ramp_raster(tmp_path / "ramp.tif")
    out = warp_to(src, tmp_path / "out.tif", "EPSG:3857",
                  resampling=Resampling.nearest)
    with rasterio.open(out) as ds:
        values = np.unique(ds.read(1))
    # Nearest neighbour must not invent values that were never in the source.
    assert set(values.tolist()) <= set(range(256)), "resampling created new values"


def test_bilinear_smooths_the_checkerboard(tmp_path) -> None:
    src = checkerboard_raster(tmp_path / "check.tif")
    out = warp_to(src, tmp_path / "out.tif", "EPSG:3857",
                  resampling=Resampling.bilinear)
    with rasterio.open(src) as a, rasterio.open(out) as b:
        assert b.read(1).var() < a.read(1).var() * 0.9, "bilinear did not smooth"


def test_area_is_exactly_one_square_kilometre() -> None:
    assert square_km().area == pytest.approx(1_000_000.0, rel=1e-9)


def test_invalid_geometry_is_repaired_not_dropped() -> None:
    repaired = repair(bowtie_polygon())
    assert repaired.is_valid and repaired.area > 0


def test_antimeridian_bbox_is_not_the_whole_world() -> None:
    bounds = safe_bounds(antimeridian_strip())
    assert bounds.width < 5, f"bbox spans {bounds.width}° — the naive answer is 359"

The ramp raster is the most useful single fixture in the set, because its structure makes resampling legible. Every pixel value equals its column index, so nearest-neighbour resampling can only ever produce values that already existed, bilinear must produce intermediate values, and a shifted transform shows up as an offset in the value at a known position. A test written against a real orthophoto can assert almost nothing about any of that; the same test against the ramp is a one-line assertion on a set of unique values.

Parameter & Option Reference#

Verification & Testing#

Fixtures need testing too, or a broken generator quietly makes every assertion vacuous.

def test_ramp_has_the_values_it_claims(tmp_path) -> None:
    with rasterio.open(ramp_raster(tmp_path / "r.tif")) as ds:
        row = ds.read(1)[0]
    assert row.tolist() == list(range(256))


def test_fixtures_are_actually_tiled(tmp_path) -> None:
    with rasterio.open(ramp_raster(tmp_path / "r.tif")) as ds:
        assert ds.profile["tiled"] and ds.block_shapes[0] == (128, 128)
        assert ds.overviews(1) == [2, 4], "no overviews: coarse-read paths untested"


def test_bowtie_is_invalid_before_repair() -> None:
    assert not bowtie_polygon().is_valid, "the fixture no longer exercises the repair"


def test_generation_is_deterministic(tmp_path) -> None:
    a = sha256_of(ramp_raster(tmp_path / "a.tif"))
    b = sha256_of(ramp_raster(tmp_path / "b.tif"))
    assert a == b, "fixtures differ between runs; failures will be unreproducible"


def test_suite_fixtures_stay_small(tmp_path) -> None:
    total = sum(p.stat().st_size for p in generate_all(tmp_path))
    assert total < 5 * 1024 ** 2, f"fixture set is {total / 1e6:.1f} MB"

The determinism test is worth more than it appears. A fixture generator that uses random data without a fixed seed produces a suite where a failure cannot be reproduced — the test that failed in CI is testing different data by the time somebody runs it locally, which turns a bug into a ghost. Either avoid randomness entirely, as the generators above do, or seed it explicitly and record the seed in the failure message.

The edge-case fixtures deserve a separate argument, because they are the ones people skip as contrived. A bow-tie polygon and a strip across the antimeridian look like exercises rather than data — until a publisher delivers a parcel whose ring self-intersects by two centimetres, or a maritime boundary layer that genuinely straddles 180 degrees. Both arrive eventually, both take down a nightly run, and both were testable from the first day at a cost of four lines. The rule worth adopting is that every production incident caused by unusual geometry ends with a synthetic fixture reproducing it, which turns a class of surprise into a permanent, instant test.

That habit also solves a problem real fixtures create. When an incident is caused by a specific delivery, the tempting response is to commit that delivery as a regression fixture — which imports its size, its licence and possibly its personal data into the repository forever. Reproducing the shape of the problem synthetically keeps the regression test and leaves the data where it belongs, and in practice the synthetic version is a better test because it isolates the one property that mattered.

Common Pitfalls#

• Committing large real samples. They slow the suite, may be licensed, and still cannot say what the right answer is.
• Asserting only that nothing raised. A warp that completes and shifts everything by a pixel passes that test.
• Fixtures without tiling or overviews. The code paths that read windows and coarse levels then go untested.
• Unseeded random data. Failures become unreproducible, which is worse than no test.
• Fixtures in a degree-based CRS for area tests. Square degrees are not an area; use a metric CRS where the answer is exact.
• No invalid fixture. The repair path is then only exercised in production, by a delivery nobody controls.

Frequently Asked Questions#

Related#

• Deployment & CI/CD for spatial workers — where this suite sits in the pipeline
• Environment parity for spatial pipelines — running the tests inside the artefact
• Rolling out GDAL upgrades without breaking flows — the layer below, with real data
• Repairing invalid geometries before load — what the bow-tie fixture exercises
• Validating coordinate systems before ETL — what the metric-CRS fixture exercises

← Deployment & CI/CD for Spatial Workers