Security Boundaries for Spatial Data

In short: a geospatial pipeline has three boundaries that ordinary data pipelines do not emphasise enough — a coordinate can identify a person, a source file is executable input to a parser written in C, and a fan-out multiplies every credential decision by its width. Scope credentials to the job, treat precise locations as personal data by default, and validate sources before a library touches them.

The usual security posture for a data pipeline is about access: who can read the warehouse, who can write to production. That matters here too, and it is not where the spatial-specific risk lives. A latitude and longitude pair with seven decimal places locates a person to within a centimetre, and a table of them with timestamps is a movement history — which is personal data under most regimes regardless of whether a name appears anywhere. Meanwhile the pipeline ingests files from publishers it does not control and hands them to GDAL, a large C library with drivers for a hundred and forty formats, some of which fetch remote resources when asked.

A third pressure comes from the shape of the work. A fan-out over four hundred tiles means four hundred processes holding whatever credential the task was given, on workers that may be interruptible and shared. A single long-lived key with broad permissions, injected everywhere because it is simpler, is the default outcome and the thing most worth changing.

There is a reason these three tend to be handled late. None of them produces a failure during development: a broad credential works, an unvalidated source parses, and a log full of coordinates is genuinely useful for debugging. Each is discovered either during a review that somebody insisted on or during an incident that nobody wanted, and the retrofit is expensive in proportion to how many call sites the pipeline has grown. Fan-outs grow call sites quickly, which is why the practical advice is to make each control structural early — a context manager, a validation task, a logging processor — rather than a rule people are asked to remember.

Prerequisites & Architecture Baseline#

• An inventory of which layers contain precise locations tied to individuals
• A credential source that issues short-lived, scoped tokens — instance identity, OIDC, or a vault
• Object-storage prefixes structured so a policy can grant one job's slice
• Database roles separated by function: read source, write output, read-only for dashboards
• A validation boundary where untrusted files are checked before any driver opens them
• Log scrubbing decided before, not after, the first incident

Core Principles#

1. A coordinate is often personal data. Precision is the variable: a municipality centroid is not identifying, a doorstep is. Treat precision as a privacy control and reduce it deliberately when the use case allows.

2. Credentials are scoped to the job, not to the pipeline. A tile task needs read on one source prefix and write on one output prefix. Anything broader is a blast radius nobody chose.

3. Sources are untrusted input. They arrive from publishers, over networks, in formats with rich parsers. Validate structure and size before handing them to a library.

4. Drivers can be restricted. GDAL will open far more than your pipeline needs, and restricting the driver list removes attack surface for free.

5. Logs leak more than databases do. A task that logs its inputs will log coordinates, and logs are replicated to more places with weaker controls than the data ever is.

6. Least privilege is easier at the start. Retrofitting scoped credentials onto a fan-out with four hundred call sites is a project; starting with them is a decorator.

Production Implementation#

Credentials first, because everything else is easier once they are short-lived and narrow.

from __future__ import annotations

from contextlib import contextmanager
from datetime import timedelta

import boto3
from prefect import task


@contextmanager
def job_credentials(job_id: str, source_prefix: str, output_prefix: str,
                    ttl: timedelta = timedelta(minutes=30)):
    """A session that can read one prefix and write one other, for half an hour."""
    policy = {
        "Version": "2012-10-17",
        "Statement": [
            {"Effect": "Allow", "Action": ["s3:GetObject"],
             "Resource": f"arn:aws:s3:::sources/{source_prefix}/*"},
            {"Effect": "Allow", "Action": ["s3:PutObject"],
             "Resource": f"arn:aws:s3:::products/{output_prefix}/*"},
        ],
    }
    creds = boto3.client("sts").assume_role(
        RoleArn=PIPELINE_ROLE,
        RoleSessionName=f"tile-{job_id}"[:64],
        Policy=json.dumps(policy),          # narrows the role; cannot widen it
        DurationSeconds=int(ttl.total_seconds()),
    )["Credentials"]
    yield boto3.Session(
        aws_access_key_id=creds["AccessKeyId"],
        aws_secret_access_key=creds["SecretAccessKey"],
        aws_session_token=creds["SessionToken"],
    )

Then the validation boundary, which is where untrusted files stop being untrusted:

ALLOWED_DRIVERS = "GTiff GPKG FlatGeobuf PostGISRaster"
MAX_BYTES = 8 * 1024 ** 3


@task(retries=1, timeout_seconds=300)
def accept_delivery(uri: str, expected_crs: str) -> Delivery:
    size = head_size(uri)
    if size > MAX_BYTES:
        raise ValueError(f"delivery is {size / 1e9:.1f} GB — above the accepted limit")

    # Restrict what the driver may do BEFORE opening anything untrusted.
    with rasterio.Env(
        GDAL_SKIP="",                       # nothing skipped, everything not allowed is off
        OGR_DRIVER_LIST=ALLOWED_DRIVERS,
        GDAL_DRIVER_LIST=ALLOWED_DRIVERS,
        CPL_VSIL_CURL_ALLOWED_EXTENSIONS=".tif,.tiff,.gpkg,.fgb",
        GDAL_HTTP_UNSAFESSL="NO",
    ):
        with rasterio.open(uri) as src:
            if src.crs is None or src.crs.to_string() != expected_crs:
                raise ValueError(f"declared CRS {src.crs} != expected {expected_crs}")
            if src.width * src.height > 40_000 ** 2:
                raise ValueError("implausible raster dimensions")
            return Delivery(uri=uri, crs=src.crs, shape=(src.width, src.height))

The inline policy on assume_role is the mechanism worth knowing about, because it inverts the usual difficulty. Normally scoping means creating a role per job, which is unmanageable at any real number of jobs; an inline session policy instead takes an existing role and narrows it for one session, and it can only ever narrow. That means a single pipeline role with sensible bounds can issue thousands of differently-scoped sessions without any per-job administration, and a task that is compromised holds a credential that expires in thirty minutes and can touch two prefixes.

The driver restriction deserves the same explanation as the credential policy, because it is doing something less obvious than it appears. It is not primarily about blocking a known exploit; it is about reducing how much code untrusted bytes can reach. GDAL’s value is that it opens almost anything, and that value is a liability at an ingest boundary where the set of formats you legitimately accept is four. Every driver not on the list is parser code that will never run on a file a publisher sent you, and the cost of the restriction is one environment variable and an explicit error when somebody delivers a format nobody agreed to.

The size check in front of the open belongs to the same family of cheap structural controls. It costs a head request, it happens before any allocation, and it converts an entire class of resource-exhaustion problem into a message naming the file and its size. What it must not be is a check performed after opening — by then the driver has read a header, possibly followed a reference, and possibly allocated on the basis of dimensions the file claimed.

A note on the third boundary, because it is the one people find hardest to take seriously. Logs feel ephemeral and are not: they are shipped to an aggregator, retained for months, indexed for search, replicated for availability, and exposed to a much wider group than the database ever is. A pipeline whose database holds precise coordinates behind two grants and whose logs print the same coordinates on every task has not protected the coordinates; it has moved them somewhere with weaker controls and better search. The asymmetry is worth stating plainly because the fix is trivial and the reasoning is what makes anyone bother.

Step-by-Step Walkthrough#

1. Inventory the precise-location layers. Anything with sub-metre coordinates attached to a household, a vehicle or a person is personal data; write down which layers those are.
2. Replace long-lived keys with short-lived sessions, scoped per job. Start with the widest fan-out, because that is where the multiplication happens.
3. Split database roles by function. The pipeline’s writer role should not be able to read tables it does not write.
4. Put a validation task at every ingest point and restrict the driver list inside it.
5. Scrub the logging path, not the call sites. A processor in the logging configuration cannot be forgotten by a new task.
6. Refuse plain-HTTP and unverified TLS at the fetch layer, so a redirect cannot quietly downgrade an authenticated read.
7. Reduce precision at publication where the use case allows, and record the decision beside the layer definition.

Edge Cases & Failure Recovery#

A credential expires mid-task. With thirty-minute sessions and fifteen-minute tasks this is rare but real, particularly on a retry after a long queue wait. Refresh inside the task rather than passing a materialised credential across the boundary.

A publisher starts serving over plain HTTP. Rare and worth catching, because a redirect from HTTPS to HTTP is silent to most clients and turns an authenticated fetch into an unauthenticated one. Disabling unsafe TLS handling and refusing plain-HTTP sources at the validation boundary costs nothing and removes a failure nobody would notice.

A source contains a reference to another file. Several formats can point at remote resources — VRT is the obvious one — and a malicious or careless source can make your worker fetch something unexpected. Restricting allowed extensions and disabling remote references for untrusted input closes it.

A delivery is enormous. A hundred-gigabyte file where eight were expected is either a mistake or an attempt to exhaust the worker. The size check happens before the open, which is the only place it helps.

Coordinates reach a log during an incident. They will, because incident debugging is when people add logging in a hurry. A scrubbing processor in the logging configuration catches what a code review would not.

An analyst needs the precise data. Then precision reduction is not the control — access is. Keep the full-precision layer in a separate schema with its own grants, and publish the reduced one.

A layer’s classification changes. A dataset published openly for years is reclassified because a new attribute joins it to something identifying — a survey response, a customer record. The classification lives with the layer definition rather than in a document, so that the pipeline can enforce it; that is only possible if there was somewhere to put it from the start.

Someone needs the pipeline’s credentials for debugging. This is the request that undoes the whole scheme, and the answer is to make it unnecessary: a debugging session should assume the same role with the same inline policy, for fifteen minutes, through the same code path. If reproducing a task’s access requires a human to hold a long-lived key, the scoping was never real.

Configuration Reference#

Frequently Asked Questions#

Related#

• Securing PostGIS connections in workflows — the database half of the credential boundary
• Masking sensitive coordinates in task logs — the log boundary in detail
• Scoping object storage credentials per tile job — the session policy pattern in full
• Validating coordinate systems before ETL — the validation boundary’s other job
• Structured logging for geospatial flows — where the scrubbing processor is installed
• Environment parity for spatial pipelines — keeping the driver restrictions identical everywhere

← Geospatial Orchestration Architecture Fundamentals