Grafana Dashboards for GIS Workflows

Grafana dashboards for GIS workflows turn raw pipeline telemetry into a single operational picture: raster throughput, tile latency percentiles, coordinate-system validation failures, dead-letter depth, and per-worker memory, all laid out so an on-call engineer can diagnose a stalled mosaic in seconds. This guide shows how to structure panels for spatial workloads, template them by pipeline and projection, and provision the whole thing as code next to your Prometheus data source.

The direct recommendation: model one dashboard per pipeline family (raster ingest, tile bake, vector ETL), drive every panel from Prometheus recording rules rather than raw series, template with pipeline, epsg, and zoom variables, and check the JSON model plus provisioning YAML into the same repository as your flow code. Treat the dashboard as a deployable artifact, not a click-ops afterthought. This page sits inside Observability & Monitoring for Geospatial Pipelines and assumes you already emit metrics via Prometheus metrics for raster throughput.

Prerequisites & Architecture Baseline#

Before you provision a single panel, confirm the telemetry pipeline underneath it is real and labelled consistently. A dashboard is only as honest as the metrics feeding it.

• Grafana 10.4+ (grafana/grafana-oss:10.4.x) with file-based provisioning enabled
• Prometheus 2.51+ scraping your Prefect or Dagster workers
• prometheus-client==0.20.0 inside every geospatial worker image
• rasterio==1.3.10 and pyproj==3.6.1 pinned in the worker environment (metric labels derive from these)
• A consistent label set across all metrics: pipeline, epsg, zoom, worker
• Recording rules for the expensive quantile queries (histogram_quantile over long windows is costly at render time)
• A dedicated Prometheus data source UID you will reference from provisioned dashboards

The label discipline is the part teams skip and regret. If gdalwarp throughput is tagged pipeline="ingest" but tile latency is tagged flow="ingest", no template variable can join them and every dashboard becomes a bespoke snowflake. Standardise labels at instrumentation time, ideally in a shared metrics module imported by every task.

One more constraint is spatial-specific: keep label cardinality bounded. It is tempting to label metrics by tile index or feature id, but a raster pipeline emits millions of tiles, and a per-tile label explodes Prometheus memory and makes every dashboard query crawl. Label by the dimensions you actually slice a dashboard on — pipeline, epsg, zoom, worker — and push high-cardinality detail like tile coordinates into traces and logs instead, where it belongs. A dashboard answers aggregate questions; per-tile forensics is a job for structured logging for geospatial flows.

Dashboard Anatomy for Spatial Pipelines#

A spatial dashboard is not a generic RED (Rate, Errors, Duration) board with a map bolted on. The signals that matter are pixel-domain and projection-domain. The SVG below shows a proven grid layout: a throughput row up top, a latency-and-correctness row in the middle, and a resource-and-backlog row at the bottom.

Core Principles for Spatial Panels#

Six principles separate a dashboard that survives an incident from one that gets ignored.

1. Panel per signal domain, not per metric. Group by what the operator is asking: “is it moving?” (throughput), “is it fast?” (latency), “is it correct?” (CRS failures), “is it healthy?” (memory/backlog). Each row answers one question.
2. Percentiles over averages for latency. A mean tile-generation time hides the tail that actually pages you. Always chart p50, p95, and p99 together from a histogram, mirroring the approach in measuring tile generation latency percentiles.
3. Rate over counter. Never plot a raw _total counter. Wrap it in rate(...[5m]) so restarts and reset don’t produce cliff artifacts.
4. Correctness is a first-class panel. CRS-validation failure rate deserves the same prominence as throughput. A pipeline that reprojects fast but silently drops the projection is worse than one that runs slow — see validating coordinate systems before ETL.
5. Thresholds encode SLOs. Colour DLQ depth and CRS failure panels with hard red/amber steps so an operator reads the state pre-attentively, before parsing any number.
6. Templating over duplication. One templated dashboard beats fifteen copy-pasted ones. Variables for pipeline, epsg, and zoom collapse the whole fleet into a single navigable surface.

Production Implementation: Panels as Code#

The panels below are expressed as PromQL, which is the substrate every Grafana panel compiles down to. Read these as the queries you paste into panel editors — or, better, bake into the provisioned JSON model shown later.

# Row 1: raster throughput — pixels processed per second, per pipeline
sum by (pipeline) (
  rate(raster_pixels_processed_total{pipeline=~"$pipeline"}[5m])
)

# Row 1: tiles baked per second, split by zoom level
sum by (zoom) (
  rate(tiles_generated_total{pipeline=~"$pipeline", zoom=~"$zoom"}[5m])
)

# Row 1: in-flight gdalwarp jobs (a gauge, not a counter)
sum by (pipeline) (gdalwarp_jobs_in_flight{pipeline=~"$pipeline"})

# Row 2: tile-generation latency percentiles from a native histogram
histogram_quantile(
  0.95,
  sum by (le, pipeline) (
    rate(tile_generation_seconds_bucket{pipeline=~"$pipeline"}[5m])
  )
)

# Row 2: CRS-validation failure RATE as a fraction of validated features
sum by (pipeline) (rate(crs_validation_failures_total{pipeline=~"$pipeline"}[5m]))
/
sum by (pipeline) (rate(crs_validation_checks_total{pipeline=~"$pipeline"}[5m]))

# Row 3: per-worker resident memory (bytes) for OOM early-warning
max by (worker) (process_resident_memory_bytes{job="geo-workers"})

# Row 3: dead-letter queue depth — geometries parked for triage
max by (pipeline) (geotask_dlq_depth{pipeline=~"$pipeline"})

The $pipeline, $epsg, and $zoom tokens are Grafana template variables. The =~"$pipeline" regex-match syntax lets a single panel serve “All” or a specific selection. The sum by (le, ...) wrapping the histogram is mandatory: histogram_quantile requires the le label to be present and aggregated correctly, and forgetting it is the single most common cause of flat-line quantile panels.

To keep these percentile queries cheap at render time, precompute them with a Prometheus recording rule so Grafana reads a single series instead of re-aggregating thousands of buckets on every refresh:

# prometheus/rules/geo_dashboard.rules.yml
groups:
  - name: geo_dashboard_recording
    interval: 30s
    rules:
      - record: pipeline:tile_generation_seconds:p95
        expr: |
          histogram_quantile(
            0.95,
            sum by (le, pipeline) (
              rate(tile_generation_seconds_bucket[5m])
            )
          )
      - record: pipeline:crs_validation:failure_ratio
        expr: |
          sum by (pipeline) (rate(crs_validation_failures_total[5m]))
          /
          clamp_min(sum by (pipeline) (rate(crs_validation_checks_total[5m])), 1e-9)

The clamp_min(..., 1e-9) guard prevents a divide-by-zero NaN when a pipeline is idle and no checks have run — without it the CRS panel flickers to “No data” during quiet periods and erodes trust in the board.

Templating with Pipeline, EPSG, and Zoom Variables#

Template variables are query-driven, so they populate themselves from live label values. Define them in the dashboard’s templating.list. The epsg variable is especially valuable for spatial fleets: it lets an operator filter every panel to, say, EPSG:3857 tiles when investigating a Web Mercator-specific defect while EPSG:4326 ingest runs clean.

{
  "templating": {
    "list": [
      {
        "name": "pipeline",
        "type": "query",
        "datasource": { "type": "prometheus", "uid": "geo-prom" },
        "query": "label_values(raster_pixels_processed_total, pipeline)",
        "includeAll": true,
        "multi": true
      },
      {
        "name": "epsg",
        "type": "query",
        "datasource": { "type": "prometheus", "uid": "geo-prom" },
        "query": "label_values(crs_validation_checks_total{pipeline=~\"$pipeline\"}, epsg)",
        "includeAll": true
      },
      {
        "name": "zoom",
        "type": "query",
        "datasource": { "type": "prometheus", "uid": "geo-prom" },
        "query": "label_values(tiles_generated_total, zoom)",
        "includeAll": true,
        "sort": 3
      }
    ]
  }
}

Note the chained filter on the epsg variable: {pipeline=~"$pipeline"} scopes the projection list to whatever pipeline is selected, so you never see EPSG codes that don’t apply to the current view. sort: 3 on zoom sorts numerically so zoom levels appear 0, 1, 2, ... 18 rather than lexically as 0, 1, 10, 11, 2.

Provisioning Dashboards as Code#

Click-built dashboards drift, get deleted, and can’t be code-reviewed. File provisioning fixes all three. Grafana reads a provider YAML at startup, then loads every JSON model from the referenced folder. Ship the data source the same way so the dashboard’s datasource.uid always resolves.

# grafana/provisioning/datasources/prometheus.yml
apiVersion: 1
datasources:
  - name: Prometheus
    uid: geo-prom
    type: prometheus
    access: proxy
    url: http://prometheus:9090
    isDefault: true
    jsonData:
      httpMethod: POST
      timeInterval: 15s

# grafana/provisioning/dashboards/geo.yml
apiVersion: 1
providers:
  - name: geospatial-pipelines
    orgId: 1
    folder: GIS Pipelines
    type: file
    disableDeletion: true
    updateIntervalSeconds: 30
    allowUiUpdates: false
    options:
      path: /var/lib/grafana/dashboards
      foldersFromFilesStructure: true

Mount your JSON models into /var/lib/grafana/dashboards, set allowUiUpdates: false so operators can’t silently override the source of truth, and the dashboard is now a reviewable artifact. The companion page building a raster pipeline Grafana dashboard walks through the full JSON model end to end.

Correlating Panels with Deploys and Traces#

A throughput dip is only actionable if you can tie it to a cause. Two techniques make a spatial dashboard diagnostic rather than merely descriptive.

First, add annotations that mark worker deploys and configuration changes directly on the time axis. When pixels-per-second drops the moment a new GDAL image rolls out, the annotation makes the regression obvious without cross-referencing a separate deploy log. Provision annotations from a Prometheus query so they appear automatically whenever a deploy_timestamp metric changes:

{
  "annotations": {
    "list": [
      {
        "name": "Worker deploys",
        "datasource": { "type": "prometheus", "uid": "geo-prom" },
        "enable": true,
        "expr": "changes(worker_build_info{pipeline=~\"$pipeline\"}[1m]) > 0",
        "iconColor": "rgba(255, 96, 96, 1)",
        "titleFormat": "Deploy: {{ version }}"
      }
    ]
  }
}

Second, add a data link from each panel to the corresponding trace view. When warp p95 spikes, an operator should click the panel and land in the trace UI filtered to slow spans. This is where dashboards and distributed tracing meet: the metric tells you that warps are slow, and OpenTelemetry tracing for spatial tasks tells you which stage — GDAL open, reproject, or write — is responsible. A data link templated with ${__value.time} and the $pipeline variable carries the operator straight from symptom to span.

Together, annotations answer “what changed” and data links answer “where is it slow”, collapsing the mean-time-to-diagnosis for a raster pipeline from minutes of tab-switching to a couple of clicks.

Step-by-Step Walkthrough#

1. Standardise labels. Audit every worker so pipeline, epsg, zoom, and worker labels are emitted identically. This is prerequisite work — templating fails without it.
2. Add recording rules. Deploy geo_dashboard.rules.yml to Prometheus so p95 latency and the CRS failure ratio are precomputed. Reload with curl -X POST http://prometheus:9090/-/reload.
3. Define template variables. Add the pipeline, epsg, and zoom query variables so the dashboard self-populates from live labels.
4. Lay out the grid. Build the three rows from the SVG: throughput, latency-and-correctness, resources-and-backlog. Keep each row answering one operator question.
5. Attach thresholds. Colour the DLQ depth and CRS failure panels with amber/red steps mapped to your SLOs so state is readable at a glance.
6. Export and provision. Export the JSON model, strip the volatile id/version fields, commit it, and mount it via the provider YAML. From here the dashboard deploys with your infrastructure.

Edge Cases & Failure Recovery#

Spatial dashboards fail in specific, diagnosable ways. These are the ones that recur.

• Flat-line quantile panels. Symptom: p95 latency renders as a constant or “No data”. Cause: the le label was dropped by an aggregation before histogram_quantile, or the metric is a summary rather than a histogram. Fix: aggregate with sum by (le, ...) and confirm the underlying metric is a _bucket series.
• CRS panel reads 100% failure during idle. Cause: divide-by-zero when no validation checks ran in the window. Fix: wrap the denominator in clamp_min(..., 1e-9) as shown, or add an OR on() vector(0) fallback.
• Template variable shows stale EPSG codes. Cause: label_values caches label sets from series that have gone stale but not yet expired. Fix: scope the variable query with the current $pipeline and rely on Prometheus’s staleness handling; reduce the variable refresh to “On Time Range Change”.
• DLQ panel undercounts. Cause: the panel sums rate() of a gauge instead of reading the gauge directly. A dead-letter depth is a level, not a flow — use max by (pipeline), not rate(). Reprocessing patterns live in reprocessing dead-letter geotasks safely.
• Memory panel misses OOM spikes. Cause: a 5-minute average smooths over the sharp allocation spike when a worker opens a large raster window. Fix: chart max_over_time(process_resident_memory_bytes[1m]) and set the panel’s minimum interval to 15s.

Configuration Reference#

The tunables below are the ones you actually adjust per environment. Defaults suit a mid-sized raster fleet.

Alert rule thresholds should mirror the panel thresholds exactly, so what an operator sees red on the board is the same condition that pages them. The dedicated walkthrough alerting on CRS validation failures in Grafana covers the alert rule, routing, and recording-rule wiring for the correctness panel.

Frequently Asked Questions#