Geospatial Orchestration Hub

Repairing Invalid Geometries Before Load

ST_MakeValid will return something for almost any input, and that is exactly why it should not be applied blindly. Classify the defect first — self-intersection, unclosed ring, duplicate vertex, degenerate sliver — apply the narrowest repair that addresses it, then verify that the repaired geometry still means what the original did by comparing area, type and vertex count. Anything the repair changes beyond a tolerance is not a fix, it is a different feature, and it belongs in quarantine rather than in the table.

When to Use This Pattern

  • A load into a typed, validity-checked PostGIS column is failing on a small fraction of features.
  • The source is known to produce self-intersections — hand-digitised data, or an export from a tool with looser rules than the simple-features specification.
  • Dropping the bad features is not acceptable, because they are real parcels, buildings or boundaries that someone will look for.
  • A previous blanket ST_MakeValid produced surprises — geometry collections in a polygon column, or areas that changed by more than rounding.

Complete Working Example

The repairer classifies, fixes narrowly, and verifies. Every branch records what it did.

from __future__ import annotations

from dataclasses import dataclass
from enum import Enum
from typing import Optional

import shapely
from shapely.geometry.base import BaseGeometry


class Defect(str, Enum):
    NONE = "none"
    SELF_INTERSECTION = "self_intersection"
    RING_SELF_INTERSECTION = "ring_self_intersection"
    DUPLICATE_VERTICES = "duplicate_vertices"
    TOO_FEW_POINTS = "too_few_points"
    UNKNOWN = "unknown"


@dataclass(frozen=True)
class RepairResult:
    defect: Defect
    repaired: Optional[BaseGeometry]
    area_change: float          # relative, 0.0 means unchanged
    type_changed: bool
    accepted: bool
    note: str


# A repair that changes the area by more than this is not a repair.
AREA_TOLERANCE = 1e-6


def classify(geom: BaseGeometry) -> Defect:
    if shapely.is_valid(geom):
        return Defect.NONE
    reason = shapely.is_valid_reason(geom) or ""
    lowered = reason.lower()
    if "ring self-intersection" in lowered:
        return Defect.RING_SELF_INTERSECTION
    if "self-intersection" in lowered:
        return Defect.SELF_INTERSECTION
    if "too few points" in lowered:
        return Defect.TOO_FEW_POINTS
    if "repeated point" in lowered or "duplicate" in lowered:
        return Defect.DUPLICATE_VERTICES
    return Defect.UNKNOWN


def repair(geom: BaseGeometry) -> RepairResult:
    defect = classify(geom)
    if defect is Defect.NONE:
        return RepairResult(defect, geom, 0.0, False, True, "already valid")

    if defect is Defect.TOO_FEW_POINTS:
        # Not repairable: a two-point "polygon" has no area to preserve. Quarantine.
        return RepairResult(defect, None, 0.0, False, False,
                            "degenerate: fewer points than the type requires")

    original_area = geom.area
    original_type = geom.geom_type

    if defect is Defect.DUPLICATE_VERTICES:
        # The narrowest possible fix: a zero-distance simplify drops repeated
        # points without touching the shape.
        fixed = shapely.simplify(geom, 0.0)
    else:
        # make_valid is the general tool. It is correct and it is blunt: a bow-tie
        # becomes a MultiPolygon, and a sliver may vanish entirely.
        fixed = shapely.make_valid(geom)

    if fixed.is_empty:
        return RepairResult(defect, None, 1.0, False, False,
                            "repair produced an empty geometry — the feature had no area")

    area_change = (
        abs(fixed.area - original_area) / original_area if original_area > 0 else 0.0
    )
    type_changed = fixed.geom_type != original_type

    if area_change > AREA_TOLERANCE:
        return RepairResult(defect, fixed, area_change, type_changed, False,
                            f"repair changed the area by {area_change:.2%} — quarantined")

    return RepairResult(defect, fixed, area_change, type_changed, True,
                        f"repaired ({defect.value}); type "
                        + ("changed to " + fixed.geom_type if type_changed else "unchanged"))

Normalising the type afterwards is what keeps the load from failing on a surprise:

def to_multipolygon(geom: BaseGeometry) -> Optional[BaseGeometry]:
    """Coerce a repair result into the column's declared type, or refuse."""
    if geom.geom_type == "MultiPolygon":
        return geom
    if geom.geom_type == "Polygon":
        return shapely.MultiPolygon([geom])
    if geom.geom_type == "GeometryCollection":
        # make_valid on a bow-tie can return a collection containing lines and
        # points alongside the polygons. Keep only the polygonal parts.
        polygons = [g for g in geom.geoms if g.geom_type in ("Polygon", "MultiPolygon")]
        if not polygons:
            return None
        return shapely.union_all(polygons)
    return None          # a LineString where a polygon was expected: quarantine it
Defect class decides the repairDuplicate vertices take a zero-distance simplify. Self-intersections take make_valid. Too few points cannot be repaired. A repair that changes the area is quarantined regardless of class.defectnarrowest repairoutcomeduplicate verticessimplify(0.0)acceptedring self-intersectionmake_valid → MultiPolygonacceptedsliver, zero areamake_valid → emptyquarantinedtoo few pointsnone possiblequarantined
Two of the four classes end in quarantine, which is the point: a repairer that accepts everything has stopped being a validator.

The classification step is worth defending, because “just call make_valid and check the area” would be shorter. Two things are lost by skipping it. First, the narrow repairs are genuinely better: a zero-distance simplify on duplicate vertices preserves the exact coordinates, where make_valid may reconstruct the ring and move nothing visibly but change the WKB, which then breaks any downstream comparison keyed on geometry bytes. Second, the defect class is the most useful thing to aggregate: a delivery whose failures are all RING_SELF_INTERSECTION came out of one tool with one bug, and telling the publisher that is far more actionable than telling them nine features were invalid.

The class distribution names the causeOf one hundred and thirty-seven invalid features, one hundred and twenty-four share a single defect class, which points at one upstream export tool rather than at the data.invalid features in one delivery, by defect classring self-intersection124duplicate vertices9too few points3unknown1 — read this one“137 invalid features” is a number. This is a conversation with the publisher.
The single unknown row is the one worth opening by hand — it is either a new defect class worth classifying or a genuine oddity worth understanding.

Parameter & Option Reference

Parameter Type Default Spatial notes
AREA_TOLERANCE float 1e-6 Relative. Floating-point reconstruction moves the area in the twelfth digit; anything larger changed the feature.
simplify(0.0) — — Removes repeated points without moving any vertex. The narrowest repair available, and the right one for duplicates.
make_valid — — Correct and blunt. Use it for genuine topology defects and always check the type and area afterwards.
type coercion MultiPolygon — Promote polygons, filter collections to their polygonal parts, refuse anything else.
empty result quarantine — An empty repair means the feature had no area to preserve. That is a data question, not a fix.
is_valid_reason — — The classification input. Its text is stable enough to match on, and it names the offending coordinate.

Verification & Testing

Each defect class gets a fixture and an assertion about what the repair preserved.

import shapely


def test_bowtie_becomes_multipolygon_with_the_same_area() -> None:
    bowtie = shapely.from_wkt("POLYGON((0 0, 2 2, 2 0, 0 2, 0 0))")
    result = repair(bowtie)
    assert result.defect is Defect.RING_SELF_INTERSECTION
    assert result.accepted
    assert result.repaired.geom_type == "MultiPolygon"
    assert result.type_changed
    assert result.area_change < AREA_TOLERANCE


def test_duplicate_vertices_take_the_narrow_repair() -> None:
    dup = shapely.from_wkt("POLYGON((0 0, 1 0, 1 0, 1 1, 0 1, 0 0))")
    result = repair(dup)
    assert result.repaired.area == dup.area
    assert not result.type_changed


def test_degenerate_is_quarantined_not_repaired() -> None:
    sliver = shapely.from_wkt("POLYGON((0 0, 1 1, 0 0))")
    result = repair(sliver)
    assert not result.accepted
    assert result.repaired is None


def test_a_repair_that_changes_the_area_is_refused(pathological_geometry) -> None:
    result = repair(pathological_geometry)
    if result.area_change > AREA_TOLERANCE:
        assert not result.accepted, "an area-changing repair must never be accepted"

In PostGIS the equivalent inspection is one query, and it is the fastest way to size the problem before writing any Python at all:

-- How many, of what kind, and where?
SELECT (ST_IsValidDetail(geom)).reason AS reason,
       count(*)                        AS features,
       ST_Extent(geom)                 AS where_they_are
FROM   parcels_stage
WHERE  NOT ST_IsValid(geom)
GROUP  BY 1
ORDER  BY 2 DESC;
The area check is what separates a fix from a changeAlmost all repairs change the area only in the twelfth decimal place. A small tail changes it by more than a per cent, and those features are quarantined rather than loaded.1e-121e-61e-31e-1relative area change produced by the repairAREA_TOLERANCEGreen is reconstruction noise. Red changed the feature, and no amount of it should reach the table unreviewed.
The gap between the two populations is wide, which is what makes a single tolerance workable. If your data has no gap, the repair strategy is wrong for it.

Common Pitfalls

  • Applying make_valid to everything. It works, and it silently turns some polygons into collections and some slivers into nothing. Classifying first costs one call to is_valid_reason and keeps the surprises visible.
  • Not checking the geometry type afterwards. A GEOMETRYCOLLECTION in a geometry(MultiPolygon) column fails at the load, several steps from the repair, with an error that names the column. Coerce and refuse explicitly.
  • Ignoring the area. A repair that changes the area by five per cent has changed the feature. The area comparison is two lines and is the only thing separating a repair from a silent edit.
  • Repairing after reprojection. Transform first and the repair operates on coordinates a projection has already distorted near the source’s edges. Repair in the source CRS, then transform — the order used in building ETL chains for vector data.
  • Discarding the original. Once repaired, the evidence is gone. Store the original WKB with the repair record, so the decision can be revisited when GEOS changes its interpretation or someone disputes the result.

Frequently Asked Questions

Should repairs happen in Python or in PostGIS?

PostGIS is faster for bulk work and ST_MakeValid is the same GEOS implementation, so for a staging table already loaded it is the natural choice. Python wins when the repair needs branching logic per defect class, which is exactly what this recipe does. A common split is to classify in SQL to size the problem, then repair in Python where the policy lives.

What tolerance should the area check use?

Start at 1e-6 relative and look at the distribution. Reconstruction noise sits many orders of magnitude below that, so the tolerance is not delicate — its job is to separate two populations that are usually far apart. If your data shows a continuum rather than a gap, that is itself a finding: the geometries are being changed rather than fixed, and the repair strategy needs revisiting.

Does a repair need to be recorded per feature?

Yes, and cheaply: the defect class, the area change and whether the type changed. Three small columns on the load record answer the question that comes up months later — “was this parcel’s boundary edited by us or by the publisher?” — which is otherwise unanswerable and occasionally contractual.

What about topology across features?

This recipe fixes each geometry in isolation, which is the right scope for a load. Gaps and overlaps between features are a different problem, needing a topological model rather than per-feature repair, and attempting it during a load is how pipelines acquire hour-long steps that nobody can explain. Validate it separately and report it, rather than fixing it inline.

← Spatial Validation & Sync Tasks

Related in this section