pygiskit 0.1.5

Recipe-driven spatial data downloader for Dutch geo-data

GISKit

Recipe-driven spatial data downloader for Netherlands geo-data

⚠️ Version Policy: Odd minor versions (0.1.x, 0.3.x, 0.5.x, etc.) may introduce breaking changes as we refine the architecture. Even minor versions (0.2.x, 0.4.x, etc.) maintain backward compatibility. Pin to specific versions in production.

---

What is GISKit?

GISKit is a Python tool for downloading Dutch spatial data using simple JSON "recipes". Define what data you need and where you need it - GISKit handles the downloads and combines everything into a single GeoPackage.

Perfect for:

• Creating project underlays for construction/infrastructure projects
• Downloading base maps for GIS analysis
• Collecting spatial context data for locations
• Automating repetitive spatial data downloads

Key Features

• Recipe-Driven: Define your data needs in simple JSON
• Multi-Provider: Access 68+ datasets from 5 providers (PDOK, BAG3D, Klimaateffectatlas, DUO, NDOV)
• Service Catalog: Discover available data with search and filtering
• Smart Downloads: Automatic bbox calculation from addresses
• Single Output: Everything combined in one GeoPackage
• CRS Handling: Automatic coordinate transformation
• Export Options: GeoPackage, GeoJSON, CityJSON, optional IFC/GLB
• Climate Data: 26 climate adaptation services for heat, flood, drought analysis

Quick Start

Installation

From PyPI (recommended):

pip install pygiskit

# With IFC export support (Python 3.10-3.12 only)
pip install pygiskit[ifc]

From source:

git clone https://github.com/sanderboer/py-giskit.git
cd py-giskit
pip install -e .

Your First Recipe

Option 1: Generate from template

# Generate complete template with all providers
giskit providers json -o my_recipe.json

# Edit the JSON to keep only the datasets you need
# Then run it:
giskit run my_recipe.json

Option 2: Write from scratch

Create dam_square.json:

{
  "name": "Dam Square Buildings",
  "description": "Download buildings and roads around Dam Square in Amsterdam",
  "location": {
    "type": "address",
    "value": "Dam 1, Amsterdam",
    "radius": 500
  },
  "datasets": [
    {
      "provider": "pdok",
      "service": "bgt",
      "layers": ["pand", "wegdeel"]
    }
  ],
  "output": {
    "path": "./dam_square.gpkg",
    "format": "gpkg",
    "crs": "EPSG:28992",
    "overwrite": true
  }
}

Run it:

giskit run dam_square.json

Result: dam_square.gpkg with buildings and roads around Dam Square!

Discovering Available Data

GISKit provides a service catalog to help you discover what data is available before writing recipes.

CLI Commands

Generate complete recipe template:

GISKit has two modes for discovering available data:

🚀 Fast Mode (Default)

Shows common layer examples - instant, works offline:

giskit providers json                    # All providers
giskit providers json -p pdok           # Only PDOK
giskit providers json -o template.json  # Save to file

Output: Common examples (e.g., BGT: ["pand", "wegdeel", "waterdeel"])

🔍 Complete Mode (--fetch-layers)

Fetches ALL layers from service APIs - complete layer lists:

giskit providers json --fetch-layers                      # All providers, all layers
giskit providers json -p pdok --fetch-layers             # PDOK complete
giskit providers json --fetch-layers -o template.json    # Save complete template
giskit providers json --fetch-layers --no-cache          # Force refresh

Output: Complete lists (e.g., BGT: all 49 layers including ondersteunendwaterdeel) Performance:

• First run: ~1-2 minutes (queries 85 service APIs)
• Cached runs: <1 second (uses ~/.cache/giskit/layers/)
• Use --no-cache to force refresh

Comparison:

Mode	BGT Layers	Speed	Internet
Fast	3 examples	Instant	No
Complete	49 complete	1-2 min (first) / <1s (cached)	Yes

The generated JSON shows:

• All available providers, services, and complete layer lists
• Predefined layers for WMTS/WCS services (e.g., AHN: dsm, dtm)
• All collections for OGC Features/WFS (e.g., BGT: all 49 layers including ondersteunendwaterdeel)
• Default options (tile_format, tile_matrix_set, format)
• Cached results for fast subsequent runs (~1 second)

Example output (Fast Mode):

{
  "service": "bgt",
  "layers": ["pand", "wegdeel", "waterdeel"],
  "_note": "Common layers - use --fetch-layers for complete list"
}

Example output (Complete Mode with --fetch-layers):

{
  "service": "bgt",
  "layers": [
    "bak", "begroeidterreindeel", "begroeidterreindeel_kruinlijn",
    "bord", "buurt", "functioneelgebied", "gebouwinstallatie",
    "installatie", "kast", "kunstwerkdeel_lijn", "kunstwerkdeel_punt",
    "kunstwerkdeel_vlak", "mast", "onbegroeidterreindeel",
    "onbegroeidterreindeel_kruinlijn", "ondersteunendwaterdeel",
    "ondersteunendwegdeel", "ondersteunendwegdeel_kruinlijn",
    "ongeclassificeerdobject", "openbareruimte", "openbareruimtelabel",
    "overbruggingsdeel", "overigbouwwerk", "overigescheiding",
    "paal", "pand", "pand_nummeraanduiding", "put",
    "scheiding_lijn", "scheiding_vlak", "sensor_lijn", "sensor_punt",
    "spoor", "stadsdeel", "straatmeubilair", "tunneldeel",
    "vegetatieobject_lijn", "vegetatieobject_punt", "vegetatieobject_vlak",
    "waterdeel", "waterinrichtingselement_lijn", "waterinrichtingselement_punt",
    "waterschap", "wegdeel", "wegdeel_kruinlijn",
    "weginrichtingselement_lijn", "weginrichtingselement_punt",
    "weginrichtingselement_vlak", "wijk"
  ],
  "_note": "Complete list from API (49 layers)"
}

Edit the generated JSON to keep only the datasets you need, then run:

giskit run template.json

Python API

from giskit.catalog import (
    print_catalog,           # Browse all providers
    search_services,         # Search by keyword
    list_services_by_protocol,  # Filter by protocol type
    export_catalog_json      # Export catalog as JSON
)

# Quick overview
print_catalog()
# Output:
# 📦 PDOK
#    PDOK - Publieke Dienstverlening Op de Kaart
#    Services: 52
#    Protocols: ogc-features, wmts, wcs

# Search for specific data
results = search_services("elevation")
# → {'pdok': [{'id': 'ahn', 'title': 'Actueel Hoogtebestand Nederland', ...}]}

# Find all raster/elevation services
wcs_services = list_services_by_protocol("wcs")
# → {'pdok': ['ahn']}

# Export catalog for external tools
export_catalog_json("catalog.json", detailed=True)

Finding the Right Service

By keyword:

# Find 3D building data
search_services("3d")
# → bag3d/bag3d, pdok/3d-basisvoorziening, pdok/bag3d

# Find elevation data
search_services("elevation")
# → pdok/ahn

# Find cadastral parcels
search_services("kadaster")
# → pdok/brk

By category:

from giskit.catalog import list_services_by_category

# See all categories
by_category = list_services_by_category()
# → {'pdok': {'base_registers': ['bgt', 'bag', 'brk'],
#             'elevation': ['ahn'],
#             'infrastructure': ['nwb-wegen', ...], ...}}

# Get only elevation data
elevation = list_services_by_category("elevation")

By protocol type:

# Vector data (OGC Features)
vector_services = list_services_by_protocol("ogc-features")

# Raster/elevation data (WCS)
raster_services = list_services_by_protocol("wcs")

# Pre-rendered tiles (WMTS)
tile_services = list_services_by_protocol("wmts")

Catalog Demo

Run the interactive catalog demo:

python examples/catalog_demo.py

This shows:

• Complete catalog overview
• Search examples
• Services by protocol and category
• How to compose recipes from search results

Recipe Examples

All examples are in the recipes/ directory. Try them out:

# Download buildings and infrastructure around Curieweg, Spijkenisse
giskit run recipes/curieweg_multi_service.json

# Amsterdam Dam Square with buildings
giskit run recipes/amsterdam_dam_square.json

# Climate risk assessment for Amsterdam
giskit run recipes/amsterdam_klimaat_check.json

# Public transport stops in Amsterdam
giskit run recipes/ov_haltes_amsterdam.json

# Schools in Amsterdam
giskit run recipes/amsterdam_scholen.json

# Simple bbox example
giskit run recipes/bbox_simple.json

Urban Planning Dataset

Comprehensive data for a project location:

{
  "name": "Curieweg Project Underlay",
  "description": "Multi-service data collection for urban planning analysis",
  "location": {
    "type": "address",
    "value": "Curieweg 7a, Spijkenisse",
    "radius": 1000
  },
  "datasets": [
    {"provider": "pdok", "service": "bgt", "layers": ["pand", "wegdeel", "waterdeel"]},
    {"provider": "pdok", "service": "bag", "layers": ["pand", "verblijfsobject"]},
    {"provider": "pdok", "service": "cbs-wijken-buurten-2024", "layers": ["buurten"]},
    {"provider": "pdok", "service": "bestuurlijkegebieden", "layers": ["gemeenten"]}
  ],
  "output": {
    "path": "./curieweg.gpkg",
    "format": "gpkg",
    "crs": "EPSG:28992",
    "overwrite": true
  }
}

Output Configuration

The output section supports multiple options:

{
  "output": {
    "path": "./output.gpkg",           // Output file path
    "format": "gpkg",                   // Format: gpkg, geojson, shp
    "crs": "EPSG:28992",                // Target coordinate system
    "overwrite": true,                  // Overwrite existing file
    "layer_prefix": "data_"             // Optional prefix for layer names
  }
}

Available formats:

• gpkg - GeoPackage (default, recommended for multiple layers)
• geojson - GeoJSON (single layer only)
• shp - Shapefile (legacy support)
• json - CityJSON (for 3D data from BAG3D)

Location Types

Address with buffer:

{
  "location": {
    "type": "address",
    "value": "Curieweg 7a, Spijkenisse",
    "radius": 1000
  }
}

Point coordinates:

{
  "location": {
    "type": "point",
    "value": [4.89, 52.37],
    "crs": "EPSG:4326",
    "radius": 500
  }
}

Bounding box:

{
  "location": {
    "type": "bbox",
    "value": [120700, 487000, 120950, 487250],
    "crs": "EPSG:28992"
  }
}

Available Data Sources

Quick Discovery: Use the catalog system to search for specific data types:

from giskit.catalog import search_services, print_catalog
print_catalog()  # Browse all 68+ services across 5 providers
search_services("elevation")  # Find specific data

Current Providers

GISKit provides access to 68+ datasets from 5 different providers:

PDOK (52 services) - Dutch government spatial data BAG3D (1 service) - 3D building models in CityJSON format Klimaateffectatlas (26 services) - Climate adaptation data (heat, flooding, drought) DUO (1 service) - School locations and data (CSV format) NDOV (1 service) - Public transport stops (GTFS format)

PDOK (Platform Digitale Overheid - Netherlands)

GISKit provides access to 52 Dutch government datasets via PDOK:

Base Registries (Basisregistraties):

• BGT - Large Scale Topography (54 layers: buildings, roads, water, terrain, etc.)
• BAG - Buildings and Addresses (buildings, addresses, residence objects)
• BAG3D - 3D Building Models (LoD 1.2, 1.3, 2.2 in CityJSON format)
• BRK - Cadastral Parcels

Infrastructure:

• NWB Roads - National Road Database (road segments, junctions)
• NWB Waterways - Waterway network

Topography & Elevation:

• AHN - Actueel Hoogtebestand Nederland (elevation data via WCS)
• Luchtfoto - Aerial imagery (WMTS tiles)

Statistics & Administration:

• CBS Neighborhoods 2024 - Statistical areas (neighborhoods, districts, municipalities)
• Administrative Boundaries - Municipalities, provinces, water boards

Environment:

• Protected Areas - Nature reserves, Natura 2000
• Soil Data - Soil types, contamination

Klimaateffectatlas (Climate Effect Atlas - Netherlands)

26 climate adaptation datasets for heat stress, flooding, and drought risk assessment:

Heat Stress (6 services):

• Social vulnerability to heat
• Heat island effect
• Perceived temperature
• Distance to cooling areas
• Shade maps
• Elderly vulnerability

Flooding Risk (9 services):

• Heavy precipitation (current + 2050)
• Extreme precipitation (current + 2050)
• Safe zones and dry floors
• Groundwater flooding risk

Drought Risk (4 services):

• 10-year precipitation deficit (current + 2050)
• Nature sensitivity to drought
• Water salinization risk

Urban Characteristics (4 services):

• Green percentage per neighborhood
• Paved surface percentage
• Water percentage
• Municipality boundaries

See docs/KLIMAATEFFECTATLAS_INTEGRATION.md for complete service list.

DUO (Education Data - Netherlands)

School Locations - All schools in Netherlands with type, students, address (CSV + geocoding)

NDOV (Public Transport - Netherlands)

OV Stops - ~60,000 public transport stops across Netherlands (GTFS format)

Current providers: pdok, bag3d, klimaateffectatlas, duo, ndov

See docs/PDOK_SERVICES.md for complete catalog with all layers.

Planned Providers

• OpenStreetMap - Global POI, buildings, roads via Overpass API

CLI Commands

Run recipes:

# Execute a recipe
giskit run recipe.json

# Validate recipe syntax
giskit recipe validate recipe.json

Explore providers:

# List available providers and services
giskit providers list

# Generate complete recipe template with all providers/services
giskit providers json

# Generate template for specific provider
giskit providers json -p pdok

# Save template to file for editing
giskit providers json -o template.json

# Show PDOK service details
giskit providers info pdok

Monitor API quirks:

# Show known API quirks for providers
giskit quirks list

# Show details for specific provider
giskit quirks show pdok ogc-features

# Monitor which quirks are being applied
giskit quirks monitor

Export Formats

Built-in formats:

• GeoPackage (.gpkg) - default, recommended
• GeoJSON (.geojson)
• Shapefile (.shp)
• CityJSON (.json) - for 3D data

Optional IFC/GLB export:

Requires pip install giskit[ifc] (Python 3.10-3.12 only)

# Export GeoPackage to IFC
giskit export ifc data.gpkg output.ifc

# Convert IFC to GLB (for web viewers)
giskit export glb data.ifc output.glb

See notes/EXPORT_GUIDE.md for details.

How It Works

1. Define - Write a JSON recipe with location and datasets
2. Geocode - GISKit converts addresses to coordinates
3. Download - Fetches data from PDOK OGC API Features
4. Transform - Converts to target CRS if needed
5. Combine - Merges all layers into single GeoPackage

┌──────────────┐
│ JSON Recipe  │
└──────┬───────┘
       │
       ▼
┌──────────────┐      ┌─────────────┐
│  Geocoding   │─────▶│ PDOK Lookup │
└──────┬───────┘      └─────────────┘
       │
       ▼
┌──────────────┐
│   Download   │──────┬────────────────┐
└──────┬───────┘      │                │
       │         ┌────▼────┐      ┌────▼────┐
       │         │  BGT    │      │   BAG   │
       │         │ (54 lyr)│      │ (3 lyr) │
       │         └────┬────┘      └────┬────┘
       ▼              │                │
┌──────────────┐      │                │
│  Transform   │◀─────┴────────────────┘
└──────┬───────┘
       │
       ▼
┌──────────────┐
│ GeoPackage   │
│ (.gpkg)      │
└──────────────┘

Project Structure

giskit/
├── cli/              # Command-line interface
├── core/             # Recipe parsing, geocoding, spatial ops
├── protocols/        # OGC Features, WMTS, WCS, WFS, GTFS, CSV protocols
├── providers/        # Multi-protocol provider system
├── exporters/        # IFC/GLB export (optional)
├── config/           # YAML configurations for 68+ services
│   ├── providers/    # Provider configs (pdok, bag3d, klimaateffectatlas, duo, ndov)
│   ├── services/     # Service-specific settings
│   └── quirks/       # API compatibility fixes
└── catalog.py        # Service discovery and search

recipes/              # Example recipes ready to use
tests/                # 123 unit and integration tests
docs/                 # Provider documentation and guides

Development

Running Tests

# All tests (123 passing)
pytest

# With coverage report
pytest --cov=giskit --cov-report=html

# Specific test suites
pytest tests/unit/
pytest tests/integration/

Code Quality

# Lint code
ruff check .

# Format code
ruff format .

Documentation

• docs/PDOK_SERVICES.md - Complete PDOK service catalog (52 services)
• docs/KLIMAATEFFECTATLAS_INTEGRATION.md - Climate data guide (26 services)
• docs/EXTERNAL_SERVICES.md - Overview of all external data sources
• notes/EXPORT_GUIDE.md - IFC/GLB export instructions
• notes/QUIRKS_SYSTEM.md - API compatibility handling
• notes/BAG3D_ARCHITECTURE.md - 3D data handling
• CHANGELOG.md - Version history

For Contributors

• docs/publishing/ - PyPI publication guides
• notes/ - Technical implementation notes

Use Cases

Urban Planning:

• Project site context data (buildings, infrastructure, parcels)
• Statistical area boundaries for reports
• Base maps for presentations
• Climate adaptation planning (heat stress, flooding risk)

Construction:

• Site underlay generation
• Existing infrastructure mapping
• Environmental constraints (protected areas, water)

Climate Adaptation:

• Heat stress vulnerability assessment
• Flooding risk analysis (current + 2050 scenarios)
• Drought impact evaluation
• Green infrastructure planning

GIS Analysis:

• Batch download base data for multiple locations
• Standardized data collection workflows
• Automated updates of project data

Research:

• Reproducible spatial data downloads
• Consistent data collection methodology
• Share data requirements via recipes

Contributing

Contributions welcome! This project is in active development.

1. Check PLAN.md for current priorities
2. Create an issue for discussion
3. Submit a pull request

Credits

Built with:

• GeoPandas - Spatial data handling
• Shapely - Geometry operations
• httpx - Async HTTP client
• Typer - CLI framework
• Pydantic - Data validation

Data sources:

• PDOK - Dutch government spatial data platform
• Nominatim - OpenStreetMap geocoding

License

MIT License - See LICENSE

---

Made by A190

Simple recipes for Dutch spatial data