geobatch

Polite, resumable batch geocoding for spreadsheets.

geobatch turns a CSV of thousands of addresses into coordinates (or a CSV of coordinates into addresses) reliably and without upsetting the services you depend on. It runs on the Python standard library alone and puts several providers behind one interface:

• Nominatim (public or self-hosted). The OpenStreetMap Foundation usage policy is enforced on the public instance.
• Photon (komoot public demo or self-hosted).
• Pelias (self-hosted or hosted).
• Custom HTTP template for any JSON geocoding API.
• Thin adapters for API-key services: OpenCage, Geoapify, Mapbox and Google. Keys are read only from environment variables and are never logged.

Every lookup goes through a persistent SQLite cache, per-provider token-bucket rate limiting, retries that honour Retry-After, a fallback chain and quality checks. A batch that stops halfway can be resumed: run the same command again.

pip install geobatch            # stdlib only
pip install "geobatch[pandas]"  # + DataFrame helpers

Requires Python 3.9 or newer.

Quick start (command line)

geobatch geocode addresses.csv --address-col address \
    --provider nominatim \
    --user-agent "MyProject/1.0 (https://my-project.org; gis-team@my-project.org)" \
    -o geocoded.csv

Illustrative output (abridged):

geobatch: Nominatim usage policy applies: https://operations.osmfoundation.org/policies/nominatim/ (1 req/s, results cached, at most 5000 new lookups per run)
geocoding [##############----------------] 1412/3000  47.1% cached=388 failed=3 elapsed=17:04 eta=19:10
...
geobatch forward summary
  rows            3000
  ok              2941
  not found       52
  errors          3
  skipped (empty) 4
  unique lookups  2710
  cache hits      388
  HTTP requests   2325 (retries 3)
  by provider     nominatim=2941
  by precision    locality=61, rooftop=2402, street=478
  quality flags   country_mismatch=4, duplicate_coordinates=18, region_centroid=61

The output keeps every input column and adds geobatch_status, geobatch_lat, geobatch_lon, geobatch_display_name, geobatch_precision, geobatch_confidence, geobatch_importance, geobatch_country_code, geobatch_provider, geobatch_flags, geobatch_from_cache and geobatch_error.

More examples:

# Structured columns, Germany only, fallback to Photon, GeoJSON output
geobatch geocode customers.csv --street-col street --postcode-col zip --city-col city \
    --country-codes de --provider nominatim,photon \
    --user-agent "MyProject/1.0 (https://my-project.org)" -o customers.geojson

# Self-hosted Nominatim: no public-policy limits, so raise the rate and parallelism
geobatch geocode big.csv --address-col addr --url nominatim=http://localhost:8080 \
    --rate 50 --concurrency 8 -o big.gpkg

# Reverse geocoding
geobatch reverse points.csv --lat-col lat --lon-col lon \
    --user-agent "MyProject/1.0 (https://my-project.org)" -o points_addresses.csv

# A commercial provider (key from the environment, never from the command line)
export OPENCAGE_API_KEY=...
geobatch geocode addresses.csv --address-col address --provider opencage -o out.csv

# Cache maintenance
geobatch cache stats
geobatch cache clear --provider photon
geobatch cache clear --expired

# Provider terms and key variables, quality flag meanings
geobatch providers
geobatch flags

Input files: UTF-8 CSV, including the BOM that Excel writes. The delimiter (, ; tab |) is detected automatically, so semicolon "CSV" exports from European Excel work as they are. --excel writes the output with a BOM so Excel shows accented characters correctly. Output format follows the file extension: .csv, .tsv, .geojson or .gpkg. GeoPackage is written with the standard-library sqlite3 as an OGC GeoPackage 1.3 point layer in EPSG:4326, so GDAL is not needed. QGIS and GDAL read it directly.

Exit codes: 0 done, 2 usage or configuration error (including policy refusals), 3 aborted after too many consecutive failures, 130 interrupted (a partial output is still written).

Responsible use

Free geocoders are run by volunteers and small teams on donated hardware. A careless batch job can get your IP blocked, and it also costs everyone else capacity. geobatch builds the rules in rather than leaving them to the documentation.

More rules that apply to every provider:

• Keys are read from environment variables only and never accepted as arguments. They are not part of cache keys, and they are replaced by *** in every error message.
• Retries use exponential backoff with jitter, and a server's Retry-After always wins. A 429 pauses the provider's token bucket for all threads, not just the thread that received it. If a server asks you to wait more than max_retry_after (300 s by default; usually a used-up daily quota), geobatch gives up on that lookup instead of hammering or blocking for hours.
• Circuit breaker: after 25 consecutive failed lookups the batch stops, because the service is down or blocking you.
• De-duplication: identical addresses (after normalisation) are looked up once.
• Attribution: OpenStreetMap-based results (Nominatim, Photon, most Pelias installations) are © OpenStreetMap contributors under the ODbL. Credit them wherever you publish the data.

For large or recurring jobs, run your own Nominatim, Photon or Pelias, or use a commercial provider. Self-hosted instances have no built-in limits in geobatch. Set --rate and --concurrency to what your server can handle.

Python API

from geobatch import Geocoder, Nominatim, Photon, Query, SearchOptions, QualityConfig

ua = "MyProject/1.0 (https://my-project.org)"
geocoder = Geocoder(
    [Nominatim(user_agent=ua), Photon(user_agent=ua)],   # fallback chain
    cache="geocache.sqlite",                             # default: per-user cache dir
    options=SearchOptions(country_codes=("de", "at"), language="en"),
    quality=QualityConfig(min_confidence=0.5, expected_countries=("de", "at")),
)

result = geocoder.geocode_many(
    [
        "Pariser Platz 1, 10117 Berlin",
        Query(street="Stephansplatz 3", postcode="1010", city="Wien", country="Austria"),
        {"city": "Hamburg"},
    ],
    progress=True,
)
for rec in result:
    if rec.ok:
        r = rec.result
        print(r.lat, r.lon, r.display_name, r.precision, r.confidence, rec.flags)
    else:
        print(rec.status, rec.error)
print(result.summary.format())

# one-offs
rec = geocoder.geocode("Brandenburger Tor, Berlin")
rec = geocoder.reverse(52.5163, 13.3777)

GeocodeResult fields: provider, lat, lon, display_name, components (normalised keys such as house_number, road, postcode, city, state, country, country_code), confidence (0..1), importance, bbox (min_lon, min_lat, max_lon, max_lat), precision (rooftop, interpolated, street, postcode, locality, region, country, unknown), result_type and raw (the provider's original JSON). If a provider gives no confidence score (Nominatim, Photon), confidence is derived from precision.

Reading and writing files:

from geobatch import read_table, build_queries, write_output

columns, rows = read_table("addresses.csv")
queries = build_queries(columns, rows, street_col="street", city_col="city", postcode_col="zip")
result = geocoder.geocode_many(queries, rows=rows)
write_output(result.records, "addresses.gpkg")

With pandas (pip install "geobatch[pandas]"):

from geobatch.pandas_ext import geocode_dataframe, reverse_dataframe

df = geocode_dataframe(df, geocoder, address_col="address", expected_country_col="iso2")
df.attrs["geobatch_summary"]

Custom HTTP template provider

Any JSON API can be described by URL templates and dotted JSON paths:

from geobatch import TemplateProvider

inhouse = TemplateProvider(
    "https://geo.internal/find?address={query}&token={env:GEO_TOKEN}",
    reverse_url="https://geo.internal/reverse?lat={lat}&lon={lon}&token={env:GEO_TOKEN}",
    results_path="data.matches", lat_field="y", lon_field="x",
    display_field="label", confidence_field="score", confidence_scale=0.01,
    name="inhouse", rate=10, concurrency=4,
)

The available placeholders are {query}, {street}, {city}, {county}, {state}, {postcode}, {country}, {country_codes}, {language}, {limit}, {lat}, {lon} and {env:NAME}. A parameter that uses {env:...} is redacted from logs and kept out of the cache key. GeoJSON FeatureCollections are recognised automatically. On the command line, use --provider template --template-url ....

How it works

Normalisation. Before lookup and before computing cache keys, addresses are cleaned: Unicode NFKC (full-width digits, non-breaking spaces), control characters, collapsed whitespace, surrounding quotes, doubled or trailing separators, and spreadsheet null tokens (NaN, N/A, -) become empty. Wording is never rewritten. Cache keys are case-insensitive.

Cache and resume. The SQLite cache (WAL mode, thread-safe) is keyed by provider and endpoint, query kind, normalised query and search options. It stores found results and "not found" answers, and each one is committed as soon as it arrives. Network errors are never cached. If a batch is interrupted (Ctrl+C, crash, laptop lid), run the same command again: finished lookups come from the cache and only the rest goes to the network. In Python, BatchInterrupted.result holds the partial result. The default location is the per-user cache directory (%LOCALAPPDATA%\geobatch, ~/Library/Caches/geobatch or ~/.cache/geobatch). Override it with --cache or GEOBATCH_CACHE.

Rate limiting and concurrency. Each provider has a token bucket (rate requests per second, no initial burst) and a concurrency limit, shared by all worker threads.

Fallback. Providers are tried in order. The next one is asked when a provider fails, and also when it finds nothing (turn that off with fallback_on_not_found=False / --no-fallback-on-not-found).

Biasing. country_codes, bbox and bounded are translated into each provider's own parameters, for example Nominatim countrycodes/viewbox/bounded, Pelias boundary.*/focus.point, Geoapify filter/bias and Google components/bounds. Structured queries use native structured endpoints where they exist (Nominatim, Pelias, Geoapify, Mapbox) and are joined into one line elsewhere.

Quality flags

Development

pip install -e ".[dev,pandas]"
pytest --cov=geobatch
ruff check . && ruff format --check src tests && mypy src

The test suite never touches the real network. It runs a local stdlib HTTP server (tests/mockserver.py) that imitates every provider's JSON, including 429/503 with Retry-After, malformed JSON, timeouts, rate-limit timing, cache hits, resume and fallback.

License

MIT © 2026 Yusuf Eminoğlu. See LICENSE. To cite geobatch, see CITATION.cff.