geosint

Geospatial OSINT collection and monitoring for points of interest and organisations. Collects from public map data, records where every fact came from, reports what changed between collections, and correlates entities by shared infrastructure.

$ geosint collect --what "bureau de change" --area "Austin, TX" --case meridian

3 entities collected (0 new to this case)

ENTITY             CATEGORY          ADDRESS    PHONE            DOMAIN             CONF  TRIAGE
-----------------  ----------------  ---------  ---------------  -----------------  ----  ----------
Alpha Exchange     bureau_de_change  1 Main St  +1 555 077 2200  newholdco.example  83    Unreviewed
Beta Money Ser...  bureau_de_change  1 Main St  +1 555 010 0100  holdco.example     83    Unreviewed
Cedar Currency     bureau_de_change  Cedar Ave  +1 555 044 7788                     67    Unreviewed

  3/3 have usable coordinates.

3 field change(s) since the last collection:

WHEN              ENTITY          FIELD    WAS                     NOW                    SOURCE
----------------  --------------  -------  ----------------------  ---------------------  ------
2026-09-09 19:31  Alpha Exchange  phone    +1 555 010 0100         +1 555 077 2200        osm
2026-09-09 19:31  Alpha Exchange  website  https://holdco.example  https://newholdco.e... osm
2026-09-09 19:31  Alpha Exchange  domain   holdco.example          newholdco.example      osm

1 previously-seen entity did not reappear:
    - Northside Exchange

Scope

This collects organisations, businesses, and infrastructure from public map data — entities that publish an address and a phone number precisely so they can be found. It queries each source's own official public API, at documented rate limits, identifying itself with a contact address.

It is not a people-search tool. There is no person lookup, no cross-referencing of individuals, and none should be added. Everything it returns describes a place or an organisation.

Start here

pip install geosint
export GEOSINT_CONTACT="you@example.com"   # required, see below
geosint start

start asks five plain questions, remembers the answers, and runs your first collection. Everything below is the same thing with flags.

---

Why this rather than a scraper

Three things, and they are the whole design.

1. Provenance on every field. Each record carries its source, source reference, collection time, case, and the query that found it. geosint show prints all of it. An answer you cannot attribute is not intelligence, it is trivia.

2. Change is the product. Re-run a collection and the tool diffs it against what is stored, writing an append-only observation for every field that moved. A phone number changing, a website going dark, an entity vanishing from the map — these are findings, and a tool that just overwrites its database destroys them.

geosint collect --what pharmacy --area "Austin, TX" --case alpha   # week 1
geosint collect --what pharmacy --area "Austin, TX" --case alpha   # week 2
geosint changes --since 7d --case alpha

3. Correlation between entities. Everything else describes entities one at a time. geosint links looks for what connects them:

Link	Meaning
shared phone	Two "different" organisations answering one number
shared domain	Two entities publishing the same web host
operator / brand	Affiliation the source states outright — the strongest signal available
co-location	Entities within metres of each other

$ geosint links --case meridian

Stated operator / brand affiliations

  Affiliation: Meridian Holdings   (2 entities)
      - Alpha Exchange
      - Beta Money Services

Co-located entities (within 25m)

  Cluster: 2 entities within 25m   (2 entities)
      - Alpha Exchange
      - Beta Money Services

A shared value is a lead, not a conclusion. Shared numbers and domains have mundane explanations — a shared reception, a franchise call centre, a serviced office — at least as often as interesting ones. The tool reports the evidence and leaves the judgement to you.

---

Setup

pip install geosint                     # core
pip install "geosint[xlsx,dotenv]"      # + Excel export, .env support
pip install "geosint[bulk]"             # + offline .osm.pbf extraction

The contact address is required

OpenStreetMap's Nominatim and Overpass are run by volunteers, and their usage policy requires requests to carry a real contact so an operator can email you rather than block you.

export GEOSINT_CONTACT="you@example.com"          # macOS / Linux
$env:GEOSINT_CONTACT="you@example.com"            # Windows PowerShell

Or pass --contact per run. It is sent to OSM and to any site you explicitly choose to check with --check-websites, and nowhere else.

Optional: the Google collector

--provider google has better contact coverage but no operator, brand, ref, or wikidata fields — the metadata that makes OSM more useful for investigation. It also costs money past a free monthly allowance, and requires billing enabled with a card on file even to stay inside it. Google prices a whole response at the tier of its most expensive requested field, so --include-ratings drops the free allowance from 5,000 to 1,000 calls/month.

1. https://console.cloud.google.com → new project
2. Enable Places API (New), enable billing
3. Create an API key, restrict it to "Places API (New)"
4. export GOOGLE_PLACES_API_KEY="your-key"

--near / --radius become a Places locationRestriction rectangle around the circle (the only shape Text Search accepts there), and results are then narrowed to the true radius. The radius is capped at 50 km, since Text Search returns at most 60 results - a wider radius is refused rather than quietly clamped. Use --provider osm for a bigger sweep. --tile is osm-only.

---

Commands

Collect

# A named area (geocoded to its bounding box)
geosint collect --what pharmacy --area "Austin, TX" --case alpha

# A point and radius - more precise, and usually what you want.
# A box around a city centre is mostly not the city centre.
geosint collect --what pharmacy --near 30.2672,-97.7431 --radius 2

# Any OSM tag, for anything not in the built-in category list
geosint collect --osm-tag "office=diplomatic" --area "Vienna, Austria"
geosint collect --osm-tag "man_made=mast" --near 51.5,-0.12 --radius 10

# Dense area under-returning? Split it into a 3x3 grid (9 queries, deduped)
geosint collect --what pharmacy --area "Chicago, IL" --tile 3

# Many collections from a CSV (header: what,area[,osm_tag]).
# Each row is its own tracked collection, so re-running the batch diffs
# every row and reports what stopped appearing, per row.
geosint batch --input targets.csv --case alpha

# A whole country or region, offline, no rate limit
geosint pbf --pbf texas-latest.osm.pbf --what pharmacy --case alpha

For pbf, download the smallest extract covering your region from https://download.geofabrik.de/.

Analyse

geosint entities --case alpha                    # browse
geosint entities --indicator shared_phone        # filter by indicator
geosint entities --missing                       # what stopped appearing
geosint show "Alpha Exchange"                    # one entity + full provenance
geosint links --case alpha --tag                 # correlate, and tag the results
geosint near 30.2672,-97.7431 2                  # what's within 2km

Record judgement

geosint triage "Alpha Exchange" Flagged --note "same address as Beta"
geosint note "Alpha Exchange" "registration MSB-11021 predates the website"
geosint recheck "Alpha Exchange" +7d             # also +2w, +3m, +1y, 2026-10-01
geosint recheck                                  # what's due

Triage states: Unreviewed, Reviewing, Corroborated, Dismissed, Flagged. Anything you write — triage, notes, recheck dates, analyst — is never overwritten by a re-collection.

Monitor and report

geosint changes --since 7d --case alpha          # what moved
geosint runs                                     # collection history
geosint report --case alpha                      # indicator + link summary
geosint cases                                    # all cases
geosint export --case alpha                      # GeoJSON by default

export writes GeoJSON by default because the data is inherently geospatial — drop the file into geojson.io, QGIS, kepler.gl, or Google Earth with no conversion. --format csv|xlsx|json also available.

---

Indicators

Findings are named indicators, not a single score. A score collapses independent facts into one number and hides which fact drove it.

Indicator	Meaning
no_web_presence	No website published
website_dead	Website published but unreachable (needs --check-websites)
no_contact_details	No phone, website, or email at all
no_address / no_coordinates	Incomplete location data
operator_named	The source names who runs it
chain_affiliated	Carries a brand tag
registration_ref	Carries a licence/registration reference
wikidata_linked	Hard link to a structured record
stale_survey	Map data not verified recently
not_operational	Source reports it closed
shared_phone / shared_domain	Written by geosint links --tag

confidence is a separate number: the percentage of core fields actually populated. It measures record completeness, not how interesting the entity is — those are different questions and should not share a scale.

---

What the sources can and cannot tell you

• OSM has no rating, review, or operational-status data. Those come back as unknown, and every check skips rather than inventing a value. A blank never reads as a zero.
• Ways carry tags but not coordinates in offline pbf mode (full geometry assembly needs a location index and far more memory). They are flagged no_coordinates so the gap is visible in the data rather than silently absent. Nodes — the common case for POIs — carry their coordinates.
• --check-websites is not an audit. One GET to each entity's own published homepage, once, recording only whether it answered. Nothing is read out of the page, and it is never a crawl. Sites whose host resolves into private or loopback address space are declined rather than fetched — the URL comes from a world-writable OSM tag.
• Coverage varies enormously by region. OSM is crowdsourced; absence of an entity is weak evidence of absence in reality.
• This tool does not scrape. pbf reaches country scale through OSM's own bulk distribution channel — a bigger free, ToS-clean source, not evasion of anyone's rate limits. It does not harvest contact details off web pages either: email comes only from the source's own email/contact:email tag, published as a contact detail by a mapper.
• Spreadsheet exports are formula-escaped. An OSM name tag is world-writable, and Excel evaluates a cell beginning =, +, - or @. Values that would execute are prefixed so they stay text. GeoJSON and JSON are untouched — nothing that reads them executes them.

---

Options (collect / batch / pbf)

Flag	Default	What it does
--what	—	Category, e.g. "pharmacy"
--osm-tag	—	Any OSM tag, e.g. "office=ngo" (overrides --what)
--area	—	Place name, geocoded to a bounding box
--near / --radius	—	Centre point (lat,lon) and radius in km. google caps the radius at 50km
--case	default	Scopes entities, changes, and correlation
--contact	$GEOSINT_CONTACT	Your email/URL, sent as User-Agent
--provider	osm	osm (free, richer metadata) or google
--tile	1	osm only — N x N grid split for fuller coverage. Raise --max-results with it
--check-websites	off	Record whether each published site answers. 8 at a time
--no-store	off	One-off collection; disables change detection
--db-path	.geosint.db	Database location
--max-results	60	osm ceiling 3000; google hard cap 60
--min-confidence	0	Drop records less than N% complete
--format	geojson	geojson, csv, xlsx, json
--output	—	Write results to this file
--cache-ttl	24	Hours before a cached query refetches (0 = never reuse)
--no-cache	off	Skip the cache entirely
--dry-run	off	Collect and analyse, write nothing
--verbose / --quiet	normal	Debug logging / warnings only

A stored collection always bypasses the cache on read: it exists to answer "what changed?", and only the network can answer that. The cache key covers every flag that changes the result set, --max-results included, so raising a cap refetches rather than replaying the smaller answer.

--max-results is applied after tiling merges and dedupes, so --tile 3 with the default 60 still keeps 60 — of the whole area, ranked by record completeness, not the first tiles queried. When results are dropped to fit the cap the run says so and names the flag to raise. A radius collection keeps the nearest N instead, because there distance is a meaningful order.

---

Development

git clone https://github.com/jenilmistryhq/geosint.git
cd geosint
pip install -e ".[dev,xlsx,dotenv]"
pytest -q

388 tests, all against mocked HTTP and temporary SQLite, or a real synthetic .osm.pbf fixture — no API key, no network, no cost. (One skips unless the bulk extra is installed.) Each test runs in its own working directory with a neutralised $HOME, so nothing leaks between tests or into the repo.

Two rules this suite learned the hard way:

• Build fixtures from a real normalizer, not by hand. dedupe once keyed on field names no normalizer produced. Every test passed; every branch of every chain silently merged into one record. tests/test_dedupe.py now builds its input by calling the collectors' own _normalize.
• Mock the transport, assert on the contract. Tests that mock a client wholesale prove only that the mock works — --provider google had no end-to-end test and silently discarded --near for as long as the flag existed.

tests/test_regressions.py holds one named test per bug that produced confident, wrong output.

Layout

geosint/
├── cli.py             argument parsing + command orchestration
├── wizard.py          `start` - guided setup
├── display.py         tables, bar charts, entity cards
├── config.py          env/.env/CLI resolution + validation (one path, all commands)
├── useragent.py       required contact -> User-Agent
├── geo.py             coordinates, radius maths, GeoJSON
├── entities.py        Entity record, indicators, confidence
├── analysis.py        correlation: shared phone/domain/operator/location
├── db.py              SQLite: entities, observations, collections, aliases, actions
├── osm_client.py      Overpass + Nominatim collector
├── api_client.py      Google Places API (New) collector
├── bulk_extract.py    offline .osm.pbf extraction
├── dedupe.py          identity keys, in-memory and at the DB boundary
├── website_health.py  does the published site resolve
├── batch.py           CSV multi-collection runner
├── cache.py           per-query cache with a TTL
└── exporters.py       GeoJSON / CSV / XLSX / JSON

Data model

collections   one row per run: query, source, area, counts, timestamps
     |
entities      current best-known state, one row per entity
     |
observations  APPEND-ONLY. Every field change, with source and timestamp.
aliases       maps a second source's id onto the canonical entity
actions       analyst triage changes and notes

Adding a category? Add it to CATEGORY_TAG_MAP in osm_client.py plus a line in test_osm_client.py.

Legal and ethical note

This queries each data source's own official public API (OpenStreetMap's Nominatim and Overpass, Google's Places API) and, with --check-websites, makes a plain GET to an entity's own published homepage. It does not scrape search results, bypass authentication or CAPTCHAs, evade rate limits, or attempt to disguise itself — the contact address is mandatory precisely so it does not.

You are responsible for the lawfulness and proportionality of your collection. Public availability is not the same as a right to compile, and the answer differs by jurisdiction and by purpose. Where collected records touch personal data, data-protection law generally applies to you as the one compiling them.

License

MIT