pk_address_checker — Documentation

Lenient, deterministic Pakistani address checker. The registry database (CSVs) ships inside this package — install and check offline, no downloads, no update run required.

pip install pk_address_checker

Address checking only reads the data files, so any install can check addresses. update is different: it rewrites the data files in place inside the installed package, so the installed location must be writable by your account. Install somewhere writable if you plan to update the data later:

# a virtual environment (recommended):
python -m venv pkvenv

# activate it — Windows:
pkvenv\Scripts\activate

# activate it — Linux/macOS:
source pkvenv/bin/activate

# then install:
pip install pk_address_checker

# or skip the venv and use your user site-packages:
pip install --user pk_address_checker

If your install is not writable and you cannot reinstall, you can point update at a different data folder instead — see "Using a different data folder" below.

To use in your python code:

from pk_address_checker import check_address, is_valid_address

result = check_address("Shop 12, Tehsil Mankera, Rahim Yar Khan")
print(result["valid"])          # False
print(result["reason_codes"])   # ['tehsil_city_conflict']

is_valid_address("Street 5, Gulistan-e-Johar, Karachi")  # True

To use from Robot Framework:

*** Settings ***
Library    pk_address_checker

*** Test Cases ***
Check An Address
    ${result}=    Check Address    Shop 12, Main Bazar, Chak 45-JB, Tehsil Shorkot, District Jhang    province=Punjab
    Should Be True    ${result}[valid]

Reject A Contradiction
    ${result}=    Check Address    Shop 12, Tehsil Mankera, Rahim Yar Khan    province=Punjab
    Should Not Be True    ${result}[valid]
    Should Contain    ${result}[reason_codes]    tehsil_city_conflict

Quick Check
    ${ok}=    Is Valid Address    Street 5, Gulistan-e-Johar, Karachi
    Should Be True    ${ok}

Robot Framework turns each function into a keyword automatically (check_address → Check Address, is_valid_address → Is Valid Address). The dict result is read with ${result}[key].

Stdlib only for checking addresses (re, csv, difflib). Tested on Python 3.14 only — modern Python (3.11+) is expected to work, but older versions are untested. pdfplumber (installed automatically as a dependency) powers the data-update pipeline's PDF parsers.
Ambiguity never causes rejection — only stated contradictions reject.
Refresh the bundled data later with python -m pk_address_checker update — install somewhere writable first (see above) so the update can rewrite the data files in place.

Using a different data folder

By default the data files live inside the installed package, in its pk_admin_data folder. To use a different folder instead (for example, to share one data folder between projects, or because the package is installed somewhere read-only), set the PK_ADMIN_DATA_DIR environment variable before importing pk_address_checker — it is read once, at import time. When it is set, both address checking and update read and write that folder.

The package never copies or creates its data files there on its own — a fresh folder stays empty. Seed it yourself: copy the installed package's pk_admin_data folder (all its CSV files) into your folder. Find that folder with:

python -c "import pk_address_checker, os; print(os.path.join(os.path.dirname(pk_address_checker.__file__), 'pk_admin_data'))"

Point the variable only at a folder that already has the data files. Without them the checker loses the registry it compares against: real contradictions slip through undetected, and valid addresses can be wrongly rejected. Once the data files are in place, update refreshes them in that folder and leaves the installed package's own copy untouched.

PowerShell: $env:PK_ADMIN_DATA_DIR = "C:\pkdata"
Python, before the import: os.environ["PK_ADMIN_DATA_DIR"] = r"C:\pkdata"

Updating the data

The package ships with a snapshot of the registry tables. To refresh them from the official online sources — the PBS census tables, housing-authority scheme lists, OpenStreetMap, Wikipedia and the Pakistan Post API — run:

python -m pk_address_checker update            # refresh in place
python -m pk_address_checker update --force    # also accept a much smaller result than before

This needs internet access. The update runs its phases one after another — admin tables, societies, OpenStreetMap neighborhoods, Wikipedia landmarks, postcodes and a Google phase — and prints what it found for each. The city-by-city phases save their progress after every city, so an interrupted update keeps everything it had already fetched. New rows are merged into the existing tables inside the package's pk_admin_data folder (or the PK_ADMIN_DATA_DIR folder, when set); old rows are never deleted, and a source that fails (offline, or its page layout changed) keeps its previous rows — an update can never leave you with less data than before.

update rewrites the data files in place, so install somewhere writable first (see above). If your install is read-only: first copy the installed package's pk_admin_data files into a writable folder, then set PK_ADMIN_DATA_DIR to that folder (see "Using a different data folder" above) — the update writes there instead.
pdfplumber, the PDF parser for the census and authority documents, is installed automatically with the package.

API Reference

Every function in the library, with its full documentation.

Quick start

from pk_address_checker import check_address, is_valid_address

# Plain True / False
is_valid_address("Shop 5, Skardu, Gilgit-Baltistan")   # True
is_valid_address("asdf qwerty")                         # False

# Full result, with the reasons an address was rejected
result = check_address("Street 12, House 12, Karachi")
result["reason_codes"]                                  # ['generic_address']

check_address returns a dict with the keys valid, reasons, reason_codes and parsed. See check_address for every reason code.

Public API (7) — The functions and the class you call directly.
Data tables (5) — Loaders for the CSV-backed registries that the checks rely on.
Update pipelines (7) — Functions that download source data and rebuild the CSV registries (run by the CLI update command).
Conflict checks (10) — Checks that detect contradictions inside an address, such as a tehsil that is not in the stated district. Most return a reason string, or None when nothing conflicts.
Parsing helpers (15) — Internal helpers that read the address text: finding cities, anchors, markers and postcodes.
Matching and registry helpers (20) — Internal helpers for fuzzy name matching and for looking up the cached registries.
Download and parse helpers (22) — Internal helpers that download and parse the source documents (PDFs, HTML pages, web APIs) behind the update pipelines.
Nested helpers (8) — Methods of the two HTML parser classes.

Public API

The functions and the class you call directly.

Name	What it does
`check_address`	Lenient plausibility check of one Pakistani address.
`is_valid_address`	True when the address passes, False when it is rejected.
`parse_address`	Extract the parts of an address deterministically (no model: every field is found in the text itself, so nothing can hallucinate).
`Parsed`	The parts extracted from one address by `parse_address()`.
`tehsil_district_conflict`	True only when the tehsil is known, the stated district is itself a real district, AND none of the tehsil's known districts resembles it.
`normalize_province`	Map a free-text province string to its canonical name.
`norm`	Normalise a tehsil/district name to a comparable token.

`check_address`

Lenient plausibility check of one Pakistani address.

check_address(address: str, province: str | None = None) -> dict

How it works

Empty/whitespace address → {"valid": False, "reasons": ["address is empty"], "reason_codes": ["empty_address"], "parsed": None}.
parse_address() extracts the fields; every rejection below is added as a (code, reason) pair and the final "valid" is simply whether any pair fired:
- a. foreign_country: parse says another country is stated.
- b. gibberish: no readable place vocabulary.
- c. no_place: no city/tehsil/district in the text, no known city word, no numbered place unit — judged on the TEXT so nothing can false-fire it.
- d. not_locatable / generic_address / city_conflict: when the text fails to pin a specific point AND there is no tehsil/district AND no non-registry city (a small city/village is only not_locatable when BOTH tehsil and district are missing — a stated tehsil or district pins the area); else a two-city contradiction (_city_mismatch); else, in a STRICT big city, generic_address when there is neither a (named anchor + physical marker) nor a town anchor.
- e. province_conflict: the stated province (from the form argument and/or the text) contradicts any extracted place field.
- f. tehsil_district_conflict, district_conflict, city_district_conflict, tehsil_city_conflict, tehsil_conflict: the contradiction checks; when any fires, the vague "not_locatable" pair is dropped (a stated contradiction IS the verdict).
- g. sector_conflict, locality_conflict, society_conflict, postcode_conflict.
Duplicate pairs are removed; the result dict carries "valid", the parallel "reasons" (human text) and "reason_codes" (stable codes) lists, and the parsed fields via asdict(p).

Parameters

address — Raw address text (any case).
province — Optional province selected elsewhere in your form; free text, normalised internally, and checked together with any province in the text.

Returns: dict — {"valid": bool, "reasons": [...], "reason_codes": [...], "parsed": {...}}; reasons and reason_codes are empty when valid. Codes: empty_address, foreign_country, gibberish, no_place, not_locatable, generic_address, city_conflict, province_conflict, tehsil_district_conflict, district_conflict, city_district_conflict, tehsil_city_conflict, tehsil_conflict, society_conflict, sector_conflict, locality_conflict, postcode_conflict.

Side effects: None.

Raises: Never raises (every stage degrades to "no evidence" on bad input).

Example

>>> check_address("Shop 12, Main Bazar, Chak 45-JB, Tehsil Shorkot, District Jhang", province="Punjab")["valid"]
True
>>> check_address("Postal Code: 54830, Karachi")["valid"]   # 54830 belongs to Lahore
False
>>> check_address("Street 12, House 12, Karachi")["reason_codes"]
['generic_address']

Why it works this way: Lenient by design: in a strict city a bare "Street 12, House 12" needs a named neighbourhood/town/sector/landmark (generic_address); smaller cities and villages stay lenient ("Chak 12, Tehsil X" is enough). Society/locality checks fire only for OFFICIAL places (authority-approved or well-known official geography) — those cannot exist in an unregistered city; unofficial-but-real places (e.g. Ghauri Town, Islamabad) pass.

Related: Called by is_valid_address(); calls parse_address() and every _*_conflict check.

`is_valid_address`

True when the address passes, False when it is rejected.

is_valid_address(address: str, province: str | None = None)

How it works

Run check_address().
Return True only when "valid" is exactly True.

Parameters

address — Raw address text.
province — Optional province selected elsewhere in your form.

Returns: bool — True = valid, False = rejected. Always a plain bool — there is no third state.

Side effects: None.

Raises: Never raises.

Example

>>> is_valid_address("Shop 5, Skardu, Gilgit-Baltistan")
True
>>> is_valid_address("asdf qwerty")
False
>>> is_valid_address("Hajira City, District Attock, District Quetta", "Khyber Pakhtunkhwa")
False
>>> is_valid_address("Street 13, Saddar, Rawalpindi", "Punjab")
True
>>> is_valid_address("Street 13, Saddar, Islamabad")   # no Saddar in the sector city
False
>>> is_valid_address("Street 232 GPO Islamabad")   # GPO anchors in the sector city
True

Related: Thin wrapper around check_address() for callers that want a plain bool.

`parse_address`

Extract the parts of an address deterministically (no model: every field is found in the text itself, so nothing can hallucinate).

parse_address(address: str) -> pk_address_checker.Parsed

How it works

Province: the LAST PROVINCE_ALIASES mention in the canonicalised text wins (the terminal province statement is the stated one — "Bank of Punjab, Lahore, Khyber Pakhtunkhwa" claims KPK), unless a road/place word follows the alias ("Islamabad Road, Khushab" is a road name).
District/tehsil: explicit markers in both orders ("Tehsil Shorkot" /"Shorkot Tehsil", "District X" / "X District"); district tries the LEADING order first (in "PO Hajira Dist Sudhnoti" the trailing order would capture "hajira" as the district). Captures are greedy multi-word, cut at suffix words, trimmed to a known name. The two fields parse INDEPENDENTLY — a district marker must not suppress the tehsil marker ("..., Tehsil X, District Y" states both). An unregistered marker value ("Tehsil Hassan Abdal" trimmed to "Hassan") is nulled so it cannot suppress the registry walk.
City: the earliest positioned _city_hints word (district/business mentions excluded). Fallback: a Title-Case phrase directly before a province/country word or postcode ("Village X, Choondiko, 66170, Sindh"), rejected when it is a known city, shorter than 4, or contains generic/business words.
Registry tehsil/district recognition (no markers): walk words; each field fills independently (a stated district must not suppress tehsil recognition — "PO Hajira Dist Sudhnoti" IS a tehsil claim). Multi-word tehsils match as whole phrases (longest span, cut at suffix words, locality names skipped, all-generic phrases skipped); exact registry names win, then near-miss spellings resolve at the 88 bar. Guards: business/road follower words ("Palandri Rd", "Multan Sohan Halwa", "store" ~ "astore"), a preceding locality-type word ("muhalla naseerabad"), a Title-Case proper noun before a tehsil word ("Badami Bagh" is a locality tail), and the cross-province guard (a registry word belonging to another province than the stated city is a locality name — unless comma-stated, which IS a place statement). A fuzzy district fills only when it would not contradict the filled tehsil (tehsil_district_conflict gate: fuzzy only ADDS acceptance).
No city/tehsil/district at all: lenient place fallback — with at least two Title-Case (or ALL-CAPS, grain-market style) non-generic names, the last one becomes the city ("Tatapani Rd Hajira Poonch" → Poonch).
Locality via _locality_from_text; a locality made only of registry tehsil/district tokens ("Hajira Poonch" is tehsil+district) is nulled.
Set country_is_pakistan_or_unstated (no _foreign_country hit), is_gibberish (_is_gibberish), and locatable (locality, _locatable markers, plus code, or known-postcode anchor).

Parameters

address — Raw address text (any case).

Returns: Parsed — The extracted fields; None means "not found in the text".

Side effects: None.

Raises: Never raises (empty or unreadable input yields a Parsed of Nones).

Example

>>> parse_address("Shop 12, Main Bazar, Chak 45-JB, Tehsil Shorkot, District Jhang, Punjab")
Parsed(country_is_pakistan_or_unstated=True, is_gibberish=False, locatable=True,
       business_name=None, locality='Tehsil Shorkot District', city='Jhang',
       tehsil='Shorkot', district='Jhang', province='Punjab')

Why it works this way: Lenient by design: ambiguity never causes rejection, and every guard above exists to stop a mention that is NOT a claim (business, road, locality tail) from being treated as one.

Related: Called by check_address() and the conflict checks (_tehsil_city_conflict consumers); calls _canonicalize, _city_hints, _locality_from_text, _foreign_country, _is_gibberish, _locatable, _postcode_anchor.

`Parsed`

The parts extracted from one address by parse_address().

Parsed(
    country_is_pakistan_or_unstated: bool = True,
    is_gibberish: bool = False,
    locatable: bool = True,
    business_name: str | None = None,
    locality: str | None = None,
    city: str | None = None,
    tehsil: str | None = None,
    district: str | None = None,
    province: str | None = None,
) -> None

Fields

country_is_pakistan_or_unstated — True when no foreign country/city is stated at the end of the text. None is never used here (default True).
is_gibberish — True when the text has no readable words or only keyboard-mash tokens (default False).
locatable — True when a specific point is pinned (locality, street/landmark marker, plus code or known postcode). None is never used here (default True).
business_name — Absorbs shop/plaza/restaurant names; the parser always leaves it None.
locality — The area/township name found in the text, or None.
city — The city found in the text, or None.
tehsil — The tehsil found in the text, or None.
district — The district found in the text, or None.
province — The province found in the text, or None.

None on locality/city/tehsil/district/province means "not found in the text" — the checks treat a missing field as no evidence, never as a contradiction.

`tehsil_district_conflict`

True only when the tehsil is known, the stated district is itself a real district, AND none of the tehsil's known districts resembles it.

tehsil_district_conflict(tehsil, district, min_score: float = 88) -> bool

How it works

Missing tehsil or district → False.
Resolve the tehsil through _alias/norm; if unknown, accept the BEST fuzzy match in the registry only when it scores >= min_score, else False.
Resolve the district; if it is not a real district even fuzzily, return False (a stray place name like "Shorkot" stated as a district must stay lenient).
Return False when any of the tehsil's registered districts matches the stated district exactly or fuzzily (>= min_score); True otherwise.

Parameters

tehsil — Tehsil name (may be None or '').
district — District name (may be None or '').
min_score — Fuzzy acceptance bar (default 88).

Returns: bool — True = the pair contradicts the registry; False = consistent, unknown, or not enough evidence.

Side effects: None.

Raises: Never raises (an empty table yields no conflict).

Example

>>> tehsil_district_conflict("Shorkot", "Sialkot")
True
>>> tehsil_district_conflict("Shorkot", "Jhang")
False
>>> tehsil_district_conflict(None, "Jhang")
False

Related: Called by check_address() and parse_address() (fuzzy district fill).

`normalize_province`

Map a free-text province string to its canonical name.

normalize_province(text: str | None) -> str | None

How it works

Empty/None → None.
Strip non-letter chars and lowercase; an exact PROVINCE_ALIASES key returns the canonical name ("kpk" → "Khyber Pakhtunkhwa").
Otherwise scan longer aliases as whole words inside the text ("Sindh Province" → "Sindh").
Unknown text → None.

Parameters

text — Free-text province name, or None.

Returns: Optional[str] — The canonical province name, or None for empty/unknown text.

Side effects: None.

Raises: Never raises.

Example

>>> normalize_province("kpk")
'Khyber Pakhtunkhwa'
>>> normalize_province("Sindh Province")
'Sindh'
>>> normalize_province("somewhere")
None

Related: Called by province_conflict(), check_address(), parse_address().

`norm`

Normalise a tehsil/district name to a comparable token.

norm(s: str) -> str

How it works

Lowercase; every non-alphanumeric char becomes a space.
_SUFFIX strips trailing "tehsil"/"tahsil"/"taluka"/"taluk"/"district"/"city"/"town"/"cantt(cantonment)"/"sub-division" words.
All remaining whitespace is removed.

Parameters

s — Any tehsil/district name string.

Returns: str — The squashed lowercase token ('Shor Kot Tehsil' and 'SHORKOT' both become 'shorkot').

Side effects: None.

Raises: Never raises.

Example

>>> norm("Dera Ghazi Khan")
'deraghazikhan'
>>> norm("Shor Kot Tehsil")
'shorkot'

Related: Called by nearly every registry lookup (tehsil_district_conflict, _place_provinces, parse_address, the conflict checks).

Data tables

Loaders for the CSV-backed registries that the checks rely on.

Name	What it does
`load_admin_table`	Load the full district/tehsil table with hand-maintained overrides applied.
`load_society_table`	Load (name, location) society rows with hand-maintained overrides applied.
`load_locality_table`	Load (name, city) locality rows with hand-maintained overrides applied.
`load_islamabad_sectors`	Load the Islamabad sector grid with hand-maintained overrides applied.
`load_postcodes`	Load all known 5-digit Pakistani postcodes (existence only).

`load_admin_table`

Load the full district/tehsil table with hand-maintained overrides applied.

load_admin_table() -> tuple

How it works

Read admin_units.csv as (province, district, tehsil) keys.
Apply admin_overrides.csv: action=add inserts the key, action=remove discards it.
Return a sorted tuple; the result is cached for the process lifetime (lru_cache) and only the update pipeline clears it.

Returns: tuple — Sorted (province, district, tehsil) triples.

Side effects: None (reads two CSVs; the lru_cache holds the result until cleared).

Raises: Never raises (missing CSVs read as empty).

Example

>>> ("Azad Kashmir", "Poonch", "Hajira") in load_admin_table()
True

Related: Called by tehsil_district_conflict(), _place_provinces(), parse_address() and most conflict checks.

`load_society_table`

Load (name, location) society rows with hand-maintained overrides applied.

load_society_table() -> tuple

How it works

Read societies.csv rows lowercased.
Apply society_overrides.csv: rows hand-edited with unquoted commas spill into an extra field, which is joined back onto the location; action=remove filters out the name, action=add appends it.
Return a sorted tuple (cached for the process lifetime).

Returns: tuple — Sorted (name, location) pairs.

Side effects: None (reads two CSVs; the lru_cache holds the result until cleared).

Raises: Never raises (missing CSVs read as empty).

Example

>>> ("chaudhari muhammad ali housing scheme", "faisalabad") in load_society_table()
True

Related: Called by _locality_from_text(), _named_anchor(), _locality_conflict(), _society_conflict().

`load_locality_table`

Load (name, city) locality rows with hand-maintained overrides applied.

load_locality_table() -> tuple

How it works

Read localities.csv lowercased; skip rows whose name is itself a known city (bbox-overlap junk), a universal place name (_UNIVERSAL_NAMES: "Main Bazar", "Saddar" — they exist in every city), equal to the row's own city, or contains a comma (concatenated fragments, never a real place name).
Apply locality_overrides.csv: action=remove filters the name, action=add appends it.
Return a sorted tuple (cached for the process lifetime).

Returns: tuple — Sorted (name, city) pairs.

Side effects: None (reads two CSVs; the lru_cache holds the result until cleared).

Raises: Never raises (missing CSVs read as empty).

Example

>>> ("hayatabad", "peshawar") in load_locality_table()
True

Why it works this way: Only official, well-known geography is registered; informal/unofficial area names are deliberately absent, so unknown names pass everywhere.

Related: Called by _locality_from_text(), _locality_index(), _locality_conflict(), _locality_norms() and parse_address().

`load_islamabad_sectors`

Load the Islamabad sector grid with hand-maintained overrides applied.

load_islamabad_sectors() -> frozenset

How it works

Read islamabad_sectors.csv uppercased into a set.
Apply islamabad_sector_overrides.csv: action=remove discards the sector, action=add inserts it (values uppercased).
Return a frozenset. Not cached: the update pipeline may rewrite the CSV mid-run.

Returns: frozenset — Uppercase sector codes like "F-8".

Side effects: None.

Raises: Never raises (missing CSVs read as empty).

Example

>>> "F-8" in load_islamabad_sectors()
True

Related: Called by _marker() and _sector_conflict().

`load_postcodes`

Load all known 5-digit Pakistani postcodes (existence only).

load_postcodes() -> frozenset

How it works

Read the postcode column of postcodes.csv into a frozenset.

Returns: frozenset — Postcode strings ("54830", ...). Details live in the CSV.

Side effects: None (the lru_cache holds the result until cleared).

Raises: Never raises (a missing CSV reads as empty).

Example

>>> "74000" in load_postcodes()
True

Related: Called by _marker(), _named_anchor(), _postcode_anchor().

Update pipelines

Functions that download source data and rebuild the CSV registries (run by the CLI update command).

Name	What it does
`update_admin_table`	Download and rebuild the district/tehsil table (`admin_units.csv`) from the official PDFs.
`update_society_table`	Rebuild `societies.csv` from the official housing-authority sources.
`update_neighborhoods_osm`	OpenStreetMap phase of the neighborhood pipeline: union city neighborhoods + landmarks into `localities.csv` with per-city incremental commits.
`update_neighborhoods_wiki`	Wikipedia landmark phase of the neighborhood pipeline: union city landmark names into `localities.csv` with per-city incremental commits.
`update_world_landmarks`	Wikipedia landmark phase for world cities: union foreign landmarks into `localities.csv` with per-city incremental commits.
`update_postcodes`	Harvest Pakistan Post postcode records into `postcodes.csv` and merge their area names into `localities.csv`.
`update_neighborhoods`	Scrape Google Search results for each big city's neighborhoods and union them into `localities.csv`.

`update_admin_table`

Download and rebuild the district/tehsil table (admin_units.csv) from the official PDFs.

update_admin_table(sources=None, force: bool = False) -> dict

How it works

Read the existing table and index it by (province, district, tehsil).
Per province in sources: fetch the first working PDF, extract its text, parse it. Failures are per-province: an empty parse, or a fresh parse less than half the previous row count (unless force=True), keeps the province's OLD rows and is recorded in failed — a broken link never wipes existing data.
Merge with UNION semantics: old rows are kept (people still use old addresses), and a freshly parsed tehsil row whose tehsil already belongs to a DIFFERENT district in the old table is dropped (the yearbook layouts attach boundary rows ambiguously, and a wrong pair is worse than a missing one).
Refuse an empty or tehsil-less result outright (RuntimeError).
Write via a .tmp file and atomic replace; clear the load caches.

Parameters

sources — Optional {province: [candidate URLs or paths]} map; defaults to PBS_SOURCES.
force — Accept a fresh parse even when it is much smaller than the previous one.

Returns: dict — {"total", "added", "removed", "failed", "updated"} — failed maps each failing province to its error text; added/removed are the sorted key triples.

Side effects: Downloads PDFs into the cache, writes meta.json (ETags), rewrites admin_units.csv, clears the load_admin_table and _index caches.

Raises: RuntimeError when nothing could be loaded, or when no province contributes any tehsil row.

Example

>>> update_admin_table(force=False)["total"] >= 750
True

Why it works this way: Union-not-replace and drop-the-ambiguous-pair are deliberate: old addresses must keep validating, and a wrong tehsil/district pairing is worse than a missing one.

Related: Called by the CLI (python -m pk_address_checker update); feeds load_admin_table().

`update_society_table`

Rebuild societies.csv from the official housing-authority sources.

update_society_table(sources=None, force: bool = False) -> dict

How it works

Load existing rows; legacy "Punjab" source labels become "Punjab (PHATA)".
Per source: fetch, parse; an empty parse or a fresh parse less than half the previous row count (unless force=True) records the failure and keeps that source's old rows — a broken page never wipes existing data.
Merge with UNION semantics: rows of successful sources replace their previous rows by (name, location) key, rows of failed sources and hand-added rows are kept, identical entries collapse into one.
Refuse an empty result outright (RuntimeError); write via a .tmp file and atomic replace; clear the load cache.

Parameters

sources — Optional {label: [(url, kind, default_city)]} map; defaults to SOCIETY_SOURCES.
force — Accept a fresh parse even when it is much smaller than the previous one.

Returns: dict — {"total", "failed", "updated"} — failed maps each failing source label to its error text.

Side effects: Downloads into the cache, writes meta.json, rewrites societies.csv, clears the load_society_table cache.

Raises: RuntimeError when nothing could be loaded.

Example

>>> update_society_table(force=False)["total"] > 0
True

Why it works this way: Union-not-replace: people still use old scheme addresses, so old rows stay valid; explicit removals go through society_overrides.csv only.

Related: Called by the CLI; feeds load_society_table().

`update_neighborhoods_osm`

OpenStreetMap phase of the neighborhood pipeline: union city neighborhoods + landmarks into localities.csv with per-city incremental commits.

update_neighborhoods_osm(cities=None, force: bool = False) -> dict

How it works

Rebuild clean (name, city, source) rows from localities.csv.
Per city: _osm_neighborhoods() (Nominatim bbox + Overpass queries); require 5..1000 rows else fail; a fresh parse less than half the previous osm rows fails unless force=True; failures print and keep the city's previous rows.
Rows get source="osm" and union in; after EACH city the CSV is rewritten (incremental commit — progress survives interruption and stays observable), then sleep 0.5s.
Clear the locality cache at the end.

Parameters

cities — Optional list of city names; defaults to _BIG_CITIES.
force — Accept a fresh parse even when it is much smaller than the previous one.

Returns: dict — {"total", "failed", "updated"} — failed maps each failing city to its error text.

Side effects: Nominatim + Overpass HTTP requests, prints per-city progress, rewrites localities.csv once per city, sleeps, clears the load_locality_table cache.

Raises: Never raises (per-city failures are recorded and skipped).

Example

>>> update_neighborhoods_osm(cities=["Skardu"])["failed"]
{}

Related: Called by the CLI; sibling of update_neighborhoods_wiki() and update_world_landmarks().

`update_neighborhoods_wiki`

Wikipedia landmark phase of the neighborhood pipeline: union city landmark names into localities.csv with per-city incremental commits.

update_neighborhoods_wiki(cities=None, force: bool = False) -> dict

How it works

Rebuild clean (name, city, source) rows from localities.csv.
Per city: _wiki_landmarks(); require 5..1000 rows else fail; a fresh parse less than half the previous wikipedia rows fails unless force=True; failures print and keep the city's previous rows (cities without landmark categories fail leniently).
Rows get source="wikipedia" and union in; after EACH city the CSV is rewritten (incremental commit), then sleep 0.5s.
Clear the locality cache at the end.

Parameters

cities — Optional list of city names; defaults to _BIG_CITIES.
force — Accept a fresh parse even when it is much smaller than the previous one.

Returns: dict — {"total", "failed", "updated"} — failed maps each failing city to its error text.

Side effects: MediaWiki API requests, prints per-city progress, rewrites localities.csv once per city, sleeps, clears the load_locality_table cache.

Raises: Never raises (per-city failures are recorded and skipped).

Example

>>> update_neighborhoods_wiki(cities=["Lahore"])["failed"]
{}

Related: Called by the CLI; sibling of update_neighborhoods_osm().

`update_world_landmarks`

Wikipedia landmark phase for world cities: union foreign landmarks into localities.csv with per-city incremental commits.

update_world_landmarks(cities=None, force: bool = False) -> dict

How it works

Rebuild clean (name, city, source) rows from localities.csv.
Per city: _wiki_landmarks(); require 3..1000 rows else fail; failures print and keep the city's previous rows.
Rows get source="wikipedia-world" and union in; after EACH city the CSV is rewritten (incremental commit), then sleep 0.5s.
Clear the locality cache at the end.

Parameters

cities — Optional list of world city names; defaults to _WORLD_CITIES.
force — Accepted for interface parity with the other update_* functions; not consulted in this one.

Returns: dict — {"total", "failed", "updated"} — failed maps each failing city to its error text.

Side effects: MediaWiki API requests, prints per-city progress, rewrites localities.csv once per city, sleeps, clears the load_locality_table cache.

Raises: Never raises (per-city failures are recorded and skipped).

Example

>>> update_world_landmarks(cities=["Paris"])["failed"]
{}

Why it works this way: The landmarks land in the registry with the FOREIGN city, so any Pakistani city hint then contradicts them ("Statue of Liberty, Karachi" becomes impossible).

Related: Called by the CLI; feeds _landmark_names() and _locality_conflict().

`update_postcodes`

Harvest Pakistan Post postcode records into postcodes.csv and merge their area names into localities.csv.

update_postcodes(force: bool = False) -> dict

How it works

Query every province plus every big/strict city (the province listing caps at ~200 rows, so city queries are needed for coverage); each query fails independently and keeps previous rows (a query returning fewer than 20 rows fails).
Union rows into postcodes.csv keyed by postcode (postcode, city, province, area_name) via a .tmp file and atomic replace.
Merge every area_name into localities.csv with source="postcode" (union, dedupe, hand-edited rows kept) — the area names feed the locality registry, and contradictions fire through the EXISTING locality_conflict with no new rejection logic.
Clear the postcode + locality caches.

Parameters

force — Accepted for interface parity with the other update_* functions; not consulted in this one.

Returns: dict — {"total", "failed", "updated"} — failed maps each failing query label to its error text.

Side effects: HTTP requests to the postcode API, prints per-query progress, rewrites postcodes.csv and localities.csv, clears the load_postcodes and load_locality_table caches.

Raises: Never raises (per-query failures are recorded and skipped).

Example

>>> update_postcodes()["total"] > 1000
True

Related: Called by the CLI; feeds load_postcodes() and load_locality_table().

`update_neighborhoods`

Scrape Google Search results for each big city's neighborhoods and union them into localities.csv.

Note: this pipeline is currently dead — Google returns consent/bot pages — but stays wired for when it works again.

update_neighborhoods(cities=None, force: bool = False) -> dict

How it works

Rebuild clean (name, city, source) rows from localities.csv (hand edits with stray commas are dropped: they would crash the write below).
Per city: request the SERP up to 3 times with growing sleeps (Google rate-limits); a page shorter than 20000 bytes is a consent/bot page and fails the attempt.
Parse name candidates (_parse_google_neighborhoods); require 5..1000 rows (the error message says 5-150 — a known stale string) else fail this city.
Guard: a fresh parse less than half the city's previous google-maps rows fails unless force=True. Failures record the error and keep the city's old rows.
Union rows with source="google-maps" (identical (name, city) entries collapse); sleep a random 2-6s between cities.
Write via a .tmp file and atomic replace; clear the locality cache.

Parameters

cities — Optional list of city names; defaults to _BIG_CITIES.
force — Accept a fresh parse even when it is much smaller than the previous one.

Returns: dict — {"total", "failed", "updated"} — failed maps each failing city to its error text.

Side effects: Makes Google Search requests, sleeps/retries, rewrites localities.csv, clears the load_locality_table cache.

Raises: Never raises (per-city failures are recorded and skipped).

Example

>>> update_neighborhoods(cities=["Lahore"])["failed"]  # dead pipeline
{'Lahore': 'parsed 0 names, outside the plausible range (5-150) -- junk parse or Google blocked the request'}

Related: Called by the CLI after the other pipeline phases; feeds load_locality_table().

Conflict checks

Checks that detect contradictions inside an address, such as a tehsil that is not in the stated district. Most return a reason string, or None when nothing conflicts.

Name	What it does
`province_conflict`	True only when a known city clearly sits in a different province than stated.
`_city_mismatch`	Two different known cities claimed in one address.
`_district_conflict`	Two different districts explicitly stated in one address.
`_city_district_conflict`	A stated city whose EVERY stated 'District X' contradicts it.
`_tehsil_mismatch`	Find two distinct known tehsils claimed in one address, or one claimed tehsil that contradicts the stated city / stated district.
`_tehsil_city_conflict`	An explicitly MARKED tehsil whose registry districts exclude the stated city's districts.
`_society_conflict`	An OFFICIAL society appears with a city its registered location does not include.
`_sector_conflict`	A known Islamabad sector in the text with a city that is not Islamabad.
`_locality_conflict`	An official, well-known locality in the text with a city it does not belong to ('Hayatabad' is Peshawar, not Islamabad).
`_postcode_conflict`	An explicitly LABELED, KNOWN postcode whose registered city contradicts the text.

`province_conflict`

True only when a known city clearly sits in a different province than stated.

province_conflict(place: str | None, province: str | None) -> bool

How it works

Normalise the province; missing place or province → False (nothing to compare, stay lenient).
Search the hand-curated CITY_TO_PROVINCE list as whole words in the place text; a hit returns whether the stated province is NOT among the allowed set.
Otherwise resolve the place via the admin table (_place_provinces): a name unique to one province conflicts when the stated province differs.
Unknown place → False.

Parameters

place — An extracted place field (city/tehsil/district text), not raw address text.
province — A province string (free text; normalised internally).

Returns: bool — True = known cross-province contradiction; False = consistent, unknown, or missing input.

Side effects: None.

Raises: Never raises.

Example

>>> province_conflict("Karachi", "Punjab")
True
>>> province_conflict("Karachi", "Sindh")
False
>>> province_conflict("Depalpur", "Balochistan")
True

Why it works this way: Works on extracted fields, not raw text, so road names such as "Peshawar Road, Rawalpindi" or "Punjab Colony, Karachi" don't trigger it. The hand-curated CITY_TO_PROVINCE list is the authority; registry-only names resolve through the admin table.

Related: Called by check_address().

`_city_mismatch`

Two different known cities claimed in one address.

_city_mismatch(address: str) -> str | None

How it works

Collect the positioned city hints, in text order; a "gilgit" hint directly followed by "baltistan" is the province phrase, not a city claim, and is skipped.
Fewer than two hints → None.
First vs last hint: identical → None. Different → conflict UNLESS the first hint is also a registered TEHSIL of the last hint's district ("Sahiwal" is a Sargodha tehsil, so "Sahiwal Road, Sargodha" names a tehsil, not a second city). The last mention is the stated city; an earlier different one contradicts it.

Parameters

address — Raw address text.

Returns: Optional[str] — A reason string ("'attock' is not in 'quetta'"), or None when there is no two-city contradiction.

Side effects: None.

Raises: Never raises.

Example

>>> _city_mismatch("Attock, Quetta")
"'attock' is not in 'quetta'"
>>> _city_mismatch("Shop 5, Sahiwal, Sargodha")
None

Related: Called by check_address().

`_district_conflict`

Two different districts explicitly stated in one address.

_district_conflict(address: str) -> str | None

How it works

Canonicalise the text and collect stated districts (_stated_districts).
Drop values that are not real districts — and not real cities either ("District Karachi" counts too).
Keep only the first occurrence of each distinct district norm, in order.
Two or more distinct districts → a conflict, UNLESS the two are fuzzy-similar (ratio >= 88, i.e. repeats or near-miss spellings of the SAME district, which stay silent).

Parameters

address — Raw address text.

Returns: Optional[str] — A reason string naming the two districts, or None when there is no two-district contradiction.

Side effects: None.

Raises: Never raises.

Example

>>> _district_conflict("District Lodhran ... District Lahore")
"'Lodhran' and 'Lahore' are two different districts"
>>> _district_conflict("District Lahore ... District Lahore")
None

Related: Called by check_address().

`_city_district_conflict`

A stated city whose EVERY stated 'District X' contradicts it.

_city_district_conflict(address: str, district: str | None) -> str | None

How it works

Missing district, no stated districts, or no positioned city hints → None.
For each positioned hint with a registry entry (_city_districts): if NO stated district matches any of its districts (exact or fuzzy 88), return the conflict naming the hint.
One matching district statement ('District Lahore' for a Lahore address) or an unknown city silences it.

Parameters

address — Raw address text.
district — The parsed district field (used only for the message).

Returns: Optional[str] — A reason string, or None when there is no city-vs-districts contradiction.

Side effects: None.

Raises: Never raises.

Example

>>> _city_district_conflict("BHAKKAR, District Ludhran", "Ludhran")
"'bhakkar' is not in district 'Ludhran'"

Why it works this way: Only explicit district markers count as evidence — a registry-filled district is not a statement.

Related: Called by check_address().

`_tehsil_mismatch`

Find two distinct known tehsils claimed in one address, or one claimed tehsil that contradicts the stated city / stated district.

_tehsil_mismatch(address: str) -> str | None

How it works

Canonicalise the text and walk its words; commas delimit fields (a follower word in the NEXT field cannot qualify the claim).
Match known tehsil names as whole phrases, longest span first ("Kot Addu", "Dera Ghazi Khan"); alias-resolve each candidate.
Drop a match when: a registered locality name covers the same or a longer span (locality-first), every token is generic/descriptor ("Model Town" names no specific tehsil), the name is itself a district ("Baloch Tehsil, Sudhnoti": Sudhnoti states the district), a road/business/place-type follower word appears within the next 3 same-field words ("Shah Faisal - Korangi Link", "Multan Sohan Halwa"), or the previous word is a district word ("District Jhang" states a district, not a tehsil). The follower checks are typo-tolerant (_looks_like).
Two or more DISTINCT claims → a conflict naming the first and last.
One claim: it still contradicts the stated city when its province (unique registry names only) differs from the last positioned city hint's province ("Jand, Quetta"); same-province pairs ("Jand, Attock") stay valid. It also contradicts an explicitly stated "District X" that excludes all of its registered districts ("Silanwali, District Sialkot").

Parameters

address — Raw address text.

Returns: Optional[str] — A reason string, or None when there is no tehsil contradiction.

Side effects: None.

Raises: Never raises (an empty table yields no claims).

Example

>>> _tehsil_mismatch("Palandri Hajira Poonch")
"'Palandri' is not in 'Hajira' tehsil"
>>> _tehsil_mismatch("Palandri Rd, Hajira, Poonch")
None

Why it works this way: Multi-word tehsils are matched as whole phrases: a single-word scan never recognises "Kot Addu" (one token, "kotaddu"), so real contradictions involving one would silently pass.

Related: Called by check_address(); calls _alias(), _locality_norms(), _place_provinces(), _stated_districts(), _city_hints().

`_tehsil_city_conflict`

An explicitly MARKED tehsil whose registry districts exclude the stated city's districts.

_tehsil_city_conflict(address: str, p) -> str | None

How it works

Look for a "Tehsil X" marker (leading or trailing order); no marker → None (ambient names stay ambiguous).
Missing parsed tehsil, unknown tehsil, no positioned city hints → None.
Exclude city hints INSIDE the marker's own span ("TEHSIL FATEH JHANG" — "jhang" is a typo tail, not a city); the last remaining hint is the stated city.
Conflict when the city has registry districts and none of them overlaps the tehsil's districts, OR the tehsil's province (unique names only) differs from the city's province. Any district overlap silences it (same-district pairs like "Tehsil Taxila, Rawalpindi" stay valid).

Parameters

address — Raw address text.
p — The Parsed object from parse_address().

Returns: Optional[str] — A reason string, or None when there is no marked-tehsil contradiction.

Side effects: None.

Raises: Never raises.

Example

>>> _tehsil_city_conflict("Tehsil Mankera, Rahim Yar Khan", parse_address("Tehsil Mankera, Rahim Yar Khan"))
"'Mankera' is not in 'rahim yar khan'"

Related: Called by check_address().

`_society_conflict`

An OFFICIAL society appears with a city its registered location does not include.

_society_conflict(address: str) -> str | None

How it works

No city hints → None.
Match registered society names in the text (_name_matches); no matches → None.
Scope gate: the check only polices the STRICT big-city set (the scheme lists are curated per-city for the big metros, not for every small town) — EXCEPT mega societies (Bahria Town, DHA), which are authority-approved and impossible outside their registered cities even in small towns.
Conflict when NO matched society has ANY registered location consistent with a city hint (exact membership or fuzzy 88 against each location word). Any consistent hint silences it.

Parameters

address — Raw address text.

Returns: Optional[str] — A reason string naming the first matched society, or None.

Side effects: None.

Raises: Never raises.

Example

>>> _society_conflict("Bahria Town Gujarat DHA 73")
"'bahria town' is not located in the city stated"

Why it works this way: Official societies cannot exist in unregistered cities, so this is impossible; UNOFFICIAL societies (not in the registry, e.g. Ghauri Town) pass everywhere.

Related: Called by check_address().

`_sector_conflict`

A known Islamabad sector in the text with a city that is not Islamabad.

_sector_conflict(address: str) -> str | None

How it works

_SECTOR_RE finds sector codes in either order ("F-8 Markaz" or "Markaz F-8"); the markaz/sector suffix keeps bare letter-number shop codes ("Shop F-8, Karachi") from matching.
Intersect with the registered sector grid; unknown sectors → None.
City hints: none, or only islamabad/rawalpindi → None (Rawalpindi borders Islamabad and people write sectors there). Any other hint conflicts.

Parameters

address — Raw address text.

Returns: Optional[str] — A reason string naming the first known sector, or None.

Side effects: None.

Raises: Never raises.

Example

>>> _sector_conflict("Shop 3, F-8 Markaz, Karachi")
"'F-8' is an Islamabad sector, not in the city stated"
>>> _sector_conflict("Shop 3, F-8 Markaz, Islamabad")
None

Related: Called by check_address().

`_locality_conflict`

An official, well-known locality in the text with a city it does not belong to ('Hayatabad' is Peshawar, not Islamabad).

_locality_conflict(address: str) -> str | None

How it works

No city hints, or no STRICT-city hint → None (the data is scraped — Google/OSM/Wikipedia — and reliably curated only for the true metros).
Collect candidate locality names via the token index; keep those matching the text (_name_matches).
Drop matches that cannot drive a conflict: names made only of generic place-type words ("bazaar"), names containing a marker/landmark-type word ("Civil Hospital Chowk" exists everywhere) — except the explicit _LANDMARK_EXCEPTIONS (Kashmir Highway, Qissa Khwani Bazar, Tariq Road) — and numbered units ("block 7" exists in every city).
A registered SOCIETY that is consistent with the city silences the conflict ("DHA Phase 5, Lahore" vs an Islamabad-only "dha phase 5" row).
Conflict only when EVERY matched locality contradicts every city hint (fuzzy 88 against each registered city); any consistent pair passes.

Parameters

address — Raw address text.

Returns: Optional[str] — A reason string naming the first matched locality and its registered cities, or None.

Side effects: None.

Raises: Never raises.

Example

>>> _locality_conflict("Shop 3, Hayatabad, Islamabad")
"'hayatabad' is in peshawar, not the city stated"

Why it works this way: Unknown names (informal/unofficial areas) pass everywhere, and the strict-city gate keeps a small town's generic locality/market name from being judged by some unrelated big city's scrape (e.g. "Bara Bazaar" registered under Peshawar) — the checker's contract is to reject only clear problems.

Related: Called by check_address().

`_postcode_conflict`

An explicitly LABELED, KNOWN postcode whose registered city contradicts the text.

_postcode_conflict(address: str) -> str | None

How it works

Extract the labeled postcode; no label → None.
Look up its registered city; an UNKNOWN code returns None (positive-only: a stale postcode table must never reject a real address).
Compare the city hints in the text against the registered city (fuzzy bar 88); hints that resemble the registered city silence the check, any other hint conflicts.

Parameters

address — Raw address text.

Returns: Optional[str] — A reason string naming the postcode and its registered city, or None when there is no labeled known postcode or the hints agree.

Side effects: None.

Raises: Never raises.

Example

>>> _postcode_conflict("Postal Code: 54830, Karachi")   # 54830 belongs to Lahore
'postcode 54830 belongs to lahore, not the city stated'
>>> _postcode_conflict("Postal Code: 54830, Lahore")
None
>>> _postcode_conflict("Postal Code: 99999, Karachi")   # unknown code
None

Related: Called by check_address().

Parsing helpers

Internal helpers that read the address text: finding cities, anchors, markers and postcodes.

Name	What it does
`_city_hints`	Find known cities mentioned in the text, exact or near-miss.
`_hint_positions`	Find the text position of each city hint.
`_stated_districts`	Extract all explicitly stated districts as (display, norm) pairs.
`_locality_from_text`	Determine the locality deterministically: the longest registry locality/society name present in the text, else a Title-Case area-name phrase.
`_marker`	Test whether the text carries a physical marker (road/street/near/opp/shop/house number, plus code, postcode or sector code).
`_named_anchor`	Test whether the text names a real place anchor: a locality that traces back to the written text, a tehsil/district, a plus code, a labeled known postcode, a near/opposite landmark reference, or a registry-known locality or society name.
`_town_anchor`	A known tehsil that is not the STRICT city in the text names a town.
`_foreign_country`	Find the first clearly-stated foreign country or foreign CITY at the END of the address.
`_locatable`	True when the text pins a specific point.
`_is_gibberish`	No readable words at all, or every word is a keyboard mash.
`_postcode_anchor`	A clearly mentioned, KNOWN postcode pins a specific postal area.
`_clear_postcode`	Extract a clearly mentioned postcode: labeled, or a bare 5-digit code in the standard trailing position AFTER a known city name ('Karachi 74000').
`_labeled_postcode`	Extract an explicitly LABELED postcode ('Postal Code: 54830').
`_canonicalize`	Map near-miss words to their canonical keyword so marker/district detection tolerates misspells.
`_canon_word`	Map a near-miss word to its canonical keyword.

`_city_hints`

Find known cities mentioned in the text, exact or near-miss.

_city_hints(text: str) -> set

How it works

Canonicalise the text (marker/district keywords typo-corrected) and index the words; commas split the text into fields (a word in the NEXT field cannot qualify the city: "BHAKKAR, District Layyah" still states Bhakkar).
Walk the words; multi-word cities are matched as whole phrases, longest first, so "Rahim Yar Khan" beats a stray "khan". Each candidate is dropped (_excluded) when the next same-field word is a district word ("Wah Cantt, Rawalpindi District" is NOT a Rawalpindi-city claim), a business word within the next 3 same-field words ("Hyderabad Sweets", "London Bakery"), or a road/place-type follower ("Peshawar Road").
A single-word city that heads a known tehsil/district phrase ("Nowshera Virkan") is part of that phrase, not a city claim — unless it is comma-stated, which IS a city statement.
Unmatched words are checked against CITY_ALIASES (deterministic variants), then a fuzzy bar of 88 for near-miss city words (only when >= 4 letters and not excluded).
All found hints are re-mapped through CITY_ALIASES so consumers see canonical names.

Parameters

text — Address text (any case; canonicalised internally).

Returns: set — Canonical lowercase city names found in the text; empty when none.

Side effects: None.

Raises: Never raises.

Example

>>> _city_hints("Hyderabad Sweets, Karachi")
{'karachi'}
>>> _city_hints("Shop 5, Rawalpindi District")
set()   # a district mention is not a city claim

Why it works this way: The exclusion words are matched loosely (_looks_like) because a misspelled follower ("Sweeets", "Distt") must not turn a business or landmark name into a false city claim — a missed follower here cascades into false city/locality/society conflicts downstream.

Related: Called by parse_address() and every city-aware conflict check.

`_hint_positions`

Find the text position of each city hint.

_hint_positions(text: str, hints) -> dict

How it works

Tokenise the lowercased text with spans.
For each hint, search the exact hint word, then any CITY_ALIASES variant word ("sarghodha" for "sargodha").
Hints with no position are omitted — fuzzy hints get no position, because fuzzy matching must never drive a rejection, only resolve real variants.

Parameters

text — Lowercased address text.
hints — Iterable of city hints (from _city_hints).

Returns: dict — {hint: character position}; hints found only fuzzily are absent.

Side effects: None.

Raises: Never raises.

Example

>>> _hint_positions("shop 5, sargodha", {"sargodha"})
{'sargodha': 8}

Related: Called by _city_mismatch(), _city_district_conflict(), _tehsil_city_conflict().

`_stated_districts`

Extract all explicitly stated districts as (display, norm) pairs.

_stated_districts(text: str) -> list

How it works

Run both district-marker regexes ("District X" leading, "X District" trailing) over the canonicalised text.
Skip values naming province/country/state; cut the capture at the first suffix word (tehsil/district/city/...).
Trim trailing words until the remainder (alias-resolved) is a KNOWN district norm — the capture is greedy ("District Rahim Yar Khan"), and words that are not districts ("District Court") drop away entirely.
Unknown values never reach the output.

Parameters

text — Canonicalised address text.

Returns: list of tuple — [(display name, district norm), ...] in text order.

Side effects: None.

Raises: Never raises.

Example

>>> _stated_districts("District Rahim Yar Khan")
[('Rahim Yar Khan', 'rahimyarkhan')]
>>> _stated_districts("District Court Road, Lahore")   # "Court" is not a district
[]

Related: Called by _district_conflict(), _city_district_conflict(), _tehsil_mismatch().

`_locality_from_text`

Determine the locality deterministically: the longest registry locality/society name present in the text, else a Title-Case area-name phrase.

_locality_from_text(address: str, city) -> str | None

How it works

Collect candidate registry locality names via the token index; the LONGEST name that matches the text and contains no road word wins (road words are roads, not areas).
No registry locality: try registered society names (same longest-wins rule); mega societies (DHA, Bahria Town) in a BIG city need a numbered phase/sector/block, like in _named_anchor.
Still nothing: walk Title-Case words; the first 2-3 word phrase (span 3 then 2) that is not a known city, does not end in the stated city, contains no road word, and LEADS with a proper noun (not all-generic "main bazar", not descriptor-led "Main Qambrani") becomes the locality. "Kabul Town" qualifies; "House 5 Kabul" does not. A single universal place name that is a proper noun ("Saddar", "gpo") also qualifies — it names the city's own central area and anchors (acceptance only; it can never drive a rejection). One exception: "Saddar" does NOT anchor in Islamabad (a planned sector city with no Saddar area).

Parameters

address — Raw address text.
city — The parsed city (used to exclude phrases that are the city or end in it, and for the mega-society gate).

Returns: Optional[str] — The Title-Cased locality name, or None.

Side effects: None.

Raises: Never raises.

Example

>>> _locality_from_text("Shop 3, Tibbi Saikhwaan, Jhang", "Jhang")
'Tibbi Saikhwaan'
>>> _locality_from_text("Main Bazar, Rawalakot", None)
None
>>> _locality_from_text("Street 232 GPO Islamabad", "Islamabad")
'Gpo'

Related: Called by parse_address().

`_marker`

Test whether the text carries a physical marker (road/street/near/opp/shop/house number, plus code, postcode or sector code).

_marker(address: str) -> bool

How it works (any hit → True)

A plus code (_PLUS_CODE) in the text.
A clearly mentioned postcode (_clear_postcode) that is in the known-postcodes registry.
_MARKER_RE against the canonicalised text: road/street/lane/link/highway/avenue words, chowk/market/bazar, near/opp/opposite/behind, masjid/school/hospital/station/bank/hotel/restaurant/plaza, shop/house/plot/flat/office/stand numbers, markaz/chak/mouza/mandi.
_SECTOR_RE: a sector code in either order ("F-8 Markaz", "Markaz F-8").
A registered LANDMARK name (_landmark_names) matched in the text — "Mazar-e-Quaid, Karachi" needs no road; area rows like "ghauri town phase 1" are NOT landmarks.
A bare Islamabad sector code that exists in the registry ("F-7 Islamabad").
A named business (_NAMED_BUSINESS: "Karachi Biryani") — it pins its own location.

Parameters

address — Raw address text.

Returns: bool — True when any marker is present; False otherwise.

Side effects: None.

Raises: Never raises.

Example

>>> _marker("Main Road, DHA 3 Lahore")
True
>>> _marker("DHA 3 Lahore")
False

Related: Called by check_address() (the generic_address rule).

`_named_anchor`

Test whether the text names a real place anchor: a locality that traces back to the written text, a tehsil/district, a plus code, a labeled known postcode, a near/opposite landmark reference, or a registry-known locality or society name.

_named_anchor(address: str, parsed) -> bool

How it works (any hit → True)

_field_anchor(parsed.locality), then (parsed.tehsil / parsed.district). A field value anchors when it is non-empty, not a bare city word, not a marker-led phrase, not all-generic ("street 5"), not a road name, not a district (broad area), a tehsil that differs from the stated city ("Nowshera Virkan" inside Gujranwala), or any other value that matches the written text (_name_matches).
A plus code in the address.
A clearly mentioned, KNOWN postcode (bare numbers never do).
A near/opposite/behind/in-front-of reference.
A registered locality name matched in the text (candidates come from the token index; road-word names excluded).
A registered society name matched in the text — except MEGA societies (DHA, Bahria Town) in a BIG city, where a bare mention is too vague and only a numbered phase/sector/block anchors ("DHA 3 Lahore").
A NAMED road ("Peshawar Road", "4-Lyton Road") whose proper noun leads (descriptors like "Main Qambrani" don't count).
A numbered place unit ("Chak No 42", "Mouza 116") or a named business ("Karachi Biryani").

Parameters

address — Raw address text.
parsed — The Parsed object from parse_address().

Returns: bool — True when any named anchor is present; False otherwise.

Side effects: None.

Raises: Never raises.

Example

>>> p = parse_address("Nowshera Virkan, Gujranwala")
>>> _named_anchor("Nowshera Virkan, Gujranwala", p)
True

Related: Called by check_address() (the generic_address rule); calls _name_matches(), _locality_index(), load_society_table().

`_town_anchor`

A known tehsil that is not the STRICT city in the text names a town.

_town_anchor(address: str, p) -> bool

How it works

No parsed tehsil → False.
A universal name ("Saddar") with the stated city Islamabad → False (the planned sector city has no Saddar area).
True when the tehsil is a registered tehsil name AND is not one of the strict-city hints in the text.

Parameters

address — Raw address text.
p — The Parsed object from parse_address().

Returns: bool — True when the parsed tehsil pins a specific town.

Side effects: None.

Raises: Never raises.

Example

>>> _town_anchor("Nowshera Virkan, Gujranwala", parse_address("Nowshera Virkan, Gujranwala"))
True

Why it works this way: A known tehsil inside a stated city names a town, which is specific enough on its own — no separate road/marker is needed.

Related: Called by check_address() (the generic_address rule).

`_foreign_country`

Find the first clearly-stated foreign country or foreign CITY at the END of the address.

_foreign_country(address: str) -> str | None

How it works

Scan the _COUNTRIES list for a name at the very end of the text (optionally followed by digits); return the first hit.
Then scan the _FOREIGN_CITIES list the same way, tolerating a city suffix word ("Kabul City", "Kabul Shehr").
Mid-text mentions are business/road names, not claims, and return None.

Parameters

address — Raw address text.

Returns: Optional[str] — The foreign country/city name, or None when none is stated at the end.

Side effects: None.

Raises: Never raises.

Example

>>> _foreign_country("Shop 3, Lahore, Dubai")
'dubai'
>>> _foreign_country("Kabul Hotel, Shorkot")
None

Why it works this way: Checked only at the end (possibly followed by a code) so a business name like "India Plaza, Lahore" is not flagged.

Related: Called by parse_address().

`_locatable`

True when the text pins a specific point.

_locatable(address: str) -> bool

How it works

A plus code (_PLUS_CODE) in the text, or
_ANCHOR_RE against the canonicalised text: street/road/lane/link/highway/avenue, chowk/market/bazar/colony/sector/phase/block/society/mohalla/town/goth/basti/chak/mouza/village/dhera/dera, near/opposite/behind, masjid/school/hospital/station/bank/hotel/restaurant/plaza, or a shop/house/plot/flat/office/stand number.

Parameters

address — Raw address text.

Returns: bool — True when a street, landmark, village marker or number is present.

Side effects: None.

Raises: Never raises.

Example

>>> _locatable("Street 12, House 12, Karachi")
True
>>> _locatable("Karachi")
False

Related: Called by parse_address().

`_is_gibberish`

No readable words at all, or every word is a keyboard mash.

_is_gibberish(address: str) -> bool

How it works

Collect lowercase words of length >= 2.
True when no word reaches 3 letters, or when all words are in the _MASH set (asdf/qwerty rows, aaa/bbb repeats, test/dummy/sample/lorem/ipsum...).

Parameters

address — Raw address text.

Returns: bool — True when the text carries no readable place vocabulary.

Side effects: None.

Raises: Never raises.

Example

>>> _is_gibberish("asdf qwerty zxcv")
True
>>> _is_gibberish("Shop 5, Karachi")
False

Related: Called by parse_address().

`_postcode_anchor`

A clearly mentioned, KNOWN postcode pins a specific postal area.

_postcode_anchor(address: str) -> bool

How it works

_clear_postcode extracts the labeled or trailing postcode.
True only when it is in the known-postcodes registry; unknown codes do not anchor.

Parameters

address — Raw address text.

Returns: bool — True when a known postcode is clearly mentioned.

Side effects: None.

Raises: Never raises.

Example

>>> _postcode_anchor("Karachi 74000")
True
>>> _postcode_anchor("Karachi 99999")
False

Related: Called by parse_address().

`_clear_postcode`

Extract a clearly mentioned postcode: labeled, or a bare 5-digit code in the standard trailing position AFTER a known city name ('Karachi 74000').

_clear_postcode(address: str) -> str | None

How it works

A labeled postcode (_labeled_postcode) always wins.
Otherwise take the LAST 5-digit group in the text; it counts only when it ends the address and a known city word appears before it.

Parameters

address — Raw address text.

Returns: Optional[str] — The 5-digit code, or None when no clear postcode is present.

Side effects: None.

Raises: Never raises.

Example

>>> _clear_postcode("Postal Code: 54830")
'54830'
>>> _clear_postcode("Karachi 74000")
'74000'
>>> _clear_postcode("Plot 54830, Clifton, Karachi")
None

Why it works this way: Used for ANCHORING only — the bare trailing form never triggers a rejection, because no model can tell a plot number from a postcode.

Related: Called by _marker(), _named_anchor(), _postcode_anchor().

`_labeled_postcode`

Extract an explicitly LABELED postcode ('Postal Code: 54830').

_labeled_postcode(address: str) -> str | None

How it works

Regex for "post code" / "postal code" / "postcode" / "zip(code)" with an optional colon/hash followed by exactly 5 digits.
Return the digits or None.

Parameters

address — Raw address text.

Returns: Optional[str] — The 5-digit code when an explicit label precedes it, else None (bare digits in the text are never "labeled").

Side effects: None.

Raises: Never raises.

Example

>>> _labeled_postcode("Postal Code: 54830, Lahore")
'54830'
>>> _labeled_postcode("Karachi 74000")
None

Why it works this way: This is the ONLY postcode form allowed to trigger a rejection.

Related: Called by _clear_postcode() and _postcode_conflict().

`_canonicalize`

Map near-miss words to their canonical keyword so marker/district detection tolerates misspells.

_canonicalize(text: str) -> str

How it works

Lowercase the text, then replace every [a-z]+ word through _canon_word.
Punctuation and digits are preserved (commas delimit fields for the district/tehsil regexes).

Parameters

text — Address text.

Returns: str — The text with keyword-like words canonicalised.

Side effects: None.

Raises: Never raises.

Example

>>> _canonicalize("Shop 5, Distict Lahore")
'shop 5, district lahore'

Related: Called by parse_address(), _city_hints(), _district_conflict(), _tehsil_mismatch(), _tehsil_city_conflict(), _locatable().

`_canon_word`

Map a near-miss word to its canonical keyword.

_canon_word(w: str) -> str

How it works

Exact _WORD_ALIASES entries win ("dist" → "district", "distt" → "district").
Words shorter than 3 chars, or the connectors "e"/"i", stay unchanged.
Otherwise the best SequenceMatcher match among _KEYWORDS wins when it reaches the bar: 75 for short words (<= 5 chars: "rode" → road, "neer" → near), 85 for longer ones ("distict" → district at 87.5). Words far from every keyword stay unchanged.

Parameters

w — A lowercase word from the address.

Returns: str — The canonical keyword, or the input word unchanged.

Side effects: None.

Raises: Never raises.

Example

>>> _canon_word("distict")
'district'
>>> _canon_word("rode")
'road'
>>> _canon_word("karachi")
'karachi'

Related: Called by _canonicalize().

Matching and registry helpers

Internal helpers for fuzzy name matching and for looking up the cached registries.

Name	What it does
`_name_matches`	Test whether a registry name appears in the text.
`_name_pattern`	Build the flexible phrase matcher for a registry name.
`_looks_like`	Typo-tolerant membership test: exact hit, or a close fuzzy match against a vocabulary.
`_ratio`	Similarity score 0-100 (stdlib SequenceMatcher).
`_alias`	Resolve a normalised tehsil/district spelling variant to its canonical registry norm.
`_index`	Build the normalised tehsil → set of normalised districts index.
`_place_provinces`	Build the normalised tehsil/district name → province map for names unique to one province.
`_universal_norms`	Normalised forms of the universal place names (`_UNIVERSAL_NAMES`: "Saddar", "Main Bazar"...).
`_locality_norms`	Normalise every registered locality name.
`_td_norms`	Normalise all known tehsil + district names.
`_locality_index`	Build the token → registry names index for fast locality lookups.
`_landmark_names`	Collect registry names that are themselves landmarks.
`_place_name_tokens`	Collect every individual word inside a known city, tehsil or district name.
`_claim_followers`	Collect the road/business/place-type words that turn a place name into a phrase, not a claim.
`_city_districts`	Build the city name norm → its districts map.
`_postcode_city`	Build the postcode → registered city map from the official data.
`_clean`	Strip a string field; non-strings and empty strings become None.
`_split`	Split a table text line into its leading label and whether numeric columns follow.
`_strip_div`	Strip a leading AJK division name from a label when more words follow.
`_ok_osm_name`	Reject OSM names containing vowel-less abbreviation chunks ('bahria twon skt').

`_name_matches`

Test whether a registry name appears in the text.

_name_matches(name: str, text: str) -> bool

How it works

Try the flexible phrase pattern (_name_pattern) first — spaces/dashes and "e"/"i" connectors are interchangeable.
Fall back to fuzzy window matching: the name's tokens (connectors dropped) are compared as a whole joined string against every same-length window of the text's words at the 88 bar, so near-miss spellings still match (the bar is strict: "mazaar" vs "mazar" scores 85.7 and does NOT count — the same bar that keeps "no karachi" from matching "north karachi"). Single-token names skip fuzz (too risky).
Additionally compare every consecutive token pair (excluding pairs containing a generic landmark-type word like "museum", which would fragment-match unrelated rows) against every word pair window — extra words around the name ("Qila Bala Hisar" vs "Bala Hissar Fortress") must not break the match.

Parameters

name — Lowercase registry name.
text — Lowercased address text.

Returns: bool — True when the name is present (pattern or fuzzy); False otherwise.

Side effects: None.

Raises: Never raises.

Example

>>> _name_matches("mazar-e-quaid", "main mazar e quaid road, karachi")
True
>>> _name_matches("north karachi", "street 12, house 12, karachi")
False

Related: Called by _locality_from_text(), _marker(), _named_anchor(), _locality_conflict(), _society_conflict().

`_name_pattern`

Build the flexible phrase matcher for a registry name.

_name_pattern(name: str) -> re.Pattern | None

How it works

Split the name on spaces/dashes/commas, dropping connector words "e"/"i".
One token → a plain word-boundary regex. Several tokens → the first is escaped, each following token is prefixed with optional spaces/dashes and an optional "e" connector, so "gulistan-e-johar" matches "gulistan johar" and "gulshan-e-maymar" matches "gulshan maymar".
Patterns are memoised in the module-level _NAME_PATTERNS dict.

Parameters

name — A lowercase registry name.

Returns: Optional[re.Pattern] — The compiled pattern, or None when the name had no usable tokens.

Side effects: Caches compiled patterns in the module-level _NAME_PATTERNS dict.

Raises: Never raises.

Example

>>> bool(_name_pattern("gulistan-e-johar").search("main gulistan johar road"))
True

Related: Called by _name_matches().

`_looks_like`

Typo-tolerant membership test: exact hit, or a close fuzzy match against a vocabulary.

_looks_like(word: str, vocab, min_len: int = 3, threshold: float = 88) -> bool

How it works

Empty word → False.
Exact membership → True regardless of length (so short followers "rd"/"st" count).
Words shorter than min_len → False.
Words that are themselves place-name vocabulary (_place_name_tokens) → False (guarded by name, see below).
Otherwise compare only vocab words whose length differs by at most 2, and return True when the best SequenceMatcher score reaches threshold.

Parameters

word — A single lowercase word from the address.
vocab — A set/frozenset of canonical vocabulary words.
min_len — Minimum word length for the fuzzy branch (default 3).
threshold — Fuzzy acceptance bar (default 88).

Returns: bool — True when the word is (or closely resembles) a member of vocab.

Side effects: None.

Raises: Never raises.

Example

>>> _looks_like("sweeets", _BUSINESS_WORDS)
True
>>> _looks_like("main", _BUSINESS_WORDS)
False
>>> _looks_like("rd", _ROAD_WORDS)
True

Why it works this way: _canon_word's looser short-word bar (75) is calibrated for _KEYWORDS, a small set of specific, mutually-dissimilar admin/marker words. The business vocabulary is 20x larger with short generic entries ("mini", "co", "point"), where the loose bar wrongly swallowed ordinary words like "main" against "mini" — so this keeps a stricter threshold plus a length-difference guard while the place-name-token guard preserves genuine typos ("Sweeets" → "Sweets").

Related: Called by _city_hints(), _tehsil_mismatch(), parse_address().

`_ratio`

Similarity score 0-100 (stdlib SequenceMatcher).

_ratio(a: str, b: str) -> float

How it works

Compute SequenceMatcher(None, a, b).ratio() * 100.

Parameters

a, b — The two strings to compare.

Returns: float — 0 (no similarity) to 100 (identical).

Side effects: None.

Raises: Never raises.

Example

>>> _ratio("karachi", "karachi")
100.0
>>> round(_ratio("nokarachi", "northkarachi"), 2)
85.71

Why it works this way: stdlib-only and deterministic on purpose: the checker's decisions must be the same in every environment.

Related: Called by _looks_like, _canon_word, tehsil_district_conflict, _name_matches and the conflict checks.

`_alias`

Resolve a normalised tehsil/district spelling variant to its canonical registry norm.

_alias(n: str) -> str

How it works

Look up the normalised name in the curated _TD_ALIASES map ("silianwali" → "sillanwali"); unknown names pass through unchanged.

Parameters

n — A normalised (norm()) tehsil/district name.

Returns: str — The canonical registry norm, or the input itself.

Side effects: None.

Raises: Never raises.

Example

>>> _alias("sudhanoti")
'sudhnoti'
>>> _alias("hajira")
'hajira'

Why it works this way: Variants resolve through this map BEFORE any fuzzy fallback, so alternations like "Silanwali" (one L) are deterministic; the fuzzy 88-bar remains only as a safety net for unknown variants.

Related: Called by tehsil_district_conflict(), _stated_districts(), parse_address(), _tehsil_mismatch(), _tehsil_city_conflict().

`_index`

Build the normalised tehsil → set of normalised districts index.

_index() -> dict

How it works

Walk load_admin_table(); for each tehsil row, add the district norm to the tehsil norm's set.

Returns: dict — {tehsil_norm: {district_norm, ...}}.

Side effects: None (the lru_cache holds the result until cleared).

Raises: Never raises (an empty table yields {}).

Example

>>> "jhang" in _index().get("shorkot", set())
True

Related: Called by tehsil_district_conflict(), _tehsil_mismatch(), _tehsil_city_conflict().

`_place_provinces`

Build the normalised tehsil/district name → province map for names unique to one province.

_place_provinces() -> dict

How it works

Walk load_admin_table(); collect every province each district/tehsil norm appears in.
Keep only names with exactly ONE province; ambiguous names ("khanpur", "saddar", "landhi") are left out and can never drive a conflict.

Returns: dict — {name_norm: province} for unambiguous names only.

Side effects: None (the lru_cache holds the result until cleared).

Raises: Never raises (an empty table yields {}).

Example

>>> _place_provinces().get("depalpur")
'Punjab'
>>> _place_provinces().get("saddar")   # shared by several provinces
None

Related: Called by province_conflict(), _tehsil_mismatch(), _tehsil_city_conflict().

`_universal_norms`

Normalised forms of the universal place names (_UNIVERSAL_NAMES: "Saddar", "Main Bazar"...). A universal name exists in virtually every city, so it can NEVER be claimed as a specific tehsil and can never drive a conflict — ambiguity never causes rejection.

_universal_norms() -> frozenset

`_locality_norms`

Normalise every registered locality name.

_locality_norms() -> frozenset

How it works

norm() each name in load_locality_table() into a frozenset.

Returns: frozenset — Normalised locality names.

Side effects: None (the lru_cache holds the result until cleared).

Raises: Never raises.

Example

>>> norm("gulistan e johar") in _locality_norms()
True

Why it works this way: A name that IS a locality is treated as a locality (user rule: locality-first), never claimed as a tehsil.

Related: Called by _tehsil_mismatch() and parse_address().

`_td_norms`

Normalise all known tehsil + district names.

_td_norms() -> frozenset

How it works

norm() every tehsil and district name from load_admin_table() into one frozenset.

Returns: frozenset — Normalised tehsil and district names.

Side effects: None (the lru_cache holds the result until cleared).

Raises: Never raises.

Example

>>> norm("kot addu") in _td_norms()
True

Related: Called by _city_hints() (the city-phrase guard).

`_locality_index`

Build the token → registry names index for fast locality lookups.

_locality_index() -> dict

How it works

For each locality name, index every token of length >= 4 (connectors "e"/"i" excluded) into a set keyed by that token.

Returns: dict — {token: {locality names containing it}}.

Side effects: None (the lru_cache holds the result until cleared).

Raises: Never raises.

Example

>>> "hayatabad" in _locality_index()
True

Why it works this way: Matching checks only candidates instead of scanning every one of the thousands of registry rows (keeps the check fast).

Related: Called by _locality_from_text(), _named_anchor(), _locality_conflict().

`_landmark_names`

Collect registry names that are themselves landmarks.

_landmark_names() -> tuple

How it works

Read localities.csv directly (bypassing overrides); keep rows whose source is "wikipedia"/"wikipedia-world", plus any row whose name contains a landmark word (_LANDMARK_WORDS: statue, darbar, shrine, masjid, fort, ...).
Return the lowercased names as a tuple.

Returns: tuple — Lowercase landmark names (Wikipedia-sourced or landmark-word names). Plain area/town names are NOT included.

Side effects: None (the lru_cache holds the result until cleared).

Raises: Never raises (a missing CSV yields ()).

Example

>>> "mazar-e-quaid" in _landmark_names()
True
>>> "ghauri town phase 1" in _landmark_names()
False

Related: Called by _marker().

`_place_name_tokens`

Collect every individual word inside a known city, tehsil or district name.

_place_name_tokens() -> frozenset

How it works

Split every CITY_TO_PROVINCE city name into words.
Split every district and tehsil name from load_admin_table() into words.
Return the frozen set.

Returns: frozenset — Place-name vocabulary words ('khan' from 'Dera Ghazi Khan', 'addu' from 'Kot Addu').

Side effects: None (the lru_cache holds the result until cleared).

Raises: Never raises.

Example

>>> "addu" in _place_name_tokens()
True

Why it works this way: A word that is itself genuine Pakistani place-name vocabulary must never be swallowed as a fuzzy near-miss of an unrelated business/follower word: 'khan' resembles 'khana' (a listed business word meaning eatery) closely enough that no single similarity threshold can separate that collision from a genuine typo like 'resturent' → 'restaurant', so it is guarded by name here instead.

Related: Called by _looks_like().

`_claim_followers`

Collect the road/business/place-type words that turn a place name into a phrase, not a claim.

_claim_followers() -> frozenset

How it works

Union _ROAD_WORDS, _BUSINESS_WORDS and _PLACE_FOLLOWERS into one frozenset.

Returns: frozenset — The follower vocabulary ("Palandri Rd", "Palandri Hotel", "Palandri Sweets" are not tehsil claims).

Side effects: None (the lru_cache holds the result until cleared).

Raises: Never raises.

Example

>>> "rd" in _claim_followers()
True

Related: Called by parse_address() (the registry walk) and _tehsil_mismatch().

`_city_districts`

Build the city name norm → its districts map.

_city_districts() -> dict

How it works

Walk load_admin_table(); every district norm maps to itself, and every tehsil norm maps to its district norm(s).
Cities split into several districts (Karachi) end up with a multi-value set, which the callers treat as "no entry" for their purposes.

Returns: dict — {name_norm: {district_norm, ...}}.

Side effects: None (the lru_cache holds the result until cleared).

Raises: Never raises.

Example

>>> _city_districts()["shorkot"]
{'jhang'}

Related: Called by _city_district_conflict() and _tehsil_city_conflict().

`_postcode_city`

Build the postcode → registered city map from the official data.

_postcode_city() -> dict

How it works

Read postcodes.csv; keep rows that have a city; key by postcode.

Returns: dict — {postcode: city}, lowercase cities.

Side effects: None (the lru_cache holds the result until cleared).

Raises: Never raises (a missing CSV reads as empty).

Example

>>> _postcode_city().get("54830")
'lahore'

Related: Called by _postcode_conflict().

`_clean`

Strip a string field; non-strings and empty strings become None.

_clean(value) -> str | None

How it works

Strings are stripped; an all-whitespace string becomes None.
Non-strings become None.

Parameters

value — Any extracted field value.

Returns: Optional[str] — The stripped string, or None.

Side effects: None.

Raises: Never raises.

Example

>>> _clean("  Lahore  ")
'Lahore'
>>> _clean("   ")
None

Related: Currently unused.

`_split`

Split a table text line into its leading label and whether numeric columns follow.

_split(line: str)

How it works

Split on whitespace and scan tokens left to right.
The first token matching _NUM (a number, optionally with commas/decimals, or a bare dash) ends the label; everything before it is the label text.
A line with no numeric token returns the whole line with found=False.

Parameters

line — One raw text line from a census table.

Returns: tuple — (label, has_num) — label is the text before the first numeric column ('' when the line starts with a number), has_num is True when a numeric token was found.

Side effects: None.

Raises: Never raises (pure function).

Example

>>> _split("ATTOCK DISTRICT 123 456")
('ATTOCK DISTRICT', True)
>>> _split("some header words")
('some header words', False)

Related: Called by parse_pbs_lines(), _parse_ajk_yearbook(), _parse_gb_glance().

`_strip_div`

Strip a leading AJK division name from a label when more words follow.

_strip_div(label: str) -> str

How it works

Split into words; if there is more than one word and the first is a division name (muzaffarabad / poonch / mirpur), drop it.
A single-word label is returned unchanged (a bare "Mirpur" IS the name).

Parameters

label — A label from the AJK year book.

Returns: str — The label without its leading division word ('Mirpur Mirpur' → 'Mirpur', 'Muzaffarabad Sharda' → 'Sharda'), or the input unchanged.

Side effects: None.

Raises: Never raises.

Example

>>> _strip_div("Muzaffarabad Sharda")
'Sharda'
>>> _strip_div("Mirpur")
'Mirpur'

Related: Called by _parse_ajk_yearbook().

`_ok_osm_name`

Reject OSM names containing vowel-less abbreviation chunks ('bahria twon skt').

_ok_osm_name(name: str) -> bool

How it works

Split the name into tokens; any token that is alphabetic, at most 4 chars and contains no vowel (a e i o u) fails the name.
All other names pass.

Parameters

name — A raw OpenStreetMap place name.

Returns: bool — True when the name has no vowel-less <=4-letter alphabetic token.

Side effects: None.

Raises: Never raises.

Example

>>> _ok_osm_name("Bahria Town")
True
>>> _ok_osm_name("Bahria Town Skt")
False

Why it works this way: Abbreviation chunks are junk that must not enter the registry.

Related: Called by _osm_neighborhoods().

Download and parse helpers

Internal helpers that download and parse the source documents (PDFs, HTML pages, web APIs) behind the update pipelines.

Name	What it does
`fetch`	Download a URL into the local cache and return its path; local paths pass through.
`fetch_first`	Try each candidate source in order and return the first that downloads.
`pdf_lines`	Yield the text lines of a PDF file, page by page.
`parse_pbs_lines`	Walk PBS Census Table 1 text and emit (province, district, tehsil) rows.
`_parse_ajk_yearbook`	Parse the AJK Statistical Year Book tables 7.10 (districts) and 7.11 (tehsils).
`_parse_gb_glance`	Parse the GB "at a Glance" district table (district-level rows only).
`_parse_admin_lines`	Route each province's source document to its parser.
`_parse_society_rows`	Extract (name, location) society rows from an HTML table page (PHATA style).
`_parse_cda_rows`	Extract scheme names from CDA card links (`/housing-schemes/<slug>`).
`_parse_pdf_rows`	Read (name, location) rows from a generic authority PDF table (LDA/RDA style).
`_parse_society_source`	Parse one approved-scheme source file into (name, location) rows by its kind.
`_fetch_postcode_rows`	Fetch ALL result rows for one postcode API query, paging past the 200-row cap.
`_wiki_landmarks`	Collect a city's landmark names from Wikipedia's category system.
`_osm_neighborhoods`	Collect a city's neighborhood + landmark names from OpenStreetMap.
`_overpass`	Run an Overpass query against the public mirrors with retries.
`_parse_google_neighborhoods`	Pull neighborhood/town name candidates out of the Google results text.
`_google_query`	Build the Google Search URL for a city's neighbourhood query (dead pipeline).
`_meta`	Read the pipeline metadata file.
`_read_csv`	Read a CSV file as a list of dicts; a missing file reads as [].
`_main`	CLI entry point: `update` (rebuild data).
`_TableParser`	Collect the first two text cells of every `<tr>` as (name, location) pairs.
`_SearchText`	Collect the plain text of a page with `<script>`/`<style>` blocks dropped.

`fetch`

Download a URL into the local cache and return its path; local paths pass through.

fetch(source: str) -> pathlib.Path

How it works

Non-http(s) sources are returned as-is as a Path (never modified).
The cache directory is created; the destination name is the first 12 hex chars of the URL's sha1 plus the URL's file suffix (defaults to ".pdf").
A previously stored ETag (from meta.json) is sent as If-None-Match when the cached file exists.
On success the body is written and the new ETag + today's date are recorded in meta.json.
An HTTP 304 (unchanged) is swallowed: the cached copy is kept and returned.

Parameters

source — URL to download, or a local path.

Returns: Path — The cached file path (the local path itself when source is not a URL).

Side effects: Writes the downloaded body into the cache directory; updates meta.json.

Raises: urllib.error.HTTPError for non-304 status codes; urllib.error.URLError or OSError for network/IO failures.

Example

>>> fetch("https://www.pbs.gov.pk/wp-content/uploads/census_tables/tables/table_1_punjab_districts.pdf")
WindowsPath('.../pk_admin_data/cache/a1b2c3d4e5f6.pdf')
>>> fetch("C:/data/table.pdf")
WindowsPath('C:/data/table.pdf')

Related: Called by fetch_first(); calls _meta().

`fetch_first`

Try each candidate source in order and return the first that downloads.

fetch_first(candidates) -> pathlib.Path

How it works

For each candidate, call fetch(); catch OSError and URLError (HTTPError is a URLError subclass) and remember the error text.
Return the first successful download's path.
If every candidate failed, raise one RuntimeError listing all of them.

Parameters

candidates — Iterable of URLs or local paths, tried left to right.

Returns: Path — The first successfully fetched file.

Side effects: None directly (fetch() writes the cache).

Raises: RuntimeError when every candidate fails (the message lists each source and its error).

Example

>>> fetch_first(["https://mirror1.example/x.pdf", "https://mirror2.example/x.pdf"])
WindowsPath('.../pk_admin_data/cache/ab12cd34ef56.pdf')

Related: Called by update_admin_table().

`pdf_lines`

Yield the text lines of a PDF file, page by page.

pdf_lines(path)

How it works

Import pdfplumber on first call (installed automatically, but needed only by the update pipelines, so the import is deferred).
For each page, extract_text() returns the page text ('' for image-only pages), which is split into lines and yielded one at a time.

Parameters

path — Path to a PDF file (a cached download or a local file).

Returns: generator of str — One line of extracted text per yield, in page order.

Side effects: None (reads the given file only).

Raises: pdfplumber's own errors for unreadable files (propagated; the caller decides how to handle them).

Example

>>> list(pdf_lines(Path("table_1_punjab_districts.pdf")))[:2]
['TABLE 1', 'ATTOCK DISTRICT']

Related: Called by update_admin_table() via _parse_admin_lines().

`parse_pbs_lines`

Walk PBS Census Table 1 text and emit (province, district, tehsil) rows.

parse_pbs_lines(lines, province: str) -> list

How it works

Split each line into a label and its numeric columns (_split). Lines without numbers are held back as a pending continuation IF they are letters-only (_LETTERS_ONLY) and not all header words (_HEADER_WORDS) — long names wrap onto two lines with the number columns on the second.
A numbers-only line (empty label) clears the pending fragment.
A pending fragment is prepended to the next line's label.
DISTRICT_RE matches labels ending in "district"/"dist." and starts a district row (tehsil=''); TEHSIL_RE matches labels ending in "tehsil"/"tahsil"/"taluka"/"taluk"/"sub-division"/"town" and attaches the label to the current district. Lines before any district is known are skipped.

Parameters

lines — Iterable of table text lines.
province — Province name stamped on every row.

Returns: list of dict — [{'province', 'district', 'tehsil'}, ...] in reading order; district rows carry tehsil == ''.

Side effects: None.

Raises: Never raises (unrecognised lines are skipped).

Example

>>> parse_pbs_lines(["ATTOCK DISTRICT 1 2", "HASSAN ABDAL TEHSIL 3 4"], "Punjab")
[{'province': 'Punjab', 'district': 'Attock', 'tehsil': ''},
 {'province': 'Punjab', 'district': 'Attock', 'tehsil': 'Hassan Abdal'}]

Related: Called by _parse_admin_lines() for Punjab, Sindh, Khyber Pakhtunkhwa, Balochistan and Islamabad Capital Territory.

`_parse_ajk_yearbook`

Parse the AJK Statistical Year Book tables 7.10 (districts) and 7.11 (tehsils).

_parse_ajk_yearbook(lines, province: str) -> list

How it works

Join the lines; regex the block between "Table 7.10" and "Table 7.11": every label+numeric line becomes a district name (deduped, division prefixes stripped, _SKIP_LABELS filtered). No 7.10 block → RuntimeError.
Regex the 7.11 block after the "Sub Division (Tehsil)" header up to "Source:".
Walk the 7.11 lines in reading order:
- a. A label line WITHOUT numbers that repeats a district name re-attaches the previous tehsil row to that district (the yearbook prints some district labels AFTER their first tehsil row — the Poonch/Sudhnoti late-row quirks).
- b. A numbered line naming a district starts a new district block (also handles district+tehsil merged on one line, "Bagh Dhirkot").
- c. Any other numbered label becomes a tehsil row.
Attach each tehsil to the NEAREST district row above or below it (a re-attach marker wins); emit one row per district name (tehsil='') plus one per tehsil.

Parameters

lines — Extracted text lines of the yearbook PDF.
province — Always "Azad Kashmir" here.

Returns: list of dict — [{'province','district','tehsil'}, ...].

Side effects: None.

Raises: RuntimeError when the 7.10 district block or the 7.11 tehsil table cannot be found (layout changed).

Example

>>> rows = _parse_ajk_yearbook(lines, "Azad Kashmir")
>>> ("Azad Kashmir", "Poonch", "Hajira") in {(r["province"], r["district"], r["tehsil"]) for r in rows}
True

Related: Called by _parse_admin_lines() for "Azad Kashmir".

`_parse_gb_glance`

Parse the GB "at a Glance" district table (district-level rows only).

_parse_gb_glance(lines, province: str) -> list

How it works

Skip lines until the table header ("District Wise Population..." /"Population and Area of GB") appears.
Stop at a "Source:" line or a "GB " prefix (the footer).
Each remaining label+numeric line becomes a district row; labels in _SKIP_LABELS or shorter than 4 chars are dropped. The official document has tehsil COUNTS, not names, so no tehsil rows are produced.
The row count must be 5..15 (GB has 10 districts) or RuntimeError — a wrong count means the layout drifted.

Parameters

lines — Extracted text lines of the GB P&D document.
province — Always "Gilgit-Baltistan" here.

Returns: list of dict — [{'province','district','tehsil':''}, ...].

Side effects: None.

Raises: RuntimeError when fewer than 5 or more than 15 district rows parse.

Example

>>> rows = _parse_gb_glance(lines, "Gilgit-Baltistan")
>>> rows[0]
{'province': 'Gilgit-Baltistan', 'district': 'Gilgit', 'tehsil': ''}

Related: Called by _parse_admin_lines() for "Gilgit-Baltistan".

`_parse_admin_lines`

Route each province's source document to its parser.

_parse_admin_lines(lines, province: str) -> list

How it works

"Azad Kashmir" → _parse_ajk_yearbook and "Gilgit-Baltistan" → _parse_gb_glance (PBS publishes no tehsil tables for the two territories).
Every other province → parse_pbs_lines.

Parameters

lines — Extracted text lines of the province's source PDF.
province — Province name (decides the parser).

Returns: list of dict — The chosen parser's output, [{'province','district','tehsil'}, ...].

Side effects: None.

Raises: The chosen parser's errors (RuntimeError on layout drift).

Example

>>> _parse_admin_lines(["ATTOCK DISTRICT 1 2"], "Punjab")[0]
{'province': 'Punjab', 'district': 'Attock', 'tehsil': ''}

Related: Called by update_admin_table().

`_parse_society_rows`

Extract (name, location) society rows from an HTML table page (PHATA style).

_parse_society_rows(html_text: str) -> list

How it works

Feed the HTML to _TableParser.
Keep rows with at least 5 cells: cell index 1 is the name (the generic suffix regex _GENERIC_SUFFIX strips a trailing "housing scheme/society/homes/town/city") and cell index 2 is the location.
Require: name length >= 4 AND a space inside it AND not "name of", plus at least two letters in the location. The multi-word rule stops a single word like "akbar" from false-matching road names later.

Parameters

html_text — The full HTML of an authority page.

Returns: list of dict — [{"name": ..., "location": ...}], lowercase.

Side effects: None.

Raises: Never raises (malformed HTML yields fewer rows).

Example

>>> _parse_society_rows(open("phata.html", encoding="utf-8").read())[0]
{'name': 'al noor orchard housing scheme', 'location': 'lahore'}

Related: Called by _parse_society_source() for kind="table".

`_parse_cda_rows`

Extract scheme names from CDA card links (/housing-schemes/<slug>).

_parse_cda_rows(html_text: str, city: str) -> list

How it works

Find every href whose path is housing-schemes/<slug>.
Turn the slug's dashes into spaces; skip duplicates, names shorter than 8 chars, and names without 3 consecutive letters.
Every row gets the authority's city as location.

Parameters

html_text — Full HTML of the CDA page.
city — The authority city stamped on each row ("islamabad").

Returns: list of dict — [{"name": ..., "location": city}].

Side effects: None.

Raises: Never raises.

Example

>>> _parse_cda_rows(html, "islamabad")[0]
{'name': 'capital enclave', 'location': 'islamabad'}

Why it works this way: The slug is used because the visible anchor text is CSS-truncated and unreliable.

Related: Called by _parse_society_source() for kind="cards".

`_parse_pdf_rows`

Read (name, location) rows from a generic authority PDF table (LDA/RDA style).

_parse_pdf_rows(path: pathlib.Path, city: str) -> list

How it works

Open with pdfplumber; for every page and every extracted table, keep rows with at least two non-empty cells.
Cell index 1 is the name: whitespace-collapsed, must be at least 8 chars and contain 3 consecutive letters.
Every row gets the authority's city as location (the whole list belongs to one authority).

Parameters

path — Path to the PDF.
city — The authority city stamped on each row.

Returns: list of dict — [{"name": ..., "location": city}].

Side effects: None.

Raises: pdfplumber's own errors for unreadable PDFs (propagated to the caller).

Example

>>> _parse_pdf_rows(Path("lda.pdf"), "lahore")[0]
{'name': 'eden valley housing scheme', 'location': 'lahore'}

Related: Called by _parse_society_source() for kind="pdf".

`_parse_society_source`

Parse one approved-scheme source file into (name, location) rows by its kind.

_parse_society_source(path: pathlib.Path, kind: str, city: str | None) -> list

How it works

kind "table" → _parse_society_rows on the HTML file text.
kind "cards" → _parse_cda_rows with the default city.
kind "pdf" → _parse_pdf_rows.
Any other kind raises ValueError.

Parameters

path — Downloaded source file.
kind — "table", "cards" or "pdf".
city — Default authority city for kinds that do not carry one; may be None.

Returns: list of dict — [{"name", "location"}, ...].

Side effects: None.

Raises: ValueError for an unknown kind.

Example

>>> _parse_society_source(Path("cda.html"), "cards", "islamabad")[0]
{'name': 'capital enclave', 'location': 'islamabad'}

Related: Called by update_society_table().

`_fetch_postcode_rows`

Fetch ALL result rows for one postcode API query, paging past the 200-row cap.

_fetch_postcode_rows(query: str) -> list

How it works

Request /postal-codes?<query>&limit=200&page=N, retrying up to 4 times with 4s sleeps (the API times out under load).
Append the batch and stop when a page returns fewer than 200 rows (the last page).

Parameters

query — URL-encoded query string, e.g. "province=Punjab" or "city=Karachi".

Returns: list of dict — All result rows for the query (each with postal_code, city, province, area_name fields from the API).

Side effects: HTTP GET requests to the postcode API; sleeps 4s per failed attempt.

Raises: The last attempt's exception (URLError/HTTPError/JSONDecodeError) after 4 failures.

Example

>>> len(_fetch_postcode_rows("city=Karachi")) >= 20
True

Related: Called by update_postcodes().

`_wiki_landmarks`

Collect a city's landmark names from Wikipedia's category system.

_wiki_landmarks(city: str) -> list

How it works

For six categories ("Landmarks in X", "Tourist attractions in X", "Shrines in X", "Forts in X", "Monuments and memorials in X", "Statues in X") query the MediaWiki API (categorymembers + redirects, 500 each).
A failed request just skips the category (a missing category is normal).
Each page title plus its redirect titles is cleaned: namespaced titles (containing ":"), "(City)" parentheticals, a trailing ", City", and "List of ..." / "... in City" titles are dropped (list pages are not places).
Titles 4..40 chars with a letter are kept, title-cased.

Parameters

city — City name (substituted into the category titles).

Returns: list of dict — [{"name", "city"}, ...] lowercase names, sorted.

Side effects: Up to 6 MediaWiki API requests per city.

Raises: Never raises (failed categories are skipped).

Example

>>> [r["name"] for r in _wiki_landmarks("Lahore")][:2]
['badshahi mosque', 'lahore fort']

Related: Called by update_neighborhoods_wiki() and update_world_landmarks().

`_osm_neighborhoods`

Collect a city's neighborhood + landmark names from OpenStreetMap.

_osm_neighborhoods(city: str) -> list

How it works

Nominatim search resolves the city to a bounding box (country restricted to Pakistan); no result → RuntimeError.
An Overpass query returns every node/way/relation tagged place=neighbourhood|suburb|quarter inside the bbox.
Names are kept when they have a letter, are 4..40 chars, and pass _ok_osm_name.
A SECOND, nodes-only Overpass query collects landmarks (historic=*, tourism=attraction|museum|..., leisure=park|garden, shrines/darbars); landmark names must be multi-word so bare category words like "Park" don't enter the registry. This query is best-effort: any failure is swallowed (landmarks are a bonus, they never fail the city).

Parameters

city — City name (also used in the Nominatim query "<city>, Pakistan").

Returns: list of dict — [{"name", "city"}, ...] lowercase names, sorted.

Side effects: HTTP requests to Nominatim and the Overpass mirrors.

Raises: RuntimeError when Nominatim finds no city; _overpass errors propagate for the neighborhood query (landmark failures are caught).

Example

>>> [r["name"] for r in _osm_neighborhoods("Karachi")][:2]
['azizabad', 'bahadurabad']

Why it works this way: OpenStreetMap is free, keyless and bot-tolerant — unlike Google's front pages — so the neighborhood phase uses it as the primary source.

Related: Called by update_neighborhoods_osm(); calls _overpass() and _ok_osm_name().

`_overpass`

Run an Overpass query against the public mirrors with retries.

_overpass(query: str) -> dict

How it works

Try overpass-api.de first, then overpass.kumi.systems (the fallback mirror).
Each endpoint gets 3 attempts with 8s sleeps between; a successful response is returned immediately.
When both endpoints are exhausted, re-raise the last captured error (or a generic RuntimeError if none was recorded).

Parameters

query — Overpass QL query text (POSTed as form data).

Returns: dict — Parsed JSON response ({"elements": [...]} for successful queries).

Side effects: HTTP POSTs to the Overpass mirrors; sleeps 8s per failed attempt.

Raises: The last captured exception (URLError, HTTPError, JSONDecodeError...) when every attempt fails.

Example

>>> _overpass('[out:json];node(1);out;')["elements"]
[]

Related: Called by _osm_neighborhoods().

`_parse_google_neighborhoods`

Pull neighborhood/town name candidates out of the Google results text.

_parse_google_neighborhoods(html_text: str, city: str) -> list

How it works

Feed the HTML to _SearchText and join the chunks.
Anchor on Google's echo of the query: the first occurrence of "neighbourhoods in <city>" / "neighborhoods in <city>" / "areas of <city>" / "areas in <city>"; only the next 4000 chars are searched (no anchor → the whole text).
_NEIGHBOR_RE harvests title-cased proper-noun phrases (a capital-led word plus up to 3 more) from that segment — area names are proper nouns; lowercase boilerplate falls away.
Filter: length 5..35, not seen, not the city itself, not a _GOOGLE_STOP word (chrome/boilerplate vocabulary), not containing google/cookies/copyright/wikipedia/http, and either 2+ capitalised words or length >= 8.

Parameters

html_text — The full Google results page HTML.
city — The city being queried (echo anchor + self-exclusion).

Returns: list of dict — [{"name", "city"}, ...] lowercase.

Side effects: None.

Raises: Never raises (malformed HTML yields fewer rows).

Example

>>> _parse_google_neighborhoods(html, "Lahore")[:2]
[{'name': 'gulberg', 'city': 'lahore'}, ...]

Why it works this way: Tolerant on purpose: Google's markup changes constantly, and the per-city plausibility guard in update_neighborhoods rejects junk parses.

Related: Called by update_neighborhoods().

`_google_query`

Build the Google Search URL for a city's neighbourhood query (dead pipeline).

_google_query(city: str) -> str

How it works

URL-encode the city into a "neighbourhoods in <city> pakistan" Google Search query.

Parameters

city — City name.

Returns: str — The full Google Search URL.

Side effects: None.

Raises: Never raises.

Example

>>> _google_query("Lahore")
'https://www.google.com/search?q=neighbourhoods+in+Lahore+pakistan&hl=en&num=20'

Related: Called by update_neighborhoods().

`_meta`

Read the pipeline metadata file.

_meta() -> dict

How it works

If meta.json exists next to the data tables, parse and return it.
Otherwise return {} (no file, no error — a fresh pipeline starts empty).

Returns: dict — The parsed metadata (download ETags, last-run dates, row counts), or {} when meta.json is missing.

Side effects: None.

Raises: json.JSONDecodeError when meta.json exists but is corrupt.

Example

>>> _meta()   # after an update run
{'https://...table_1_punjab_districts.pdf': {'etag': '"abc"', 'fetched': '2026-10-03'}}

Related: Called by fetch() to read and record ETags.

`_read_csv`

Read a CSV file as a list of dicts; a missing file reads as [].

_read_csv(path: pathlib.Path) -> list

How it works

If the file does not exist, return [] immediately — absent registry data must never crash the checks.
Otherwise read with csv.DictReader (utf-8; the header row becomes the keys).

Parameters

path — CSV file path.

Returns: list of dict — One dict per data row (keys = header names, values = strings), or [] when the file is missing.

Side effects: None.

Raises: Never raises for a missing file; UnicodeDecodeError on non-utf-8 content.

Example

>>> _read_csv(TABLE_CSV)[0]
{'province': 'Azad Kashmir', 'district': 'Bagh', 'tehsil': ''}

Why it works this way: Lenient on missing files on purpose: the checks must never fail because a registry file is absent.

Related: Called by every load_* and update_* function.

`_main`

CLI entry point: update (rebuild data).

_main(argv=None)

How it works

argparse defines one subcommand, update [--force].
update runs, in order: update_admin_table, update_society_table, update_neighborhoods_osm, update_neighborhoods_wiki, update_postcodes, update_neighborhoods (Google, currently dead); each prints totals and per-province/city failures.

Parameters

argv — Optional argument list; defaults to sys.argv.

Returns: None.

Side effects: The update subcommand downloads from the internet and rewrites the data CSVs.

Raises: argparse exits on bad arguments.

Related: Run as python -m pk_address_checker ... under __main__.

`_TableParser`

Collect the first two text cells of every <tr> as (name, location) pairs.

_TableParser()

How it works

<tr> starts a new row list; <td> inside a row starts a new cell buffer.
handle_data appends text into the open cell buffer (separate runs join).
</td> closes the cell (whitespace-collapsed); </tr> closes the row and appends it if non-empty.

Side effects: Mutates self.rows (the collected rows).

Raises: Never raises (malformed HTML simply yields fewer rows).

Related: Used by _parse_society_rows(); the content filters live there.

`_SearchText`

Collect the plain text of a page with <script>/<style> blocks dropped.

_SearchText()

How it works

handle_starttag increments a skip counter on <script>/<style>; handle_endtag decrements it (they can nest).
handle_data appends text chunks only while the counter is zero — script and style content is mostly JavaScript/CSS noise, not readable content.

Side effects: Mutates self.chunks (the collected text pieces).

Raises: Never raises (malformed HTML simply yields fewer chunks).

Related: Used by _parse_google_neighborhoods().

Nested helpers

Methods of the two HTML parser classes.

Name	What it does
`_TableParser.__init__`	Initialise the rows list and the current row/cell state.
`_TableParser.handle_starttag`	Start a row list on `<tr>` and a cell buffer on `<td>` inside a row.
`_TableParser.handle_data`	Append text into the open cell buffer (ignored outside cells).
`_TableParser.handle_endtag`	Close a cell (join and collapse whitespace) on `</td>`; close a row on `</tr>`.
`_SearchText.__init__`	Initialise the text chunk list and the skip counter.
`_SearchText.handle_starttag`	Increment the skip counter on `<script>`/`<style>`.
`_SearchText.handle_endtag`	Decrement the skip counter on `</script>`/`</style>` (guarded against 0).
`_SearchText.handle_data`	Append text into chunks while no script/style block is open.

Methods of `_TableParser`

`_TableParser.init`

Initialise the rows list and the current row/cell state.

__init__(self)

`_TableParser.handle_starttag`

Start a row list on <tr> and a cell buffer on <td> inside a row.

handle_starttag(self, tag, attrs)

`_TableParser.handle_data`

Append text into the open cell buffer (ignored outside cells).

handle_data(self, data)

`_TableParser.handle_endtag`

Close a cell (join and collapse whitespace) on </td>; close a row on </tr>.

handle_endtag(self, tag)

Methods of `_SearchText`

`_SearchText.init`

Initialise the text chunk list and the skip counter.

__init__(self)

`_SearchText.handle_starttag`

Increment the skip counter on <script>/<style>.

handle_starttag(self, tag, attrs)

`_SearchText.handle_endtag`

Decrement the skip counter on </script>/</style> (guarded against 0).

handle_endtag(self, tag)

`_SearchText.handle_data`

Append text into chunks while no script/style block is open.

handle_data(self, data)

License
- OSI Approved :: MIT License
Operating System
- OS Independent
Programming Language
- Python :: 3

Metadata

License
- OSI Approved :: MIT License
Operating System
- OS Independent
Programming Language
- Python :: 3

Release files for pk-address-checker 1.0.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for pk-address-checker 1.0.2
File	Size	Uploaded
pk_address_checker-1.0.2.tar.gz	211.9 kB	Oct 4, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for pk-address-checker 1.0.2
File	Interpreter	ABI	Platform	Reset
pk_address_checker-1.0.2-py3-none-any.whl 150.0 kB Oct 4, 2026	Python 3	none	any	Details

Total release size: 361.9 kB

Release files / pk_address_checker-1.0.2.tar.gz

Download URL	pk_address_checker-1.0.2.tar.gz
Size	211.9 kB
Tags	Source
SHA-256 checksum How to use checksums	`d523dc645fca0e7d8d9e96abf069ce0d5d79ea6f8c24b7d1db636576a6b6c077`
BLAKE2b-256 checksum How to use checksums	`1c7523cdb20d497a043b0e1f97b442405374f09f821f9d6f6860528b27d13171`
Upload date	Oct 4, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/7.0.0 CPython/3.13.14`

Release files / pk_address_checker-1.0.2-py3-none-any.whl

Download URL	pk_address_checker-1.0.2-py3-none-any.whl
Size	150.0 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`47072abcc9f50c8a3cf729a8371d8df210dd1d734c642e916c2a064b3744aff8`
BLAKE2b-256 checksum How to use checksums	`2fd40ee92fa1037d324cc3a1df29f70a2975f65456081afa9d23fafc572a3adb`
Upload date	Oct 4, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/7.0.0 CPython/3.13.14`

Release history Release notifications | RSS feed

This release

1.0.2 This release

Oct 4, 2026 2 release files

1.0.1

Oct 3, 2026 2 release files

1.0.0

Oct 3, 2026 2 release files

pk-address-checker 1.0.2

pk_address_checker — Documentation

Using a different data folder

Updating the data

API Reference

Quick start

Contents

Public API

check_address

is_valid_address

parse_address

Parsed

tehsil_district_conflict

normalize_province

norm

Data tables

load_admin_table

load_society_table

load_locality_table

load_islamabad_sectors

load_postcodes

Update pipelines

update_admin_table

update_society_table

update_neighborhoods_osm

update_neighborhoods_wiki

update_world_landmarks

update_postcodes

update_neighborhoods

Conflict checks

province_conflict

_city_mismatch

_district_conflict

_city_district_conflict

_tehsil_mismatch

_tehsil_city_conflict

_society_conflict

_sector_conflict

_locality_conflict

_postcode_conflict

Parsing helpers

_city_hints

_hint_positions

_stated_districts

_locality_from_text

_marker

_named_anchor

_town_anchor

_foreign_country

_locatable

_is_gibberish

_postcode_anchor

_clear_postcode

_labeled_postcode

_canonicalize

_canon_word

Matching and registry helpers

_name_matches

_name_pattern

_looks_like

_ratio

_alias

_index

_place_provinces

_universal_norms

_locality_norms

_td_norms

_locality_index

_landmark_names

_place_name_tokens

_claim_followers

_city_districts

_postcode_city

_clean

_split

_strip_div

_ok_osm_name

Download and parse helpers

fetch

fetch_first

`check_address`

`is_valid_address`

`parse_address`

`Parsed`

`tehsil_district_conflict`

`normalize_province`

`norm`

`load_admin_table`

`load_society_table`

`load_locality_table`

`load_islamabad_sectors`

`load_postcodes`

`update_admin_table`

`update_society_table`

`update_neighborhoods_osm`

`update_neighborhoods_wiki`

`update_world_landmarks`

`update_postcodes`

`update_neighborhoods`

`province_conflict`

`_city_mismatch`

`_district_conflict`

`_city_district_conflict`

`_tehsil_mismatch`

`_tehsil_city_conflict`

`_society_conflict`

`_sector_conflict`

`_locality_conflict`

`_postcode_conflict`

`_city_hints`

`_hint_positions`

`_stated_districts`

`_locality_from_text`

`_marker`

`_named_anchor`

`_town_anchor`

`_foreign_country`

`_locatable`

`_is_gibberish`

`_postcode_anchor`

`_clear_postcode`

`_labeled_postcode`

`_canonicalize`

`_canon_word`

`_name_matches`

`_name_pattern`

`_looks_like`

`_ratio`

`_alias`

`_index`

`_place_provinces`

`_universal_norms`

`_locality_norms`

`_td_norms`

`_locality_index`

`_landmark_names`

`_place_name_tokens`

`_claim_followers`

`_city_districts`

`_postcode_city`

`_clean`

`_split`

`_strip_div`

`_ok_osm_name`

`fetch`

`fetch_first`

`pdf_lines`

`parse_pbs_lines`

`_parse_ajk_yearbook`

`_parse_gb_glance`

`_parse_admin_lines`

`_parse_society_rows`

`_parse_cda_rows`

`_parse_pdf_rows`

`_parse_society_source`

`_fetch_postcode_rows`

`_wiki_landmarks`

`_osm_neighborhoods`

`_overpass`

`_parse_google_neighborhoods`