property-shared 1.1.2

Pure-Python core library + FastAPI service for UK property data (PPD, EPC, Rightmove, Planning).

Property Shared

FastAPI service + pure-Python core library for UK property data. Integrates Land Registry (PPD), EPC, Rightmove, and Planning portal lookup (98 councils; optional vision-guided scraping via Playwright + OpenAI). Use as a library, HTTP API, CLI, or MCP server.

How to run

Dev (uv)

1. Create .env from .env.example (set EPC_API_EMAIL/EPC_API_KEY if you want EPC enabled)
2. Install deps: uv sync --extra dev
3. Run API: uv run property-api (or uv run uvicorn app.main:app --reload)
4. CLI (core mode): uv run --extra cli property-cli meta (add --api-url http://localhost:8000 to hit the running API instead of local core)
5. Demo UI: visit http://localhost:8000/demo (served by the same FastAPI app)
6. Quick checks:
   • Health: curl http://localhost:8000/v1/health
   • Integration status: curl http://localhost:8000/v1/meta/integrations
   • Rightmove: curl 'http://localhost:8000/v1/rightmove/search-url?postcode=SW1A%201AA&radius=0.25' then curl 'http://localhost:8000/v1/rightmove/listings?search_url=<pasted_url>&max_pages=1'
   • PPD address search: curl 'http://localhost:8000/v1/ppd/address-search?postcode_prefix=B1&street=Broad%20Street&limit=5'
   • PPD comps: curl 'http://localhost:8000/v1/ppd/comps?postcode=NG1%201GF&months=24&limit=5'

Live integration tests

Live tests make real network calls and are gated:

• Run: RUN_LIVE_TESTS=1 uv run --extra dev pytest -q -s

Fly.io (high-level)

• Set secrets: fly secrets set EPC_API_EMAIL=... EPC_API_KEY=...
• Deploy: fly deploy

Python SDK (OpenAPI)

Generate a typed client from the running service OpenAPI:

1. Run the API: uv run uvicorn app.main:app --reload
2. Generate client: uv run --extra dev openapi-python-client generate --url http://localhost:8000/openapi.json --output-path clients/python

Structure

• property_core/ – pure-Python core library (no FastAPI, no DB/Redis assumptions)
  • models/ – domain Pydantic models (PPDTransaction, EPCData, PropertyReport, etc.)
  • ppd_client.py – transport: Land Registry SPARQL + Linked Data API → typed PPD models
  • epc_client.py – transport: EPC registry (async) → typed EPCData models
  • rightmove_scraper.py – transport: listings scraper (sync) → typed Pydantic models
  • rightmove_location.py – transport: search URL builder
  • ppd_service.py – domain service: SPARQL parsing → typed PPD models (sync)
  • planning_service.py – domain service: council matching + URL building (sync)
  • report_service.py – product pipeline: multi-source aggregation → PropertyReport (async)
  • enrichment.py – EPC enrichment pipeline for PPD comps
  • address_matching.py – fuzzy address matching for EPC enrichment
  • yield_service.py – yield analysis: PPD sales + Rightmove rentals
  • planning_scraper.py – vision-guided planning portal scraper (Playwright + OpenAI)
  • planning_councils.json – verified council database (98 councils, 6 system types)
• app/ – FastAPI service wrapping property_core
  • api/v1/ – versioned routers (thin HTTP wrappers)
  • schemas/ – API envelope models (import domain models from core)
  • services/ – API-specific adapters (async threading, rate limiting)
  • core/config.py – settings via pydantic-settings
• property_cli/ – Typer CLI with dual mode (core direct vs API)
• docs/ – examples and reference documentation
• USER_GUIDE.md – quickstart and endpoint/CLI usage

Local setup

1. Copy .env.example to .env and fill values (EPC keys, OPENAI_API_KEY for planning scraper)
2. Install deps: uv sync --extra dev
3. Run: uv run property-api (or uv run uvicorn app.main:app --reload)

Notes

• All domain models carry a raw field with original source data (always populated by classmethods).
• Rightmove politeness is in-memory (app/services/rightmove_service.py) for now; projects can swap in Redis later if needed.
• Rightmove search URLs default to a small radius (0.25 miles); override radius to widen/narrow.
• Station distances in listing details are rounded to 1 decimal place (e.g., "1.9 miles").
• Rental analysis (analyze_rentals) uses IQR-based outlier filtering for the rent range.
• See docs/examples.md for copy-paste usage examples with real output.

API I/O contracts (summary)

• GET /v1/health → { "status": "ok" }
• GET /v1/meta/integrations → { environment, integrations: { ppd|rightmove|epc: { available, configured } } }
• GET /v1/ppd/download-url?kind=complete|monthly|year&year?&part?&fmt=csv|txt → { url }
• GET /v1/ppd/transactions?postcode|postcode_prefix&limit&filters...&include_raw=bool (one of postcode/postcode_prefix) → { count, limit, offset, results: [ { transaction_id, price, date, postcode, property_type, estate_type, transaction_category, new_build, paon, saon, street, town, county, locality, district } ], warnings, raw? }
• GET /v1/ppd/address-search?paon?&saon?&street?&town?&county?&locality?&district?&postcode?&postcode_prefix?&limit&include_raw=bool (requires ≥2 fields, limit≤50) → same shape as /transactions (note: PPD town can differ from local usage; locality often matches better)
• GET /v1/ppd/comps?postcode&property_type?&months?&limit?&search_level=postcode|sector|district&enrich_epc=bool → { query, count, median, mean, min, max, thin_market, transactions: [PPDTransaction] } (when enrich_epc=true, each transaction gains epc_match (full normalized cert), epc_match_score (0-100 fuzzy confidence), epc_floor_area_sqm, epc_floor_area_sqft, price_per_sqm, price_per_sqft, epc_rating, epc_score, epc_construction_age, epc_built_form)
• GET /v1/ppd/transaction/{id}?view=all|basic&include_raw=bool → { record: { transaction_id, price_paid, transaction_date, property/transaction metadata... }, raw? }
• GET /v1/epc/search?postcode&address?&include_raw=bool → { record, raw? } (returns 501-style response if EPC creds not configured)
• GET /v1/rightmove/search-url?postcode&property_type=sale|rent&radius?&min/max price/bedrooms? → { url }
• GET /v1/rightmove/listings?search_url&max_pages? → { count, results: [ { id, url, price, currency, bedrooms, bathrooms, address, summary, property_type, agent_name, agent_branch, first_visible_date, images, raw } ] } (raw always included)
• GET /v1/rightmove/listing/{property_id} → { result: { id, url, price, bedrooms, bathrooms, address, description, property_type, tenure_type, years_remaining_on_lease, annual_service_charge, annual_ground_rent, ground_rent_review_period_years, council_tax_band, latitude, longitude, floorplans, key_features, display_size, raw, ... } }
• Planning API routes exist in code (app/api/v1/planning.py) but are disabled in the router — scraping requires a UK residential IP. Use property_core.PlanningService and property_core.planning_scraper directly as a library instead.
• POST /v1/property/report body: { address, include_rentals?, include_sales_market?, ppd_months?, search_radius? } → PropertyReport { report_id, key_insights, estimated_value_low/high, sale_history, market_analysis, energy_performance, rental_analysis, current_market, sources } (supports ?format=html)

Rightmove CLI snippets

• Build a search URL: uv run --extra cli property-cli rightmove search-url --postcode SW1A 1AA --property-type sale --radius 0.25
• Fetch listings from a search URL: uv run --extra cli property-cli rightmove listings --search-url "<rightmove_url>" --max-pages 1
• Fetch individual listing detail: uv run --extra cli property-cli rightmove listing 161151632

Other CLI commands (core mode; add --api-url to hit the API)

• Meta integrations: uv run --extra cli property-cli meta
• PPD comps (postcode is positional): uv run --extra cli property-cli ppd comps "SW1A 1AA" --months 24 --limit 20 --search-level sector
• PPD comps with EPC enrichment: uv run --extra cli property-cli ppd comps "B1 1BB" --search-level sector --enrich-epc
• PPD transactions (postcode/prefix): uv run --extra cli property-cli ppd search --postcode-prefix SW1A --limit 10
• PPD transaction record: uv run --extra cli property-cli ppd transaction 31C68072-E0B5-FEE3-E063-4804A8C04F37 --include-raw (replace with a real transaction id)
• EPC search (requires EPC_API_EMAIL/EPC_API_KEY set): uv run --extra cli property-cli epc search "SW1A 1AA" --address "10 Downing Street" --include-raw
• Property report: uv run --extra cli property-cli report generate "10 Downing Street, SW1A 2AA" -o report.html --html

Library also contains:

✅ Typed Pydantic models throughout — all transport clients and domain services return typed models ✅ Every model carries raw field with original source data (always populated) ✅ Standalone address matching module for EPC enrichment ✅ CLAUDE.md as living documentation ✅ Tests catch regressions