Where this data comes from

Data sources

How listings reach the map

1. Scrape — each source is fetched via a small Node.js script (npm run data:bayut, npm run data:propertyfinder) producing raw per-source JSON files in data/raw/.
2. Normalise — each listing is reduced to a common shape: id, source, url, title, type, price (SAR), area (m²), bedrooms, bathrooms, lat/lng, image, scraped_at.
3. Geo-match — each listing's lat/lng is point-in-polygon-tested against the 103 OSM neighbourhood polygons; only matches are surfaced on the map. 85% of listings currently match — the rest sit just outside any named neighbourhood.
4. Aggregate — per neighbourhood we compute count, median, min and max price for the badges and panels.
5. Build — the result ships as static data/listings.json, served by Next.js. No database, no runtime API for the public portal.

How distance to Masjid Nabawi is calculated

For every neighbourhood, we compute the area-weighted polygon centroid and ask OpenRouteService for both the driving and walking route to Masjid Nabawi (24.4672°N, 39.6112°E). The result is cached in data/distances.json and re-used on every page load — no live traffic, no per-visit API calls.

Two semantic post-processing flags are applied because raw routing produces misleading numbers around the Haram precinct:

• contains_destination — when a polygon contains Masjid Nabawi itself, distances are forced to zero. Currently flags 1 neighbourhood (Al Haram Al Sharif).
• pedestrian_zone_nearby — when a driving route runs more than 3× the walking distance and walking is under 2 km, the panel surfaces walking time as the realistic metric. The car has to detour around the pedestrian Haram complex; walkers don't. Currently flags 4 neighbourhoods adjacent to the Haram.

Known limitations

• One photo per listing

  Bayut JSON-LD only exposes a single image per listing. Multi-photo galleries require either a different scrape strategy (Algolia API, browser rendering) or photo enrichment. Tracked as backlog item B2.

• Wasalt is blocked

  Wasalt.sa sits behind a Cloudflare managed challenge and currently cannot be scraped without a headless browser. The infrastructure is in place — only the source is missing.

• Zone status is curated, not authoritative

  The Saudi government publishes official ownership zones in stages under the 22 January 2026 law. Until those publications are complete, this portal treats most of the city as 'Pending'. Always verify with an official source before transferring any funds.

• No live traffic

  Driving times reflect static OSM road weights, not current congestion. For a buyer comparing neighbourhoods this is fine; for an actual departure plan, use a live navigation app.

• Listings are public-portal snapshots

  Properties may sell, change price or be re-listed between refreshes. Treat the count and price aggregates as a market signal, not a real-time inventory.

Replicating this stack

Anyone with access to the codebase can rebuild the data layer from scratch:

npm run data:neighbourhoods   # OSM polygons
npm run data:amenities        # OSM amenity points
npm run data:bayut            # Bayut listings (raw)
npm run data:propertyfinder   # PropertyFinder listings (raw)
npm run data:listings         # combine + geo-match
npm run data:distances        # ORS routes (requires ORS_TOKEN)
npm run data:distances:fix    # post-process flags