libkdv 1.1.20

A library of feature heatmap algorithm

LIBKDV - A Versatile Kernel Density Visualization Library for Geospatial Analytics

Kernel Density Visualization (KDV) has been extensively used for many geospatial analysis tasks. Some representative examples include traffic accident hotspot detection, crime hotspot detection, and disease outbreak detection. Although many scientific software packages, including Scipy and Scikit-learn, geographical software packages, including QGIS and ArcGIS, and visualization software packages, including Deck.gl and KDV-Explorer, can also support KDV, none of these tools, to the best of our knowledge, can be scalable to high resolution size (e.g., 2560 x 1920) and large-scale datasets (e.g., one million data points). Therefore, the huge computational cost limits the applicability of using the off-the-shelf software tools to support advanced (or more complex) geospatial analytics, e.g., bandwidth-tuning analysis and spatiotemporal analysis, which involves computing multiple KDVs in one batch.

Introduction:

To overcome the above issue, we develop the first versatile programming library (LIBKDV), by combining our recent studies (SLAM [1] and SWS [2]), which can reduce the worst-case time complexity for supporting different types of KDV-based geospatial analytics, including:

(1) Bandwidth-tuning analysis (cf. Figure 1): Domain experts can first set multiple bandwidths in a batch, and then generate multiple KDVs with respect to these bandwidths.

(2) Spatiotemporal analysis (cf. Figure 2): Domain experts can leverage a more complex spatiotemporal kernel density function to generate time-dependent hotspot maps that correspond to different timestamps.

To further enhance the efficiency for these two tasks, we fully parallelize our methods, SLAM and SWS.

Installation Guidelines:

1. First, build the virtual environment in the Anaconda (recommended Python 3.9)

conda create -n libkdv python=3.9

2. Activate the virtual environment

conda activate libkdv

3. Install the dependencies and the library

conda install -c conda-forge geopandas keplergl notebook
pip install libkdv

4. Anticipated problem(s) and possible solution(s)

OSError: could not find or load spatialindex_c-64.dll

pip install rtree==0.9.3

How to Use:

1. Import LIBKDV and Pandas in your code

import libkdv
import pandas as pd

2. Create the LIBKDV object and compute the heatmap

libkdv_obj = kdv(dataset, KDV_type,
                 GPS=true, 
                 bandwidth_s=1000, row_pixels=800, col_pixels=640, 
                 bandwidth_t=6, t_pixels=32,
                 num_threads=8)
libkdv_obj.compute()

Required arguments

dataset: Pandas object, the dataset. (for preparation, please refer to the steps in data_processing.ipynb)
KDV_type: String, "KDV" - single KDV or "STKDV" - Spatio-Temporal KDV.

Optional arguments

GPS: Boolean, true *- use geographic coordinate system * or false - use simple (X, Y) coordinates (evaluation.ipynb).
bandwidth_s: Float, the spatial bandwidth (in terms of meters), default is 1000.
row_pixels: Integer, the number of grids in the x-axis, default is 800.
col_pixels: Integer, the number of grids in the y-axis, default is 640.
bandwidth_t: Float, the temporal bandwidth (in terms of days), default is 6. REQUIRED if KDV_type="STKDV".
t_pixels: Integer, the number of grids in the t-axis, default is 32. REQUIRED if KDV_type="STKDV".
num_threads: Integer, the number of threads, default is 8.

Example for computing a single KDV:

NewYork = pd.read_csv('./Datasets/New_York.csv')
traffic_kdv = kdv(NewYork,KDV_type="KDV",bandwidth_s=1000)
traffic_kdv.compute()

Example for supporting the bandwidth-tuning analysis task:

bandwidths_traffic_kdv = [500,700,900,1100,1300,1500,1700,1900,2100,2300] #Set the bandwidths
result_traffic_kdv = [] #Stores the final results
traffic_kdv = kdv(NewYork,KDV_type="KDV")
for band in bandwidths_traffic_kdv:
    kdv_traffic_kdv.bandwidth_s = band
    result_traffic_kdv.append(traffic_kdv.compute())

Example for supporting the spatiotemporal analysis task:

NewYork = pd.read_csv('./Datasets/New_York.csv')
traffic_kdv = kdv(NewYork,KDV_type="STKDV",bandwidth_s=1000,bandwidth_t=10)
traffic_kdv.compute()

3. Show the heatmaps by KerplerGL

To generate a single KDV or support the spatiotemporal analysis task, you can use the following code.

from keplergl import KeplerGl
map_traffic_kdv = KeplerGl(height=600, data={"data_1": traffic_kdv.result})
map_traffic_kdv

To support the bandwidth-tuning analysis task, you can use the following code.

from keplergl import KeplerGl
map_traffic_kdv_bands = KeplerGl(height=500)

for i in range(len(bandwidths_traffic_kdv)):
    map_traffic_kdv_bands.add_data(data=result_traffic_kdv[i], name='data_%d'%(i+1))
map_traffic_kdv_bands

Sample datasets:

We offer three sample datasets for testing, which are (1) Seattle crime dataset [a], (2) New York traffic accident dataset [b], and (3) Ontario COVID-19 dataset [c]. The python code (data_processing.py) and the Jupyter notebook (data_processing.ipynb) for extracting these datasets are provided in this Github link.

[a] Seattle Open Data. https://data.seattle.gov/Public-Safety/SPD-Crime-Data-2008-Present/tazs-3rd5.
[b] NYC Open Data. https://data.cityofnewyork.us/Public-Safety/Motor-Vehicle-Collisions-Crashes/h9gi-nx95.
[c] Ontario Open Data. https://data.ontario.ca/dataset/confirmed-positive-cases-of-covid-19-in-ontario.

Advantages:

There are three main advantages for using our LIBKDV.
Easy-to-use software package: Domain experts only need to write a few lines of python codes for using our LIBKDV, which is as easy as using other python packages, including Scikit-learn and Scipy.
High efficiency: LIBKDV is the first library that can reduce the worst-case time complexity for generating KDV, which cannot be achieved by other software tools. Therefore, instead of calling the KDV function in other python packages (Scikit-learn and Scipy), domain experts can call our efficient KDV function in LIBKDV.
High versatility: Due to the high efficiency of LIBKDV, our library can support more KDV-based geospatial analysis tasks, including bandwidth-tuning analysis and spatiotemporal analysis, which cannot be natively and feasibly supported by other software tools.

Project Members:

Prof. (Edison) Tsz Nam Chan, Hong Kong Baptist University
Mr. Pak Lon Ip, Universiy of Macau
Mr. Kaiyan Zhao, Universiy of Macau
Prof. (Ryan) Leong Hou U, Universiy of Macau
Prof. Byron Choi, Hong Kong Baptist University
Prof. Jianliang Xu, Hong Kong Baptist University