pywhu3d 0.2.16

Example pywhu3d tool Package

Description

This is a Python tool for the WHU-Urban3D dataset. Contact Xu Han (hanxuwhu[at]whu[dot]edu[dot]com or [e-mail]) if you have any questions.

Installation

To use the pywhu3d tool, you need to install the pwhu3d library for your interpreter. We recommend you use python=3.7 to follow this tutorial.

# this will install the latest version of pywhu3d
pip install pywhu3d

Usage

Initialization

Create a WHU3D object:

from pywhu3d.tool import WHU3D

data_root = '/data/datasets/whu3d-dataset'  
scenes = ['0404', '0940']
# whu3d = WHU3D(data_root=data_root, data_type='mls', format='txt')
whu3d = WHU3D(data_root=data_root, data_type='mls', format='h5', scenes=scenes)

Parameters:

• data_root: [data root folder]
• data_type: als, mls, pc, img
• format: txt, ply, npy, h5, pickle
• [optional] scenes: a list of scenes, if not specified, will be represented by all of the files

The structure of the data folder should be like this:

data_root
├── als
│   ├── h5
│   │   ├── [scene_1].h5
│   │   ├── [scene_2].h5
│   │   └── [scene_*].h5
│   └── [optional] pkl/npy/pth
└── mls
	├── images
    ├── h5
    │   ├── [scene_1].h5
    │   ├── [scene_2].h5
    │   └── [scene_*].h5
    └── [optional] pkl/npy/pth

If you have not downloaded the dataset, you could use the get_download function. For the case where we define whu3d as WHU3D(data_root=data_root, data_type='mls', format='h5', scenes=scenes), the downloaded data is MLS point cloud with h5 format. Or you could download the whole dataset using the parameter full=True. You could also choose to download the dataset from BaiduNetdisk or Google Drive.

# this will open a download page for your defined whu3d
whu3d.get_download(src='google')

# if you want to download the full dataset
whu3d.get_download(full=True, src='baidu')

It is also recommended to use default split scenes to create a whu3d object, by using whu3d.train_split.

# print(whu3d.split.val)
whu3d = WHU3D(data_root=data_root, data_type='mls', format='txt', scenes=whu3d.val_split)

Then some of the attributes could be directly accessed, including data_root, data_type, scenes, download_link

# e.g., you could print current scenes
print(whu3d.scenes)

Attributes

The attributes of whu3d may differ depending on your operations (e.g., after applying the compute_normals function, the attributes may include normals that may not exist before). Nonetheless, you could always use the list_attributes function to see the current attributes that you could currently access.

# this command will show you a table with all the attributes
# that you could currently use.
whu3d.list_attributes()

You could simply get a specific attribute of all scenes by using get_attribute function.

# this function will return a list of the attributes
attr = whu3d.get_attribute('coords')

Data

You could access the data of a specific scene by using whu3d.data[scene][attribute].

xyz = whu3d.data['0414']['coords']

Labels

Labels could also be directly accessed.

semantics = whu3d.labels['0414']['semantics']
instances = whu3d.labels['0414']['instances']

If you have interpreted the labels by using interprete_labels function, you could also get interpreted labels.

semantics = whu3d.interpreted_labels['0414']['semantics']
instances = whu3d.interpreted_labels['0414']['instances']

Visualization

Point cloud

You can visualize a specific scene or a list of scenes using the vis function. By default, this function will show both the point cloud and image frames, and the points are randomly sampled with sample_ratio = 0.01 for faster visualization. It will show color according to the height of the point if color is not specified, or you could choose a specific color, including intensity, normals, semantics, instances, and other features (some features should be computed first via whu3d functions if they do not exist, and you could use whu3d.list_attributes() to see the current attributes first).

# This will show sampled points and images
whu3d.vis(scene='0414', type='pc', color='intensity')

# Show all the points
whu3d.vis(scene='0414', sample_ratio=1.0, type='pc', color='intensity')

# if you want to show normals, please set 'show_normals' to True
whu3d.vis(scene='0414', type='pc', color='normals', show_normals=True)

or you can use a remote visualization function that allows you to visualize the scene on your local machine if the script is run on a remote server.

# This function should be used if you want to visualize points
# and the script is run on a remote machine.
whu3d.remote_vis(scene='0424', type='pc', color='intensity')

Before running the remove_vis function on your remote machine, you should start another ssh connection to your remote machine, and launch open3d on your local machine.

Images

Similarly, you could use the vis function to see a series of images of a specific scene.

whu3d.vis(scene='0414', type='img')

BEV

[Will be available soon.]

Renderings

[Will be available soon.]

Labels

If you want to visualize the labels of semantics or instances, you must run the interprete_labels function first (please refer to the 'labels interpretation' section).

# you should run this function first to interpret the labels
info, labels = whu3d.interprete_labels()

# you could visualize semantics with specified colors
whu3d.vis(scene='0414', type='pc', color='semantics')

# or you could visualize instances with random colors
whu3d.vis(scene='0414', type='pc', color='instances')

Export

Note that all the export functions will export data to self.data_path by default and you should better not change it if you want to load it later via pywhu3d.

Export data

You could export other formats of whu3d, including las, ply, numpy, pickle, h5py, image, et al, by just using the export_[type] function.

scenes = ['0404', '0940']
whu3d.export_h5(output='.')
whu3d.export_images(output='.', scenes=scenes)

# this will export las to the '[self.data_path]/las' folder if
# output is not specified, you can also specify 'scenes'
whu3d.export_las()

If scenes is not specified, it will export all the scenes by default.

Export labels

export_labels function could export raw labels or interpreted labels.

# this will export '[scene].labels' files to your 'output' folder
whu3d.export_labels(output='./labels', scenes=scenes)
# whu3d.export_labels()

Export statistics

You could also export detailed statistics of the data and label to excel by using the export_statistics function.

whu3d.export_statistics(output='./whu3d_statistics.xlsx')

For the export of metrics, you could refer to the 'Evaluation' part.

Custom export

You could use the export function to export a specified type of data.

whu3d.export(output='', attribute='interpreted_labels')

Labels interpretation

You could use the interprete_labels function to merge similar categories and remap the labels to consecutive numbers like 0, 1, 2, ...

# this will interpret the labels and create the 'gt' attribute
whu3d.interprete_labels()

After applying this function, you could access the interpreted labels by using whu3d.gt. For more information, you could use the get_label_map function to see the interpretation table.

# this will output a table showing the detailed information
# this only shows you the information of semantics
whu3d.get_label_map()

Block division

If you want to divide the whole scene into rectangle blocks along XY plane, you could use save_divided_blocks function. This function will directly save the divided blocks into .h5 file.

# this will divide the scene into 10m * 10m blocks with 5m overlap$
whu3d.save_divided_blocks(out_dir='', num_points=4096, size=(10, 10), stride=5, threshold=100, show_points=False)

Custom interpretation

If you could use your own file to interpret the labels, you should follow the steps:

Step1: Create label_interpretion.json. This file should include

{
"sem_no_list_ins": "2, 3, 7",
"sem_label_mapping": [
		{"175": "2"},
		{"18": "5"}
	]
}

sem_no_list_ins exclude the categories which should be not interpreted as instances; sem_label_mapping specifies the mapping rules of semantic labels.

Step 2: Put the JSON file into the data root folder.

Step 3: Perform the interprete_labels function.

Evaluation

The interpretation of predicted results should be consistent with that of the interpreted labels.

Semantic segmentation evaluation

Or you could use the evaluation tool as in the 'instance segmentation evaluation' section, just by replacing the instance results with semantics.

Instance segmentation evaluation

For instance segmentation evaluation, you should use our evaluation.Evaluator tool.

# define an evaluator for evaluation
# preds is a list with num_scenes items: 
# [scene_1_gt_arr, ..., scene_k_gt_arr]. Each item is a 2D
# array with shape (num_points, 2), of which the first column
# is semantic prediction and the second is instance prediction
# there are two ways to create an evaluator
# first way
evaluator = whu3d.create_evaluator(preds)
# second way
from pywhu3d.evluation import Evaluator
evaluator = Evaluator(whu3d, preds)

# then you could use evaluator functions
evaluator.compute_metrics()

You could get metrics, including:

• instance metrics: MUCov, MWCov, Pre, Rec, F1-score
• semantic metrics: oAcc, mAcc, mIoU

print(evaluator.info)
print(evaluator.eval_list)
print(evaluator.eval_table)

You could also export evaluation results.

# this will export an Excel file with detailed metrics
evaluator.export(output_dir='./')

Custom evaluation

If you want to define a different list of ground truth labels instead of using the default labels, you could use set_gt function to set the ground truth labels

from pywhu3d.evluation import Evaluator
evaluator = Evaluator(whu3d, preds)

# use this script to define your custom labels
# truths: a list of scenes [scene_1_gt_arr, ..., scene_k_gt_arr]
# gt_arr is a numpy array with shape (num_points, 2)
eval.set_gt(truths)

# then you could use evaluator functions
evaluator.compute_metrics()

Custom dataset

You can also use the whu3d tool to customize your own dataset for all pywhu3d features simply by using the format function.

data_root = '/data/datasets/you_custom_dataset'  
scenes = ['scene1', 'scene2']  
whu3d = WHU3D(data_root=data_root, data_type='mls', format='txt', scenes=scenes)

# this will format your data as whu3d format
# 'attributes' should be consistent with your input data
in_attributes = ['coords', 'semantics', 'instances', 'intensities']
whu3d.format(attributes=in_attributes)

After applying the format function, you could use all the features the whu3d tool provides just as the whu3d-dataset.

Demo

This is a demo for preprocessing MLS dataset.

from pywhu3d.tool import WHU3D

data_root = 'data/whu-dataset'  
mls_scenes = ['0404', '0940']  
# als_scenes = ['5033', '3922']
# whu3d = WHU3D(data_root=data_root, data_type='mls', format='txt')
whu3d = WHU3D(data_root=data_root, data_type='mls', format='h5', scenes=mls_scenes)

whu3d.norm_coords()  
# self.compute_normals()  
whu3d.interprete_labels() # only for the dataset of city A
whu3d.compute_normals(radius=0.8)
whu3d.save_divided_blocks(out_dir='', num_points=60000, size=(20, 20), stride=10, threshold=100, show_points=False)

More

pywhu3d is a tool to manage the whu3d dataset, with limited ability to process the dataset (e.g., segmentation). But if you need more features for processing the outdoor scene dataset, you could refer to [well soon be available]. For more details about our dataset, please refer to our website.