SP-SEG: Proof of concept tool for manual segmentation by spheripixel projection

In this repository, you can find a proof of concept of the tool for manual point cloud labeling by spherical point-pixel projection.

❗ Limitations

The current version is just the proof of concept version.

🔧 Environment and installation

python >= 3.10

Required packages:

"PyQt5",
"Pillow",
"numpy",
"pydantic",
"scikit-learn",
"scipy",
"colormath",
"tqdm",
"toml"

To install the software:

1. Clone the repository and run inside:

pip install .

1. Or use:

pip install sp-seg

If you're experiencing issues with installing the package, install firstly the dependencies:

pip install PyQt5 Pillow numpy pydantic scikit-learn scipy colormath tqdm toml

Then, you can run the software with the command:

D:\>spseg

📖 Instruction

The main view of the software looks like the following:

1. Starting the application
2. Opening point cloud
3. Loading labels definition
4. Zooming in and out
5. Depth preview
6. Depth scale
7. Vertical roll
8. Horizontal roll
9. Object segentation
10. Deselecting region
11. Changing mask opacity
12. Accepting an object
13. Saving segmentation result

Starting the application

Having installed the application, just run the following command in the command line:

D:\>spseg

Opening a point cloud

To open a point cloud, click Open icon in the toolbar and choose a proper file.

❗ At the moment only PTS files are supported

❗ If you need to change the columns meaning of PTS opener, modify the opener.column key of the config file. Currently interpretable letters are X, Y, Z, R, G, and B

Loading labels definition

To open labels click the proper icon in the toolbar and choose the JSON file with labels definitions. Example labels definition may look like below:

[
  {
    "label": "ceiling",
    "code": 0,
    "color": [4, 255, 0]
  },
  {
    "label": "floor",
    "code": 1,
    "color": [255, 128, 0]
  }
]

Zooming in and out

For easier detail selection you can zoom in and zoom out. To do so keep Ctrl pressed and use mouse wheel.

Depth preview

Sometimes, it may be useful to get insight into depth distribution. To see the depth map, just press Alt button.

Depth scale

To change $\gamma$ of the depth map, use + or - buttons while Alt is pressed.

Vertical roll

To roll the scene vertically, press Q button.

Horizontal roll

To roll the scene horizontally, press W button.

Object segmentation

To segment an object, choose the proper category in the menu of the left and start selecting the polygon. To add a vertex to the polygon, use left mouse button. If you want to close a polygon, click right mouse button.

❗ Closing polygon creates an edge between the first and the last added vertex!

❗ You don't need to select the entire object using a single polygon. You can draw as many polygons for an object as you wish, until the object segmentation will be accepted by pressing F1 button.

Deselecting region

You can deselect fragments of scene selected by mistake. To do so, check the mode to Deselect (radio button in the tollbar) or use Caps Lock button. For deselecting you also draws a polygon.

❗ Remember to change the mode to Select when you already deselected the desired points.

Changing mask opacity

To change the opacity of the mask of currently segmented object, use the slider just below the toolbar.

Accepting an object

When you finish the object segmentation you need to accept the instance of the category by clicking F1 button.

Saving segmentation result

To save the PTS result, choose the proper icon in the toolbar menu.

🏎️ Further development

• code optimization
• code cleaning
• add handling other point cloud formats
• handle non-single-setup point clouds
• enable saving draft segmentation result

📝 Citation

@ONLINE{sp-seg,
  author = {Walczak, J., Osuch, A., Wojciechowski, A.},
  title = {{SP-SEG}: Proof of concept tool for manual segmentation by spheripixel projection},
  year = 2023,
  url = {https://github.com/jamesWalczak/sp-seg-tool},
  urldate = {<access date here>}
}

⚖️ 3rd party attribution

Icons attribution: Save icons created by Freepik - Flaticon
Animation attribution: Animation generated from Icons8

🎁 Ackowledgement

This software was developed as a part of the research project "Semantic analysis of 3D point clouds", co-funded by the Polish National Center for Research and Development under the LIDER XI program [grant number 0092/L-11/2019].