Learning3D: A Modern Library for Deep Learning on 3D Point Clouds Data.

Documentation | Blog | Demo

Learning3D is an open-source library that supports the development of deep learning algorithms that deal with 3D data. The Learning3D exposes a set of state of art deep neural networks in python. A modular code has been provided for further development. We welcome contributions from the open-source community.

Latest News:

1. [28 Feb, 2025]: CurveNet is now a part of learning3d library.
2. [7 Apr, 2024]: Now, learning3d is available as pypi package.
3. [24 Oct, 2023]: MaskNet++ is now a part of learning3d library.
4. [12 May, 2022]: ChamferDistance loss function is incorporated in learning3d. This is a purely pytorch based loss function.
5. [24 Dec. 2020]: MaskNet is now ready to enhance the performance of registration algorithms in learning3d for occluded point clouds.
6. [24 Dec. 2020]: Loss based on the predicted and ground truth correspondences is added in learning3d after consideration of Correspondence Matrices are Underrated paper.
7. [24 Dec. 2020]: PointConv, latent feature estimation using convolutions on point clouds is now available in learning3d.
8. [16 Oct. 2020]: DeepGMR, registration using gaussian mixture models is now available in learning3d
9. [14 Oct. 2020]: Now, use your own data in learning3d. (Check out UserData functionality!)

PyPI package setup

Setup from pypi server

pip install learning3d

Setup using code

git clone https://github.com/vinits5/learning3d.git
cd learning3d
git checkout pypi_v0.1.0
python3 -m pip install .

Available Computer Vision Algorithms in Learning3D

Available Pretrained Models

1. PointNet
2. PCN
3. PointNetLK
4. PCRNet
5. DCP
6. PRNet
7. FlowNet3D
8. RPM-Net (clean-trained.pth, noisy-trained.pth, partial-pretrained.pth)
9. DeepGMR
10. PointConv (Download from this link)
11. MaskNet
12. MaskNet++ / MaskNet2
13. CurveNet

Available Datasets

1. ModelNet40

Available Loss Functions

1. Classification Loss (Cross Entropy)
2. Registration Losses (FrobeniusNormLoss, RMSEFeaturesLoss)
3. Distance Losses (Chamfer Distance, Earth Mover's Distance)
4. Correspondence Loss (based on this paper)

Technical Details

Supported OS

1. Ubuntu 16.04
2. Ubuntu 18.04
3. Ubuntu 20.04.6
4. Linux Mint
5. macOS Sequoia 15.3.1

Requirements

1. CUDA 10.0 or higher
2. Pytorch 1.3 or higher
3. Python 3.8

How to use this library?

Important Note: Clone this repository in your project. Please don't add your codes in "learning3d" folder.

1. All networks are defined in the module "models".
2. All loss functions are defined in the module "losses".
3. Data loaders are pre-defined in data_utils/dataloaders.py file.
4. All pretrained models are provided in learning3d/pretrained folder.

Documentation

B: Batch Size, N: No. of points and C: Channels.

Use of Point Embedding Networks:

from learning3d.models import PointNet, DGCNN, PPFNet
pn = PointNet(emb_dims=1024, input_shape='bnc', use_bn=False)
dgcnn = DGCNN(emb_dims=1024, input_shape='bnc')
ppf = PPFNet(features=['ppf', 'dxyz', 'xyz'], emb_dims=96, radius='0.3', num_neighbours=64)

Use of Classification / Segmentation Network:

from learning3d.models import Classifier, PointNet, Segmentation
classifier = Classifier(feature_model=PointNet(), num_classes=40)
seg = Segmentation(feature_model=PointNet(), num_classes=40)

Use of Registration Networks:

from learning3d.models import PointNet, PointNetLK, DCP, iPCRNet, PRNet, PPFNet, RPMNet
pnlk = PointNetLK(feature_model=PointNet(), delta=1e-02, xtol=1e-07, p0_zero_mean=True, p1_zero_mean=True, pooling='max')
dcp = DCP(feature_model=PointNet(), pointer_='transformer', head='svd')
pcrnet = iPCRNet(feature_moodel=PointNet(), pooling='max')
rpmnet = RPMNet(feature_model=PPFNet())
deepgmr = DeepGMR(use_rri=True, feature_model=PointNet(), nearest_neighbors=20)

Use of Inlier Estimation Network (MaskNet):

from learning3d.models import MaskNet, PointNet, MaskNet2
masknet = MaskNet(feature_model=PointNet(), is_training=True) masknet2 = MaskNet2(feature_model=PointNet(), is_training=True)

Use of Point Completion Network:

from learning3d.models import PCN
pcn = PCN(emb_dims=1024, input_shape='bnc', num_coarse=1024, grid_size=4, detailed_output=True)

Use of PointConv:

Use the following to create pretrained model provided by authors.

from learning3d.models import create_pointconv
PointConv = create_pointconv(classifier=True, pretrained='path of checkpoint')
ptconv = PointConv(emb_dims=1024, input_shape='bnc', input_channel_dim=6, classifier=True)

OR
Use the following to create your own PointConv model.

PointConv = create_pointconv(classifier=False, pretrained=None)
ptconv = PointConv(emb_dims=1024, input_shape='bnc', input_channel_dim=3, classifier=True)

PointConv variable is a class. Users can use it to create a sub-class to override create_classifier and create_structure methods in order to change PointConv's network architecture.

Use of Flow Estimation Network:

from learning3d.models import FlowNet3D
flownet = FlowNet3D()

Use of Data Loaders:

from learning3d.data_utils import ModelNet40Data, ClassificationData, RegistrationData, FlowData
modelnet40 = ModelNet40Data(train=True, num_points=1024, download=True)
classification_data = ClassificationData(data_class=ModelNet40Data())
registration_data = RegistrationData(algorithm='PointNetLK', data_class=ModelNet40Data(), partial_source=False, partial_template=False, noise=False)
flow_data = FlowData()

Use Your Own Data:

from learning3d.data_utils import UserData
dataset = UserData(application, data_dict)

Use of Loss Functions:

from learning3d.losses import RMSEFeaturesLoss, FrobeniusNormLoss, ClassificationLoss, EMDLoss, ChamferDistanceLoss, CorrespondenceLoss
rmse = RMSEFeaturesLoss()
fn_loss = FrobeniusNormLoss()
classification_loss = ClassificationLoss()
emd = EMDLoss()
cd = ChamferDistanceLoss()
corr = CorrespondenceLoss()

To run codes from examples:

1. Copy the file from "examples" folder outside of the directory "learning3d"
2. Now, run the file. (ex. python test_pointnet.py)

• Your Directory/Location
  • learning3d
  • test_pointnet.py

References:

1. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
2. Dynamic Graph CNN for Learning on Point Clouds
3. PPFNet: Global Context Aware Local Features for Robust 3D Point Matching
4. PointConv: Deep Convolutional Networks on 3D Point Clouds
5. PointNetLK: Robust & Efficient Point Cloud Registration using PointNet
6. PCRNet: Point Cloud Registration Network using PointNet Encoding
7. Deep Closest Point: Learning Representations for Point Cloud Registration
8. PRNet: Self-Supervised Learning for Partial-to-Partial Registration
9. FlowNet3D: Learning Scene Flow in 3D Point Clouds
10. PCN: Point Completion Network
11. RPM-Net: Robust Point Matching using Learned Features
12. 3D ShapeNets: A Deep Representation for Volumetric Shapes
13. DeepGMR: Learning Latent Gaussian Mixture Models for Registration
14. CMU: Correspondence Matrices are Underrated
15. MaskNet: A Fully-Convolutional Network to Estimate Inlier Points
16. MaskNet++: Inlier/outlier identification for two point clouds
17. CurveNet: Walk in the Cloud: Learning Curves for Point Clouds Shape Analysis