GRNet: Gridding Residual Network for Dense Point Cloud Completion

Introduction

With the rapid development of 3D acquisition technologies, 3D sensors (e.g., LiDARs) are becoming increasingly available and affordable. As a commonly used format, point clouds are the preferred representation for describing the 3D shape of an object. Complete 3D shapes are required in many applications, including semantic segmentation and SLAM . However, due to limited sensor resolution and occlusion, highly sparse and incomplete point clouds can be acquired, which causes loss in geometric and semantic information. Consequently, recovering the complete point clouds from partial observations, named point cloud completion, is very important for practical applications.

In the recent few years, convolutional neural networks (CNNs) have been applied to 2D images and 3D voxels. Since the convolution can not be directly applied to point clouds due to their irregularity and unorderedness, most of the existing methods voxelize the point cloud into binary voxels, where 3D convolutional neural networks can be applied. However, the voxelization operation leads to an irreversible loss of geometric information. Other approaches use the Multi-Layer Perceptrons (MLPs) to process point clouds directly. However, these approaches use max pooling to aggregate information across points in a global or hierarchical manner, which do not fully consider the connectivity across points and the context of neighboring points. More recently, several attempts have been made to incorporate graph convolutional networks (GCN) to build local graphs in the neighborhood of each point in the point cloud. However, constructing the graph relies on the K-nearest neighbor (KNN) algorithm, which is sensitive to the point cloud density .

Several attempts in point cloud segmentation have been made to capture spatial relationships in point clouds through more general convolution operations. SPLATNet and InterpConv perform convolution on high-dimensional lattices and 3D cubes interpolated from neighboring points, respectively. However, both of them are based on a strong assumption that the 3D coordinates of the output points are the same as the input points and thus can not be used for 3D point completion.

To address the issues mentioned above, we introduce 3D grids as intermediate representations to regularize unordered point clouds, which explicitly preserves the structural and context of point clouds. Consequently, we propose a novel Gridding Residual Network (GRNet) for point cloud completion, as shown in Figure 1. Besides 3D CNN and MLP, we devise three differentiable layers: Gridding, Gridding Reverse, and Cubic Feature Sampling. In Gridding, for each point of the point cloud, eight vertices of the 3D grid cell that the point lies in are first weighted using an interpolation function that explicitly measures the geometric relations of the point cloud. Then, a 3D convolutional neural network (3D CNN) with skip connections is adopted to learn context-aware and spatially-aware features, which allows the network to complete missing parts of the incomplete point cloud. Next, Gridding Reverse converts the output 3D grid to a coarse point cloud by replacing each 3D grid cell with a new point whose coordinate is the weighted sum of the eight vertices of the 3D grid cell. The following Cubic Feature Sampling extracts features for each point in the coarse point cloud by concatenating the features of the corresponding eight vertices of the 3D grid cell that the point lies in. The coarse point cloud and the features are forwarded to an MLP to obtain the final completed point cloud.

Existing methods adopt Chamfer Distance in PSGN as the loss function to train the neural networks. This loss function penalizes the prediction deviating from the ground-truth. However, there is no guarantee that the predicted point clouds follow the geometric layout of objects, and the networks tend to output a mean shape that minimizes the distance . Some recent works attempt to solve the unorderness while preserving fine-grained details by projecting the 3D point cloud to an image, which is then supervised by the corresponding ground truth masks. However, the projection requires extrinsic camera parameters, which are challenging to estimate in most scenarios . To solve the unorderedness of point clouds, we propose Gridding Loss, which calculates the L1 distance between the generated points and ground truth by representing them in regular 3D grids with the proposed Gridding layer.

The contributions can be summarized as follows:

We innovatively introduce 3D grids as intermediate representations to regularize unordered point clouds, which explicitly preserve the structural and context of point clouds.

We propose a novel Gridding Residual Network (GRNet) for point cloud completion. We design three differentiable layers: Gridding, Gridding Reverse, and Cubic Feature Sampling, as well as a new Gridding Loss.

Extensive experiments are conducted on the ShapeNet, Completion3D, and KITTI benchmarks, which indicate that the proposed GRNet performs favorably against state-of-the-art methods.

Related Work

According to the network architecture used in point cloud completion and reconstruction, existing networks can be roughly categorized into MLP-based, graph-based, and convolution-based networks.

MLP-based Networks. Pioneered by PointNet , several works use MLP for point cloud processing and reconstruction because of its simplicity and strong representation ability. These methods model each point independently using several Multi-layer Perceptrons and then aggregate a global feature using a symmetric function (e.g., Max Pooling). However, the geometric relationships among 3D points are not fully considered. PointNet++ and TopNet incorporate a hierarchical architecture to consider the geometric structure. To relief the structure loss caused by MLP, AtlasNet and MSN recover the complete point cloud of an object by estimating a collection of parametric surface elements.

Graph-based Networks. By considering each point in a point cloud as a vertex of a graph, graph-based networks generate directed edges for the graph based on the neighbors of each point. In these methods, convolution is usually operated on spatial neighbors, and pooling is used to produce a new coarse graph by aggregating information from each point’s neighbors. Compared with MLP-based methods, graph-based networks take local geometric structures into account. In DGCNN , a graph is constructed in the feature space and dynamically updated after each layer of the network. Further, LDGCNN removes the transformation network and link the hierarchical features from different layers in DGCNN to improve its performance and reduce the model size. Inspired by DGCNN, Hassani and Haley introduce the multi-scale graph-based network to learn point and shape features for self-supervised classification and reconstruction. DCG also follows DGCNN to encode additional local connection into a feature vector and progressively evolves from coarse to fine point clouds.

Convolution-based Networks. Early works usually apply 3D convolutional neural networks (CNNs) build upon the volumetric representation of 3D point clouds. However, converting point clouds into 3D volumes introduces a quantization effect that discards some details of the data and is not suitable for representing fine-grained information. To the best of our knowledge, no work directly applies CNNs on irregular point clouds for shape completion. In point cloud understanding, several works develop CNNs operating on discrete 3D grids that are transformed from point clouds. Hua et al. define convolutional kernels on regular 3D grids, where the points are assigned with the same weights when falling into the same grid. PointCNN achieves permutation invariance through a $\chi$-conv transformation. Besides CNNs on discrete space, several methods define convolutional kernels on continuous space. Thomas et al. propose both rigid and deformable kernel point convolution (KPConv) operators for 3D point clouds using a set of learnable kernel points. Compared with graph-based networks, convolution-based networks are more efficient and robust to point cloud density .

Gridding Residual Network

The proposed GRNet aims to recover the complete point cloud from an incomplete one in a coarse-to-fine fashion. It consists of five components, including Gridding (Section 3.2), 3D Convolutional Neural Network (Section 3.3), Gridding Reverse (Section 3.4), Cubic Feature Sampling (Section 3.5), and Multi-layer Perceptron (Section 3.6), as shown in Figure 1. Given an incomplete point cloud $P$ as input, Gridding is first used to obtain a 3D grid $\mathcal{G}=<V,W>$, where $V$ and $W$ are the vertex set and value set of $\mathcal{G}$, respectively. Then, $W$ is fed to a 3D CNN, whose output is $W^{\prime}$. Next, Gridding Reverse produces a coarse point cloud $P^{c}$ from the 3D grid $\mathcal{G}^{\prime}=<V,W^{\prime}>$. Subsequently, Cubic Feature Sampling generates features $F^{c}$ for the coarse point cloud $P^{c}$. Finally, MLP takes the coarse point cloud $P^{c}$ and the corresponding features $F^{c}$ as input to produce the final completed point cloud $P^{f}$.

2 Gridding

2D and 3D convolutions have been developed to process regularly arranged data such as images and voxel grids. However, it is challenging to directly apply standard 2D and 3D convolutions to unordered and irregular point clouds. Several methods convert point clouds into 3D voxels and then apply 3D convolutions to them. However, the voxelization process leads to an irreversible loss of geometric information. Recent methods adopt Multi-layer Perceptrons (MLPs) to directly operate on point clouds and aggregate information across points with max pooling. However, MLP-based methods may lose local context information because the connectivity and layouts of points are not fully considered. Recent studies also indicate that simply applying MLPs to point clouds cannot always work in practice .

However, this voxelization process introduces a quantization effect that discards some details of an object. In addition, voxelization is not differentiable and thus can not be applied to point cloud reconstruction. As illustrated in Figure 1 (b), given a vertex $v_{i}$ and its neighboring points $p\in\mathcal{N}(v_{i})$, the proposed Gridding layer computes the corresponding value $w_{i}$ of this vertex $v_{i}$ as

where $|\mathcal{N}(v_{i})|$ is the number of neighboring points of $v_{i}$. Specially, we define $w_{i}=0$ if $|\mathcal{N}(v_{i})|=0$. The interpolation function $w(v_{i},p)$ is defined as

3 3D Convolutional Neural Network

The 3D Convolutional Neural Network (3D CNN) with skip connections aims to complete the missing parts of the incomplete point cloud. It follows the idea of a 3D encoder-decoder with U-net connections . Given $W$ as input, the 3D CNN can be formulated as

As shown in Figure 2, the encoder of the 3D CNN has four 3D convolutional layers, each of which has a bank of $4^{3}$ filters with padding of 2, followed by batch normalization, leaky ReLU activation, and a max pooling layer with a kernel size of $2^{3}$. The numbers of output channels of convolutional layers are $32$, $64$, $128$, $256$, respectively. The encoder is finally followed by two fully connected layers with dimensions of $2048$ and $16384$. The decoder consists of four transposed convolutional layers, each of which has a bank of $4^{3}$ filters with padding of $2$ and stride of $1$, followed by a batch normalization layer and a ReLU activation.

4 Gridding Reverse

Specially, we ignore the point $p_{i}^{c}$ for this cell if $\sum_{\theta\in\Theta^{i}}w_{\theta}^{\prime}=0$.

5 Cubic Feature Sampling

MLP-based methods (e.g., PCN) are unable to take the context of neighboring points into account due to no local spatial connectivity across points. These methods use max-pooling to aggregate information globally, which may lose local context information.

where $[\cdot]$ is the concatenation operation. $\{f_{\theta_{j}^{i}}^{v}\}_{j=1}^{8}$ denotes the features of eight vertices of the $i$-th 3D gird cell where $p_{i}^{c}$ lies in.

In GRNet, Cubic Feature Sampling extracts the point features from feature maps generated by the first three transposed convolutional layers in 3D CNN. To reduce the redundancy of these features and generate a fixed number of points, we randomly sample $2,048$ points from the coarse point cloud $P^{c}$. Consequently, it produces a feature map of size $2048\times 1792$.

6 Multi-layer Perceptron

The Multi-layer Perceptron (MLP) is used to recover the details from the coarse point cloud by learning residual offsets between the coordinates of points in the coarse and final completed point cloud. It takes the coarse point cloud $P^{c}$ and the corresponding features $F^{c}$ as input, and outputs the final completed point cloud $P^{f}=\{p_{i}^{f}\}_{i=1}^{k}$ as

In GRNet, $r$ is set to $8$. The MLP consists of four fully connected layers with dimensions of $1792$, $448$, $112$, and $24$, respectively. The output of MLP is reshaped to $16384\times 3$, which corresponds to the offsets of the coordinates of $16,384$ points.

7 Gridding Loss

Existing methods adopt Chamfer Distance as the loss function to train the neural networks. This loss function penalizes the prediction deviating from the ground-truth. However, it can not guarantee that the predicted points follow the geometric layout of the object. Therefore the networks tend to output a mean shape that minimizes the distance, which causes the loss of the object’s details .

Experiments

ShapeNet. The ShapeNet dataset for point cloud completion is derived from PCN , which consists of 30,974 3D models from 8 categories. The ground truth point clouds containing 16,384 points are uniformly sampled on mesh surfaces. The partial point clouds are generated by back-projecting 2.5D depth maps into 3D. For a fair comparison, we use the same train/val/test splits as PCN.

Completion3D. The Completion3D benchmark is composed of 28,974 and 800 samples for training and validation, respectively. Different from the ShapeNet dataset generated by PCN, there are only 2,048 points in the ground truth point clouds.

KITTI. The KITTI dataset is composed of a sequence of real-world Velodyne LiDAR scans, also derived from PCN . For each frame, the car objects are extracted according to the 3D bounding boxes, which results in 2,401 partial point clouds. The partial point clouds in KITTI are highly sparse and do not have complete point clouds as ground truth.

2 Evaluation Metrics

Let $\mathcal{T}=\{(x_{i},y_{i},z_{i})\}_{i=1}^{n_{\mathcal{T}}}$ be the ground truth and $\mathcal{R}=\{(x_{i},y_{i},z_{i})\}_{i=1}^{n_{\mathcal{R}}}$ be a reconstructed point set being evaluated, where $n_{\mathcal{T}}$ and $n_{\mathcal{R}}$ are the numbers of points of $\mathcal{T}$ and $\mathcal{R}$, respectively. In our experiments, we use both Chamfer Distance and F-Score as quantitative evaluation metrics.

Chamfer Distance. Follow PSGN and TopNet , the distance between $\mathcal{T}$ and $\mathcal{R}$ are defined as

F-Score. As pointed out in , Chamfer Distance may sometimes be misleading. As suggested in , we take F-Score as an extra metric to evaluate the performance of point completion results, which can be defined as following

where $P(d)$ and $R(d)$ denote the precision and recall for a distance threshold $d$, respectively.

3 Implementation Details

We implement our network using PyTorch and CUDAThe source code is available at https://github.com/hzxie/GRNet.. All models are optimized with an Adam optimizer with $\beta_{1}=0.9$ and $\beta_{2}=0.999$. We train the network with a batch size of $32$ on two NVIDIA TITAN Xp GPUs. The initial learning rate is set to $1e-4$ and decayed by 2 after 50 epochs. The optimization is set to stop after 150 epochs.

4 Shape Completion on ShapeNet

To compare the performance of GRNet with other state-of-the-art methods, we conduct experiments on the ShapeNet dataset. AtlasNet generates a point cloud with a set of parametric surface elements. To compare with other methods fairly, we sample 16,384 points from the generated primitive surface elements. PCN completes the partial point cloud with a stacked version of PointNet , which directly outputs the coordinates of 16,384 points. FoldingNet is a baseline method adopted in PCN , which deforms a $128\times 128$ 2D grid into 3D point cloud. TopNet incorporates a decoder following a hierarchical rooted tree structure to consider the topology of point clouds. Due to the scalable architecture of TopNet, it can easily generate 16,384 points by setting the number of nodes and the size of feature embedding. A very recent method MSN generates dense point cloud containing 8,192 points in a coarse-to-fine fashion. To generate 16,384 points, we combine the generated points of 2 times forward propagation.

Quantitative results in Tables 2 and 1 indicate that GRNet outperforms all competitive methods in terms of Chamfer Distance and F-Score@1%. Figure 3 shows the qualitative results for point completion on ShapeNet, which indicates that the proposed method recovers better details of objects (e.g., chairs and lamps) than the other methods.

5 Shape Completion on Completion3D

Using the model with the lowest Chamfer Distance (CD) on the validation set, we recover the complete point clouds for 1,184 objects in the Completion3D testing set. Then, random subsampling is applied to the generated point clouds to obtain 2,048 points for benchmark evaluation. According to the online leaderboard https://completion3d.stanford.edu/results, as shown in Table 3, the overall CD for the proposed GRNet is $10.64$, which remarkably outperforms state-of-the-art methods and ranks first on this benchmark.

6 Shape Completion on KITTI

To evaluate the performance of the proposed method on real-world LiDAR scans, we test GRNet on the KITTI dataset for completing sparse point clouds of cars. Unlike ShapeNet generated by back-projected from 2.5D images, point clouds from LiDAR scans can be highly sparse, which are much sparser than those in ShapeNet.

We fine-tuned all competitive methods on ShapeNetCars (the cars from ShapeNet) except PCN that directly uses released output for evaluation. During testing, each point cloud is transformed into the bounding box’s coordinates and transformed back to the world frame after completion. The models trained specifically on cars are able to incorporate prior knowledge of the object class.

Since there are no complete ground truth point clouds for KITTI, we use Consistency and Uniformity to evaluate the performance of all competitive methods. Consistency in PCN is the average CD between the output of the same car instance in $n_{f}$ consecutive frames. Let $\mathcal{R}_{t_{i}}^{j}$ be the output for the $j$-th car instance at time $t_{i}$. The Consistency for the $j$-th car can be calculated as

Following PU-GAN , we adopt Uniformity to evaluate the distribution uniformity of the completed point clouds, which can be formulated as

where $S_{i}(i=1,2,\dots,M)$ is a point subset cropped from a patch of the output $\mathcal{R}$ using the farthest sampling and ball query of radius $\sqrt{p}$. The term $\rm U_{imbalance}$ and $\rm U_{clutter}$ account for the global and local distribution uniformity, respectively.

where $\hat{n}=p|\mathcal{R}|$ is the expected number of points in $S_{i}$.

where $d_{i,j}$ represents the distance to the nearest neighbor for the $j$-th point in $S_{i}$, and $\hat{d}$ is roughly $\sqrt{\frac{2\pi p}{|S_{i}|\sqrt{3}}}$ if $S_{i}$ has a uniform distribution .

Table 9 shows the completion results for cars in the LiDAR scans from the KITTI dataset. Experimental results indicate that GRNet outperforms other competitive methods in terms of Consistency and Uniformity. Benefited from Gridding and Gridding Reverse, GRNet is more sensitive to the spatial structure of the input points, which leads to better consistency between the two consecutive frames. As shown in Figure 4, the cars are barely recognizable due to incompleteness of the input data. In contrast, the completed point clouds provide more geometric information. In addition, the qualitative results also demonstrate the proposed method generates more reasonable shape completion.

7 Ablation Study

The performance improvement of GRNet should be attributed to three key components, including Gridding, Cubic Feature Sampling, and Gridding Loss. To demonstrate the effectiveness of each component in the proposed method, we evaluate the performance with different parameters.

Gridding. Table 5 shows the results of different resolutions of 3D grids generated by Gridding. The F-Score of final completed point clouds increases with the 3D grids’ resolutions. However, the numbers of parameters and the backward time also increases. To archive a balance between effect and efficiency, we choose the resolution of size $64^{3}$ for Gridding in GRNet.

Cubic Feature Sampling. To quantitatively evaluate the effect of Cubic Feature Sampling, we compare the performance without Cubic Feature Sampling and with different feature maps fed into it. The experimental results presented in Table 6 indicate that Cubic Feature Sampling improves the point cloud completion results significantly. In addition, with more feature maps are fed, the completion quality becomes better without a significant increase in the numbers of parameters and backward time.

Gridding Loss. We further validate the effects of Gridding Loss, as shown in Table 7. There is a decrease in terms of both CD and F-Score when removing Gridding Loss. When increasing the resolution of 3D grids from $64^{3}$ to $128^{3}$, there are $25.9\%$ and $5.4\%$ improvements in CD and F-Score, respectively.

Conclusion

In this paper, we study how to recover the complete 3D point cloud from an incomplete one. The main motivation of this work is to enable the convolutions on 3D point clouds while preserving their structural and context information. To this aim, we introduce 3D grids as intermediate representations to regularize unordered point clouds. We then propose a novel Gridding Residual Network (GRNet) for point cloud completion, which contains three novel differentiable layers: Gridding, Gridding Reverse, and Cubic Feature Sampling, as well as a new Gridding Loss. Extensive comparisons are conducted on the ShapeNet, Completion3D, and KITTI benchmarks, which indicate that the proposed GRNet performs favorably against state-of-the-art methods.

Acknowledgements. This work is supported by the National Natural Science Foundation of China (Nos. 61772158, 61702136 and 61872112), National Key Research and Development Program of China (Nos. 2018YFC0806802 and 2018YFC0832105), and Self-Planned Task (No. SKLRS202002D) of State Key Laboratory of Robotics and System (HIT).

References

Appendix 0.A More Explanations on Gridding, Gridding Reverse, and Cubic Feature Sampling

According to the manuscript, given a vertex $v_{i}$ and its neighboring points $p\in\mathcal{N}(v_{i})$. The proposed Gridding layer computes the corresponding value $w_{i}$ of this vertex $v_{i}$ as

where $|\mathcal{N}(v_{i})|$ is the number of neighboring points of $v_{i}$ and $w(v_{i},p)$ is defined as

Based on Equations 17 and 18, the partial derivative with respect to $x$ can be calculated as follows

where $x$ and $x_{i}^{v}$ are the x-coordinates of the point $p$ and vertex $v_{i}$, respectively. Similarly, the partial derivative with respect to $y$ and $z$ can be calculated as follows

where $y$ and $y_{i}^{v}$ are the y-coordinates of the point $p$ and vertex $v_{i}$, respectively. $z$ and $z_{i}^{v}$ are the z-coordinates of the point $p$ and vertex $v_{i}$, respectively.

A.2 Gridding Reverse

Point Coordinates Normalization. Gridding Reverse generates point $p^{c}_{i}=(x_{i}^{c},y_{i}^{c},z_{i}^{c})$ for the $i$-th grid cell by a weighted combination of eight vertices $\{v_{\theta}|\theta\in\Theta^{i}\}$ and the corresponding values $\{w^{\prime}_{\theta}|\theta\in\Theta^{i}\}$ in this cell, which is calculated as

where $\sum_{\theta\in\Theta^{i}}w_{\theta}^{\prime}\neq 0$ and $\Theta^{i}=\{\theta^{i}_{j}\}_{j=1}^{8}$ represents the index set of vertices of this 3D grid cell. Let $(x_{\theta}^{v},y_{\theta}^{v},z_{\theta}^{v})$ be the coordinate of the vertex $v_{\theta}$, where $x_{\theta}^{v},y_{\theta}^{v},z_{\theta}^{v}\in\{-\frac{N}{2},-\frac{N}{2}+1,\dots,-\frac{N}{2}-1\}$ and $N$ is the resolution of the 3D grid. The x-, y-, and z- coordinates of $p_{i}^{c}$ is calculated as

Since the coordinate $(x_{i}^{gt},y_{i}^{gt},z_{i}^{gt})$ of the point in the ground truth point cloud satisfies $-1<x_{i}^{gt},y_{i}^{gt},z_{i}^{gt}<1$. The coordinates of the point $p_{i}^{c}$ are normalized to $(-1,1)$ by dividing $-\frac{N}{2}$.

Backward of Gridding Reverse. The partial derivative with respect to $w_{\theta}^{\prime}$ can be calculated as

A.3 Cubic Feature Sampling

where $\{f_{\theta_{j}^{i}}^{v}\}_{j=1}^{8}$ denotes the features of eight vertices of the $i$-th 3D gird cell where $p_{i}^{c}$ lies in. Specifically, the coordinates of the eight vertices $\{(x_{\theta^{i}_{j}}^{v},y_{\theta^{i}_{j}}^{v},z_{\theta^{i}_{j}}^{v})\}_{j=1}^{8}$ satisfy $x_{\theta^{i}_{j}}^{v}\in\{\lfloor\frac{t}{2}x_{i}^{c}\rfloor,\lceil\frac{t}{2}x_{i}^{c}\rceil\}$, $y_{\theta^{i}_{j}}^{v}\in\{\lfloor\frac{t}{2}y_{i}^{c}\rfloor,\lceil\frac{t}{2}y_{i}^{c}\rceil\}$, and $z_{\theta^{i}_{j}}^{v}\in\{\lfloor\frac{t}{2}z_{i}^{c}\rfloor,\lceil\frac{t}{2}z_{i}^{c}\rceil\}$, respectively.

Backward of Cubic Feature Sampling. During backward propagation, the partial derivative with respect to $f_{\theta_{j}^{i}}^{v}$ can be presented as

where $j\in\{1,2,\dots,8\}$ and $f_{i,j}^{c}$ denotes the $j$-th element in $f_{i}^{c}$.

Since $\lfloor\cdot\rfloor$ and $\lceil\cdot\rceil$ is not differentiable, the partial derivatives with respect to $x_{i}^{c}$, $y_{i}^{c}$, and $z_{i}^{c}$ are , which can be formulated as follows:

Appendix 0.B Additional Quantitative Results on ShapeNet

According to the manuscript, the Chamfer Distance is with L2 norm. However, PCN adopts the Chamfer Distance with L1 norm as an evaluation metric, which can be formulated as follows

where $\mathcal{T}=\{(x_{i},y_{i},z_{i})\}_{i=1}^{n_{\mathcal{T}}}$ is the ground truth and $\mathcal{R}=\{(x_{i},y_{i},z_{i})\}_{i=1}^{n_{\mathcal{R}}}$ is the reconstructed point set being evaluated. $n_{\mathcal{T}}$ and $n_{\mathcal{R}}$ are the numbers of points of $\mathcal{T}$ and $\mathcal{R}$, respectively.

Table 8 shows the results of point cloud completion using the Chamfer Distance calculated with Equation 34. The values of PCN are exactly the same as Table 4 in the original paper https://arxiv.org/pdf/1808.00671.

Appendix 0.C Quantitative Results on Completion3D

Figure 5 is the screenshot of the leaderboard results on the Completion3D benchmark, which is available online at https://completion3d.stanford.edu/results.

Appendix 0.D Additional Quantitative Results on KITTI

PCN uses the Fidelity Distance (FD) and Minimal Matching Distance (MMD) as evaluation metrics for KITTI. FD is the average distance from each point in the input to its nearest neighbor in the output, which can be defined as follows

where $\mathcal{I}$ denotes the input point cloud. MMD is the Chamfer Distance (CD) between the output and the car point cloud from ShapeNet that is the closest to the output point cloud in terms of CD. The Fidelity and MMD on KITTI of the compared methods are shown in Table 9.

However, both FD and MMD are not suitable metrics for KITTI. As shown in Figure 6, real-world LiDAR scans usually contain clutters which should be removed in the recovered point cloud. MSN incorporates the minimum density sampling (MDS) to preserve the structure of the input point cloud. Although MSN outperforms other methods in terms of FD, the clutters in the input point cloud are also preserved. MMD s measures how much the output resembles the cars in ShapeNet. However, cars from ShapeNet cannot cover all types of cars in the real-world.

Appendix 0.E Additional Ablation Studies

Number of Sampling Points. Gridding Reverse generates a coarse point cloud from a 3D grid. We randomly sample 2,048 points from the coarse point cloud to generate a point cloud containing a fixed number of points for the following MLP. Table 10 shows the Chamfer Distance (CD) and F-Score@1% with different numbers of points sampled.

Experimental results indicate that sampling 2,048 points from the coarse point clouds archives the best performance in terms of CD and F-Score. The coarse point cloud of an object usually contains about 3,000-4,000 points, oversampling 4,096 points from the coarse point cloud leads to redundant information in the sampled point cloud. Sampling 1,024 points from the coarse point cloud may lose too much information for the subsequent processing.

Appendix 0.F Qualitative Comparisons

In this section, we provide more visual comparisons with the state-of-the-art methods for point cloud completion on ShapeNet .