Abstract

MPEG Geometry-based Point Cloud Compression (G-PCC) standard is developed for lossy encoding of point clouds to enable immersive services over the Internet. However, lossy G-PCC introduces superimposed distortions from both geometry and attribute information, seriously deteriorating the Quality of Experience (QoE). This paper thus proposes the Enhanced G-PCC (G-PCC++), to effectively address the compression distortion and restore the quality. G-PCC++ separates the enhancement into two stages: it first enhances the geometry and then maps the decoded attribute to the enhanced geometry for refinement. As for geometry restoration, a Nearest Neighbors (𝑘NN)-based Linear Interpolation is first used to generate a denser geometry representation, on top of which the GeoNet further generates sufficient candidates to restore geometry through probability-sorted selection. For attribute enhancement, a 𝑘NN-based Gaussian Distance Weighted Mapping is devised to re-colorize all points in enhanced geometry tensor, which are then refined by the AttNet for the final reconstruction. G-PCC++ is the first solution addressing the geometry and attribute artifacts together. Extensive experiments on several public datasets demonstrate the superiority of G-PCC++, e.g., on the solid point cloud dataset 8iVFB, G-PCC++ outperforms G-PCC by 88.24% (80.54%) BD-BR in D1 (D2) measurement of geometry and by 14.64% (13.09%) BD-BR in Y (YUV) attribute information. Moreover, when considering both geometry and attribute, G-PCC++ also largely surpasses the G-PCC by 25.58% BD-BR using PCQM assessment.

Overview

G-PCC++: Using Local Neighbors For Quality Enhancement

G-PCC++ separately enhances the quality of G-PCC compressed geometry and attribute. It first restores the geometry and upon restored geometry representation, attribute restoration is fulfilled. The use of localized computation, e.g., sparse convolutions (SConv), 𝑘NN-based Linear Interpolation, and 𝑘NN-based Gaussian Distance Weighted Mapping, makes G-PCC++ very lightweight and attractive in practice. 𝑑 denotes the number of input channels in InceptionResNet (IRN).

Experiment Results

All results confirm the superiority of the proposed G-PCC++. R-D curves plotted in figure consistently evidence the leading performance of G-PCC++ across a variety of bitrates.

Rate-Distortion (R-D) comparison between G-PCC (TMC13v19) and G-PCC++ (Ours) when compressing geometry, attribute, and their superimposition..

More Rate-Distortion comparison between G-PCC using PCQM quality assessment.

More Quality Comparison

In addition, we visualize reconstructed point clouds. The reconstructions of G-PCC compressed point clouds contain visually-annoying black holes and blocking artifacts, particularly at low bitrates, which are introduced by the point vanishing in geometry compression and quantization error accumulation in attribute compression. Overall, G-PCC reconstructions lack spatial details with blurry and blocky appearances across a variety of content. By contrast, G-PCC++ effectively addresses these issues by filling the holes and removing the artifacts, resulting in visually clear and aesthetically pleasant reconstructions that are close to the ground truth.

Visualization of G-PCC and G-PCC++ reconstructions. For fair comparisons, we show different point clouds compressed at a wide range of bitrates.

Citation

Paper accepted in ACM MultiMedia 2023. Citation available soon.
If GPCC++ is useful for your research, please consider citing.