OctMamba: Mamba-based octree context entropy model for point cloud geometry compression

In this paper, we investigate two fundamental problems of inter-prediction for Light Detection and Ranging (LiDAR) point cloud geometries: motion estimation (ME) and coding structure under the inter-exploration model of geometry-based point cloud compression (G-PCC). Under the inter-exploration model of G-PCC, the key to a good ME algorithm is to design an accurate criterion for estimating the bit cost of an octree node. In the previous work, a logarithmic relationship between the prediction distortion and the bit cost was used as the criterion. We first note that the multiscale binary prediction residue, instead of the prediction distortion, is the key factor in determining the bit cost. Then, a linear relationship between the number of 1s and 0s in the multiscale binary prediction residue and the bit cost is built and used as the ME criterion. In terms of the coding structure, only the IPPP coding structure is investigated in all previous geometry inter-prediction algorithms. The use of the hierarchical coding structure is first investigated in this paper. We further propose determining the use of the IPPP or hierarchical coding structure at the group of pictures (GoP)-level based on rate distortion optimization to improve the performance. The proposed algorithms are implemented in the inter-exploration model of G-PCC. The experimental results show that compared with the inter-exploration model of G-PCC, the proposed algorithms can provide an average of 2.1% bitrate savings.

The geometry of a point cloud is commonly represented by an octree recursively decomposing a 3D volume into eight child sub-volumes. Said volumes and sub-volumes are associated with nodes and child-nodes of the octree. The geometry is defined by the occupancy information indicating the presence or not of a point in each of the sub-volumes. This naturally leads to an eight-bit occupancy information to be coded for each internal node of the tree. This paper introduces a new binarization scheme to efficiently compress the occupancy information using an optimal set of binary entropy coders. Then, it is shown how using the occupancy information of neighbouring nodes helps to compress the occupancy bits associated with the child nodes of the current node. This information is used to contextualise the binarization scheme by computing, firstly a neighbour configuration, secondly a number of neighbours with occupied child nodes adjacent to the current child node, and thirdly an intra predictor. Objective results show lossless geometry compression gains between 60% and 75% on virtual reality oriented dense point clouds used by MPEG, reaching sub-bit per point bit-rates for the lossless intra coding of such point clouds. Solid gains (between 5% and 25% depending upon the sampling) are also observed on sparse point clouds captured by a LiDAR (Light Detection and Ranging) device attached to a moving vehicle or representing 3D maps.

The increase in popularity of point-cloud-oriented applications has triggered the development of specialized compression algorithms. In this paper, a novel algorithm is developed for the lossless geometry compression of voxelized point clouds following an intra-frame design. The encoded voxels are arranged into runs and are encoded through a single-pass application directly on the voxel domain. This is done without representing the point cloud via an octree nor rendering the voxel space through an occupancy matrix, therefore decreasing the memory requirements of the method. Each run is compressed using a context-adaptive arithmetic encoder yielding state-of-the-art compression results, with gains of up to 15% over TMC13, MPEG's standard for point cloud geometry compression. Several proposed contributions accelerate the calculations of each run's probability limits prior to arithmetic encoding. As a result, the encoder attains a low computational complexity described by a linear relation to the number of occupied voxels leading to an average speedup of 1.8 over TMC13 in encoding speeds. Various experiments are conducted assessing the proposed algorithm's state-of-the-art performance in terms of compression ratio and encoding speeds.

Point cloud data have been put under the spotlight by many applications that play an increasingly important role in our every day lives. Their large size and ever-growing prevalent use cases have raised the interest in specialized compression algorithms for point cloud data. In this paper we propose a lossless intra-frame encoder for point cloud geometry. It relies on a single projection of the entire point cloud on a predetermined plane, combined with a context-adaptive binary arithmetic encoder. Our approach simplifies the current best performing approach for intra-frame compression. The experimental results indicate that our proposal not only improves the performance of all other intra-frame approaches, but it even surpasses the performance of state-of-the-art inter-frame approaches. Furthermore, we suggest to replace the adaptive encoder with a semi-adaptive approach for further performance gains.

Abstract. Training LiDAR point clouds object detectors requires a significant amount of annotated data, which is time-consuming and effort-demanding. Although weakly supervised 3D LiDAR-based methods have been proposed to reduce the annotation cost, their performance could be further improved. In this work, we propose a weakly supervised LiDAR-based point clouds vehicle detector that does not require any labels for the proposal generation stage and needs only a few labels for the refinement stage. It comprises two primary modules. The first is an unsupervised proposal generation module based on the geometry of point clouds. The second is the pseudo-label refinement module. We validate our method on two point clouds based object detection datasets, namely KITTI and ONCE, and compare it with various existing weakly supervised point clouds object detection methods. The experimental results demonstrate the method’s effectiveness with a small amount of labeled LiDAR point clouds.

Point cloud geometry (PCG) is used to precisely represent arbitrary-shaped 3D objects and scenes, is of great interest to vast applications which puts forward the pressing desire of high-efficiency PCG compression for transmission and storage. Existing PCG coding mostly relies on the octree model by which point-wise processing is applied without exploring nonlocal regional geometry similarity across the entire 3D surface. This work, instead, suggests the region-wise processing to leverage the region similarity to exploit inter-region redundancy for efficient lossy point cloud geometry compression. Towards this goal, a given PCG is first segmented into numerous local regions each of which comprises a portion of point cloud surface, and can be represented by a surface vector that describes the geometry shape numerically in a projected principal space. Subsequently, these regions are grouped into several discriminative clusters, assuring that inter-cluster similarity is minimized and intra-cluster similarity is maximized simultaneously, where the similarity is calculated using the regional surface vectors. In each cluster, we set a reference region having the largest similarity score to the others, which enables the non-reference region prediction from the reference one using alignment transform. In the end, we encode the reference regions directly using the lossless mode of the Geometry-based Point Cloud Compression (G-PCC), while corresponding non-reference regions are signaled using associated transform parameters. Compared with the state-of-the-art G-PCC using octree model, our region-wise approach can offer remarkable coding efficiency improvement, e.g., 32.4% and 22.0% Bjontegaard-delta rate (BD-Rate) gains for respective point-to-point ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$D1$ </tex-math></inline-formula> ) and point-to-plane ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$D2$ </tex-math></inline-formula> ) distortion evaluations, across a variety of common test sequences used in standard committee.

This paper presents a novel end-to-end Learned Point Cloud Geometry\nCompression (a.k.a., Learned-PCGC) framework, to efficiently compress the point\ncloud geometry (PCG) using deep neural networks (DNN) based variational\nautoencoders (VAE). In our approach, PCG is first voxelized, scaled and\npartitioned into non-overlapped 3D cubes, which is then fed into stacked 3D\nconvolutions for compact latent feature and hyperprior generation. Hyperpriors\nare used to improve the conditional probability modeling of latent features. A\nweighted binary cross-entropy (WBCE) loss is applied in training while an\nadaptive thresholding is used in inference to remove unnecessary voxels and\nreduce the distortion. Objectively, our method exceeds the geometry-based point\ncloud compression (G-PCC) algorithm standardized by well-known Moving Picture\nExperts Group (MPEG) with a significant performance margin, e.g., at least 60%\nBD-Rate (Bjontegaard Delta Rate) gains, using common test datasets.\nSubjectively, our method has presented better visual quality with smoother\nsurface reconstruction and appealing details, in comparison to all existing\nMPEG standard compliant PCC methods. Our method requires about 2.5MB parameters\nin total, which is a fairly small size for practical implementation, even on\nembedded platform. Additional ablation studies analyze a variety of aspects\n(e.g., cube size, kernels, etc) to explore the application potentials of our\nlearned-PCGC.\n

Point cloud compression is a pivotal technology for efficient storage and transmission of 3D point cloud data, which has significant implications for practical applications in virtual reality, autonomous driving, and cultural heritage preservation. In this paper, we propose a new learning-based model using the Retentive Network (RetNet) for point cloud compression, which achieves a lower bitrate while maintaining a high peak signal-to-noise ratio (PSNR). We first use an octree structure to segment the point cloud objects. Then, we use octree-based contextual windows to extract pivotal features from relevant sibling and ancestor nodes. Finally, we employ our proposed Octree-Retention model to effectively exploit the prior information between the spatially adjacent nodes for compression. The experimental results show that our method outperforms the state-of-the-art methods on both the LIDAR dataset(SemanticKITTI) and the object dataset(MPEG 8i), demonstrating its effectiveness.

Since the development of 3D applications, the point cloud, as a spatial description easily acquired by sensors, has been widely used in multiple areas such as SLAM and 3D reconstruction. Point Cloud Compression (PCC) has also attracted more attention as a primary step before point cloud transferring and saving, where the geometry compression is an important component of PCC to compress the points geometrical structures. However, existing non-learning-based geometry compression methods are often limited by manually pre-defined compression rules. Though learning-based compression methods can significantly improve the algorithm performances by learning compression rules from data, they still have some defects. Voxel-based compression networks introduce precision errors due to the voxelized operations, while point-based methods may have relatively weak robustness and are mainly designed for sparse point clouds. In this work, we propose a novel learning-based point cloud compression framework named 3D Point Cloud Geometry Quantiation Compression Network (3QNet), which overcomes the robustness limitation of existing point-based methods and can handle dense points. By learning a codebook including common structural features from simple and sparse shapes, 3QNet can efficiently deal with multiple kinds of point clouds. According to experiments on object models, indoor scenes, and outdoor scans, 3QNet can achieve better compression performances than many representative methods.

Point clouds have been gaining importance as a solution to the problem of efficient representation of 3D geometric and visual information. They are commonly represented by large amounts of data, and compression schemes are important for their manipulation transmission and storing. However, the selection of appropriate compression schemes requires effective quality evaluation. In this work a subjective quality evaluation of point clouds using a surface representation is analyzed. Using a set of point cloud data objects encoded with the popular octree pruning method with different qualities, a subjective evaluation was designed. The point cloud geometry was presented to observers in the form of a movie showing the 3D Poisson reconstructed surface without textural information with the point of view changing in time. Subjective evaluations were performed in three different laboratories. Scores obtained from each test were correlated and no statistical differences were observed. Scores were also correlated with previous subjective tests and a good correlation was obtained when compared with mesh rendering in 2D monitors. Moreover, the results were correlated with state of the art point cloud objective metrics revealing poor correlation. Likewise, the correlation with a subjective test using a different representation of the point cloud data also showed poor correlation. These results suggest the need for more reliable objective quality metrics and further studies on adequate point cloud data representations.

Efficient point cloud compression is essential for applications like virtual and mixed reality, autonomous driving, and cultural heritage. This paper proposes a deep learning-based inter-frame encoding scheme for dynamic point cloud geometry compression. We propose a lossy geometry compression scheme that predicts the latent representation of the current frame using the previous frame by employing a novel feature space inter-prediction network. The proposed network utilizes sparse convolutions with hierarchical multiscale 3D feature learning to encode the current frame using the previous frame. The proposed method introduces a novel predictor network for motion compensation in the feature domain to map the latent representation of the previous frame to the coordinates of the current frame to predict the current frame's feature embedding. The framework transmits the residual of the predicted features and the actual features by compressing them using a learned probabilistic factorized entropy model. At the receiver, the decoder hierarchically reconstructs the current frame by progressively rescaling the feature embedding. The proposed framework is compared to the state-of-the-art Video-based Point Cloud Compression (V-PCC) and Geometry-based Point Cloud Compression (G-PCC) schemes standardized by the Moving Picture Experts Group (MPEG). The proposed method achieves more than 88% BD-Rate (Bjøntegaard Delta Rate) reduction against G-PCCv20 Octree, more than 56% BD-Rate savings against G-PCCv20 Trisoup, more than 62% BD-Rate reduction against V-PCC intra-frame encoding mode, and more than 52% BD-Rate savings against V-PCC P-frame-based inter-frame encoding mode using HEVC. These significant performance gains are cross-checked and verified in the MPEG working group.

The emerging semantic compression has been receiving increasing research efforts most recently, capable of achieving high fidelity restoration during compression, even at extremely low bitrates. However, existing semantic compression methods typically combine standard pipelines with either pre-defined or high-dimensional semantics, thus suffering from deficiency in compression. To address this issue, we propose a novel hierarchical semantic compression (HSC) framework that purely operates within intrinsic semantic spaces from generative models, which is able to achieve efficient compression for consistent semantic restoration. More specifically, we first analyse the entropy models for the semantic compression, which motivates us to employ a hierarchical architecture based on a newly developed general inversion encoder. Then, we propose the feature compression network (FCN) and semantic compression network (SCN), such that the middle-level semantic feature and core semantics are hierarchically compressed to restore both accuracy and consistency of image semantics, via an entropy model progressively shared by channel-wise context. Experimental results demonstrate that the proposed HSC framework achieves the state-of-the-art performance on subjective quality and consistency for human vision, together with superior performances on machine vision tasks given compressed bitstreams. This essentially coincides with human visual system in understanding images, thus providing a new framework for future image/video compression paradigms. The source code and trained models are available at https://github.com/bblgbr/HSC-TIP2025.

Learned video compression has drawn great attention and shown promising compression performance recently. In this article, we focus on the two components in the learned video compression framework, the conditional entropy model and quality enhancement module, to improve compression performance. Specifically, we propose an adaptive spatial-temporal entropy model for image, motion, and residual compression, which introduces a temporal prior to reduce temporal redundancy of latents and an additional modulated mask to evaluate the similarity and perform refinement. In addition, a quality enhancement module is proposed for predicted frame and reconstructed frame to improve frame quality and reduce the bitrate cost of residual coding. The module reuses decoded optical flow as a motion prior and utilizes deformable convolution to mine high-quality information from the reference frame in a bit-free manner. The two proposed coding tools are integrated into a pixel-domain residual coding–based compression framework to evaluate their effectiveness. Experimental results demonstrate that our framework achieves competitive compression performance in the low-delay scenario compared with recent learning-based methods and traditional H.265/HEVC in terms of Peak Signal-to-Noise Ratio (PSNR) and Multi-Scale Structural Similarity Index (MS-SSIM). The code is available at OpenLVC.

We present a study on the impact of packet loss on dynamic 3D point cloud streaming, encoded with MPEG Video-based Point Cloud Compression (V-PCC) standard. We show the distortion when different channels of V-PCC bitstream are lost, with the loss of occupancy and geometry data impacting the quality most significantly. Our results point to the need for better error concealment techniques. We end the paper by presenting preliminary thoughts and experimental results of two naive error concealment techniques in the point cloud domain, for attributes and geometry data, respectively, and highlight the limitations of each.

Extension of a compressible lattice model to CO 2 + cosolvent + polymer systems