PointAcc: Efficient Point Cloud Accelerator

Abstract

Translate Article

Deep learning on point clouds plays a vital role in a wide range of\napplications such as autonomous driving and AR/VR. These applications interact\nwith people in real-time on edge devices and thus require low latency and low\nenergy. Compared to projecting the point cloud to 2D space, directly processing\nthe 3D point cloud yields higher accuracy and lower #MACs. However, the\nextremely sparse nature of point cloud poses challenges to hardware\nacceleration. For example, we need to explicitly determine the nonzero outputs\nand search for the nonzero neighbors (mapping operation), which is unsupported\nin existing accelerators. Furthermore, explicit gather and scatter of sparse\nfeatures are required, resulting in large data movement overhead. In this\npaper, we comprehensively analyze the performance bottleneck of modern point\ncloud networks on CPU/GPU/TPU. To address the challenges, we then present\nPointAcc, a novel point cloud deep learning accelerator. PointAcc maps diverse\nmapping operations onto one versatile ranking-based kernel, streams the sparse\ncomputation with configurable caching, and temporally fuses consecutive dense\nlayers to reduce the memory footprint. Evaluated on 8 point cloud models across\n4 applications, PointAcc achieves 3.7X speedup and 22X energy savings over RTX\n2080Ti GPU. Co-designed with light-weight neural networks, PointAcc rivals the\nprior accelerator Mesorasi by 100X speedup with 9.1% higher accuracy running\nsegmentation on the S3DIS dataset. PointAcc paves the way for efficient point\ncloud recognition.\n