Deep Hybrid Compression Network for Lidar Point Cloud Classification and Segmentation

Network quantization, which strives to reduce the precision of model parameters and/or features, is one of the most efficient ways to accelerate model inference and reduce memory consumption, particularly for deep models when performing a variety of real-time vision tasks on edge platforms with constrained resources. Existing quantization approaches function well when using relatively high bit widths but suffer from a decline in accuracy at ultra-low precision. In this paper, we propose a bit-weight adjustment (BWA) module to bridge uniform and non-uniform quantization, successfully quantizing the model to ultra-low bit widths without bringing about noticeable performance degradation. Given uniformly quantized data, the BWA module adaptively transforms these data into non-uniformly quantized data by simply introducing trainable scaling factors. With the BWA module, we combine uniform and non-uniform quantization in a single network, allowing low-precision networks to benefit from both the hardware friendliness of uniform quantization and the high performance of non-uniform quantization. We optimize the proposed BWA module by directly minimizing the classification loss through end-to-end training. Numerous experiments on the ImageNet and CIFAR-10 datasets reveal that the proposed approach outperforms state-of-the-art approaches across various bit-width settings and can even produce low-precision quantized models that are competitive with their full-precision counterparts.

Energy-efficient processors are crucial for accelerating deep neural networks (DNNs) on edge devices with limited battery capacity. To reduce energy consumption, time-domain computing-in-memory (TD-CIM) is a splendid architecture, which consumes low computation and memory access energy due to low toggle rate of time-based signals and less data movements, respectively. When deploying DNNs in TD-CIMs, quantization is required, which has two types: uniform quantization (UQ) and nonuniform quantization (NUQ). To reach the same accuracy for one DNN, NUQ achieves smaller model size than UQ. Due to varying weight distributions across layers, mixed-precision quantization can further reduce model size, without degrading accuracy. However, previous TD-CIMs are inefficient for mixed-precision NUQ-DNNs due to their adopted bit-serial convolution increasing computation amount significantly. To address that, we propose a unique-weight convolution to accelerate mixed-precision NUQ-DNNs by a special kernel decomposition, reducing computation count remarkably. Based on that, we design a TD-CIM-based processor, TIMAQ, with three architectural techniques: 1) bit-cross-flipping-based kernel decomposer to reduce memory accesses and operations of decomposing kernels; 2) dual-mode-complementary predictor to remove redundant computations; and 3) activation-weight-adaptive pulse quantizer to decrease pulse quantization energy and error. Fabricated in 28-nm CMOS technology and tested on 1–8-b NUQ-DNNs, TIMAQ achieves 2.4–152.7-TOPS/W peak energy efficiency.

Hessian-based mixed-precision quantization with transition aware training for neural networks.

Edge servers and mobile devices are often assigned a large number of computing tasks. However, the data involved in computing tasks is often sensitive in terms of privacy. Our initial proposal is a federated edge learning strategy based on real-world scenarios, which combines blurred data or pseudo shared data. Federated learning is used to train device models with the aim of protecting privacy while enabling mobile devices to more effectively utilize data for decision-making. In the case of limited energy on mobile devices, we propose a federated edge learning algorithm with blurred data sharing. This algorithm can generate more accurate models by uploading partially blurred data. In order to further improve model accuracy and protect privacy of mobile devices, we propose a federated edge learning algorithm with pseudo data sharing based on dataset distillation and generative adversarial networks (GANs) in scenarios with relatively sufficient energy. The experimental results on several traditional datasets show that our proposed algorithms outperform traditional algorithms in terms of accuracy and energy consumption.

This article investigates the problem of quantized state feedback tracking control for discrete‐time nonlinear systems subject to privacy protection and hybrid attacks. To capture the inherent nonlinear dynamics of the plant and improve modeling accuracy, an interval type‐2 (IT2) Takagi‐Sugeno (T‐S) fuzzy model is adopted. To mitigate the communication burden in networked transmission, three uniform dynamic quantizers are incorporated into the communication links to reduce the number of transmitted packets. The primary objective of this study is to design an IT2 fuzzy state feedback tracking controller that guarantees both the mean‐square asymptotic stability and tracking performance of the closed‐loop system, even in the presence of privacy protection constraints and hybrid cyber‐attacks. Moreover, all design parameters can be obtained by solving a set of linear matrix inequalities. Finally, two simulation examples are presented to verify the effectiveness of the proposed method.

Many research works have been developed for stereo image compression purpose where most of them aim at encoding a reference image, a residual one and a disparity map. While the disparity field is often losslessly encoded, we are mainly interested in this paper in the bit allocation problem between the reference and residual images. Generally, the bit allocation is expressed as an optimization problem which involves the computation of the operational rate-distortion (RD) functions for all the wavelet subbands and for different quantization steps. However, this strategy is computationally intensive. To solve this problem, we consider the uniform scalar quantization of the wavelet subbands of both images modeled by a Generalized Gaussian distribution. Thanks to recent approximations of the entropy and distortion functions, we develop an optimal and fast bit allocation method. The obtained results confirm the efficiency of the proposed bit allocation method in the context of stereo image coding.

It is appealing but challenging to achieve real-time deep neural network (DNN) inference on mobile devices, because even the powerful modern mobile devices are considered as "resource-constrained" when executing large-scale DNNs. It necessitates the sparse model inference via weight pruning, i.e., DNN weight sparsity, and it is desirable to design a new DNN weight sparsity scheme that can facilitate real-time inference on mobile devices while preserving a high sparse model accuracy. This paper designs a novel mobile inference acceleration framework GRIM that is General to both convolutional neural networks (CNNs) and recurrent neural networks (RNNs) and that achieves Real-time execution and high accuracy, leveraging fine-grained structured sparse model Inference and compiler optimizations for Mobiles. We start by proposing a new fine-grained structured sparsity scheme through the Block-based Column-Row (BCR) pruning. Based on this new fine-grained structured sparsity, our GRIM framework consists of two parts: (a) the compiler optimization and code generation for real-time mobile inference; and (b) the BCR pruning optimizations for determining pruning hyperparameters and performing weight pruning. We compare GRIM with Alibaba MNN, TVM, TensorFlow-Lite, a sparse implementation based on CSR, PatDNN, and ESE (a representative FPGA inference acceleration framework for RNNs), and achieve up to 14.08× speedup.

Nowadays, mobile robot exploration needs a rangefinder to obtain a large number of measurement points to achieve a detailed and precise description of a surrounding area and objects, which is called the point cloud. However, a single point cloud scan does not cover the whole area, so multiple point cloud scans must be acquired and compared together to find the right matching between them in a process called registration method. This method requires further processing and places high demands on memory consumption, especially for small embedded devices in mobile robots. This paper describes a novel method to reduce the burden of processing for multiple point cloud scans. We introduce our approach to preprocess an input point cloud in order to detect planar surfaces, simplify space description, fill gaps in point clouds, and get important space features. All of these processes are achieved by applying advanced image processing methods in combination with the quantization of physical space points. The results show the reliability of our approach to detect close parallel walls with suitable parameter settings. More importantly, planar surface detection shows a 99% decrease in necessary descriptive points almost in all cases. This proposed approach is verified on the real indoor point clouds.

Automatic and accurate tooth segmentation on 3D dental point clouds plays a pivotal role in computer-aided dentistry. Existing Transformer-based methods focus on aggregating local features, but fail to directly model global contexts due to memory limitations and high computational cost. In this paper, we propose a novel Transformer-based 3D tooth segmentation network, called PointRegion, which can process the entire point cloud at a low cost. Following a novel modeling of semantic segmentation that interprets the point cloud as a tessellation of learnable regions, we first design a RegionPartition module to partition the 3D point cloud into a certain number of regions and project these regions as embeddings in an effective way. Then, an offset-attention based RegionEncoder module is applied on all region embeddings to model global context among regions and predict the class logits for each region. Considering the irregularity and disorder of 3D point cloud data, a novel mechanism is proposed to build the point-to-region association to replace traditional convolutional operations. The mechanism, as a medium between points and regions, automatically learns the probabilities that each point belongs to its neighboring regions from the similarity between point and region features, achieving the goal of point-level segmentation. Since the number of regions is far less than the number of points, our proposed PointRegion model can leverage the capability of the global-based Transformer on large-scale point clouds with low computational cost and memory consumption. Finally, extensive experiments demonstrate the effectiveness and superiority of our method on our dental dataset.

<p>Deep learning models have achieved remarkable success in many different fields and attracted many interests. Several researchers attempted to apply deep learning models to synthetic aperture radar (SAR) data processing, but it did not have the same breakthrough as the other fields, including optical remote sensing. SAR data are in complex domain by nature and processing them with real-valued (RV) networks neglects the phase component which conveys important and distinctive information. A complex-valued (CV) end-to-end deep network is developed in this study for the reconstruction and classification of CV-SAR data. Azimuth subaperture decomposition is utilized to incorporate physics-aware attributes of the CV-SAR into the deep model. Moreover, the correlation coefficient amplitude (coherence) of the CV-SAR images depends on the SAR system characteristics and physical properties of the target. This coherency should be considered and preserved in the processing chain of the CV-SAR data. The coherency preservation of the CV deep networks for CV-SAR images, which is mostly neglected in the literature, is evaluated in this study. Furthermore, a large-scale CV-SAR annotated dataset for the evaluation of the CV deep networks is lacking. A semantically annotated CVSAR dataset from Sentinel-1 single look complex stripmap mode data [S1SLC_CVDL (complex-valued deep learning) dataset] is developed and introduced in this study. The experimental analysis demonstrated the better performance of the developed CV deep network for CV-SAR data classification and reconstruction in comparison with the equivalent RV model and more complicated RV architectures, as well as its coherency preservation and physics-aware capability.</p>

Deep learning has recently become vitally important in the field of medical image segmentation. Segmentation architectures include more network layers than classification architectures and require roughly twice as many network parameters. This large number of network layers may result in vanishing gradient or redundant computation, increased computational complexity and more memory consumption. Therefore, it is essential to develop an efficient deep model for segmenting medical images. In this work, a novel deep model named MIS-Net is proposed for medical image segmentation. The proposed MIS-Net model is a 69-layer encoder/decoder convolutional model that employs the concept of memorised pooling indices and residual learning. The proposed model is examined for three medical segmentation challenges, namely liver segmentation from the LITS and 3DIRCADb datasets, lung and COVID-19 infection segmentations from the COVID-19 CT dataset. The experiment results for the segmentation of the liver, lung, and COVID-19 infection acquire average dice scores of 0.989, 0.992, and 0.934, respectively. The MIS-Net model surpasses the other deep segmentation models with 0.34M (million) network parameters. The proposed model is a parameter efficient model that reduces the computational complexity and memory utilisation of the model.

Recently, several approaches have been proposed to automatically model indoor environments. Most of such efforts principally rely on the crowd to sense data such as motion traces, images, and WiFi footprints. However, large datasets are usually required to derive precise indoor models which can negatively affect the energy efficiency of the mobile devices participating in the crowd-sensing system. Furthermore, the aforementioned data types are hardly suitable for deriving 3D indoor models. To overcome these challenges, we propose GraMap, a QoS-aware automatic indoor modeling approach through crowd-sensing 3D point clouds. GraMap exploits a recently-developed sensors fusion mechanism, namely Tango technology, to cooperatively collect point clouds from the crowd. Afterward, a set of backend servers extracts the required geometrical information to derive indoor models. For the sake of improving the energy efficiency of the mobile devices, GraMap performs data quality assurance along with 3D data compression. Specifically, we propose a probabilistic quality model---implemented on the mobile devices---to ensure high-quality of the captured point clouds. In this manner, we conserve energy via sidestepping the repetition of sensing queries due to uploading low-quality point clouds. Nevertheless, the resultant indoor models may still suffer from incompleteness and inaccuracies. Therefore, GraMap leverages formal grammars which encode design-time knowledge, i.e. structural information about the building, to enhance the quality of the derived models. To demonstrate the effectiveness of GraMap, we implemented a crowd-sensing Android App to collect point clouds from volunteers. We show that GraMap derives highly-accurate models while reducing the energy costs of pre-processing and reporting the point clouds.

At present, there have been many studies to design various CNN hardware accelerators to accelerate the inference of deep neural network models. The FPGA-based CNN reasoning accelerator can provide sufficient computing power support with flexible data accuracy, lower energy consumption and lower application cost, and has received a lot of attention in the application field of IoT terminal devices with limited computing power and energy consumption. Widespread concern. However, although the current FPGA-based CNN accelerator has greatly improved the speed of model reasoning through various methods, most of the methods cannot be effectively applied to actual terminal scenarios due to limitations in memory and energy consumption. In response to this situation, we designed an acceleration framework that takes into account both inference acceleration and energy consumption. Aiming at the limitation of computing power in the terminal environment, optimize a large number of multiplication operations in the convolution operation that consumes the most computing power in the CNN inference stage, by using local cache and matrix transformation formulas, and skipping pairings by zero values in the calculation process the model inference operation is further accelerated while reducing energy consumption. The experimental results show that compared with the current advanced neural network accelerator, not only the computing power has been significantly improved, but also the energy efficiency ratio has achieved better results. Moreover, this method can not only be implemented in FPGA, but also be migrated to other embedded terminals.

Due to the limited computational capability of mobile devices, the research organization and academia are working on computationally secure schemes that have capability for offloading the computational intensive data access operations on the cloud/trusted entity for execution. Most of the existing security schemes, such as proxy re-encryption, manager-based re-encryption, and cloud-based re-encryption, are based on El-Gamal cryptosystem for offloading the computational intensive data access operation on the cloud/trusted entity. However, the resource hungry pairing-based cryptographic operations, such as encryption and decryption, are executed using the limited computational power of mobile device. Similarly, if the data owner wants to modify the encrypted file uploaded on the cloud storage, after modification the data owner must encrypt and upload the entire file on the cloud storage without considering the altered portion(s) of the file. In this paper, we have proposed an incremental version of proxy re-encryption scheme for improving the file modification operation and compared with the original version of the proxy re-encryption scheme on the basis of turnaround time, energy consumption, CPU utilization, and memory consumption while executing the security operations on mobile device. The incremental version of proxy re-encryption scheme shows significant improvement in results while performing file modification operations using limited processing capability of mobile devices.

The extended use of mobile multimedia devices in applications like gaming, 3D video and audio reproduction, immersive teleconferencing, or virtual and augmented reality, is demanding efficient algorithms and methodologies. All these applications require real-time spatial audio engines with the capability of dealing with intensive signal processing operations while facing a number of constraints related to computational cost, latency and energy consumption. Most mobile multimedia devices include a Graphics Processing Unit (GPU) that is primarily used to accelerate video processing tasks, providing high computational capabilities due to its inherent parallel architecture. This paper describes a scalable parallel implementation of a real-time binaural audio engine for GPU-equipped mobile devices. The engine is based on a set of head-related transfer functions (HRTFs) modelled with a parametric parallel structure, allowing efficient synthesis and interpolation while reducing the size required for HRTF data storage. Several strategies to optimize the GPU implementation are evaluated over a well-known kind of processor present in a wide range of mobile devices. In this context, we analyze both the energy consumption and real-time capabilities of the system by exploring different GPU and CPU configuration alternatives. Moreover, the implementation has been conducted using the OpenCL framework, guarantying the portability of the code.