Binary Neural Network for Multispectral Image Classification

Speaker verification based on deep neural network is one of the most prospective biometric authentication methods. But massive computation and parameters make it difficult to deploy speaker verification systems on memory and power constrained embedded devices. Binary neural networks with weights and activations compressed to 1 bit can reduce computational and memory cost significantly. However, speaker verification system based on binary neural networks suffers performance degradation problem. In this paper, to mitigate this problem and facilitate information flow across layers, we optimize binary neural network structure by adding a shortcut to connect layers. We propose an energy and memory efficient speaker system based on the optimized binary networks. Experimental results on text-dependent and text-independent datasets show that performance of proposed speaker verification system outperforms conventional binary neural networks significantly. Compared with full precision networks, the proposed system can largely maintain the performance while reducing memory and computational costs.

Deep neural network (DNN) has powerful feature extraction and classification abilities and is extensively used in mechanical fault diagnosis. However, in the DNN-based fault diagnosis method, too many model parameters, high computational complexity, and occupying a large amount of storage space make it very difficult to optimize and retrain the DNN model. To overcome this deficiency, a binary neural network (BNN) is introduced into fault diagnosis. In the proposed BNN-based fault diagnosis method, the floating-point weights and activations are binarized as ‘+1’ and ‘−1’ to save storage space, and the floating-point multiply-accumulate operations (OPs) are replaced by bitwise OPs to reduce the computational complexity. The BNN-based fault diagnosis method effectively compresses the structure of DNNs and accelerates model inference. However, the binarization OP may bring about the quantization error and gradient error, which will accumulate with the increase of the network depth, and lead to a degradation in model performance. To solve these deficiencies in the proposed BNN-based fault diagnosis method, an improved BNN fault diagnosis method based on adaptive depth-to-width ratio (ADTWR) is proposed. For the convenience in description, the proposed method is called the BNN-ADTWR method. In the proposed BNN-ADTWR method, the network structure is reshaped by the comparison of the gains brought by the depth and width of the backbone network, and the backbone network is further optimized by the pruning technique. To evaluate the effectiveness of the proposed method, the comparative experiments, such as ablation, anti-noise, and computational complexity, had successfully completed on several rolling bearing datasets. The experiment results show that the proposed BNN-ADTWR method significantly reduces computational complexity and helps with rapid fault diagnosis. The constructed ADTWR backbone network decreases activations in the traditional backbone network and improves absolute accuracy while the computational cost is equivalent.

Recent advances in deep learning have shown that Binary Neural Networks (BNNs) are capable of providing a satisfying accuracy on various image datasets with significant reduction in computation and memory cost. With both weights and activations binarized to +1 or -1 in BNNs, the high-precision multiply-and-accumulate (MAC) operations can be replaced by XNOR and bit-counting operations. In this work, we propose a RRAM synaptic architecture (XNOR-RRAM) with a bit-cell design of complementary word lines that implements equivalent XNOR and bit-counting operation in a parallel fashion. For large-scale matrices in fully connected layers or when the convolution kernels are unrolled in multiple channels, the array partition is necessary. Multi-level sense amplifiers (MLSAs) are employed as the intermediate interface for accumulating partial weighted sum. However, a low bit-level MLSA and intrinsic offset of MLSA may degrade the classification accuracy. We investigate the impact of sensing offsets on classification accuracy and analyze various design options with different sub-array sizes and sensing bit-levels. Experimental results with RRAM models and 65nm CMOS PDK show that the system with 128×128 sub-array size and 3-bit MLSA can achieve accuracies of 98.43% for MLP on MNIST and 86.08% for CNN on CIFAR-10, showing 0.34% and 2.39% degradation respectively compared to the accuracies of ideal BNN algorithms. The projected energy-efficiency of XNOR-RRAM is 141.18 TOPS/W, showing ~33X improvement compared to the conventional RRAM synaptic architecture with sequential row-by-row read-out.

Spectral information of an underwater scene, which can be found from a set of underwater multispectral images, is a very useful technique for extracting information from underwater world. But, underwater imaging is problematic even in multispectral image format. The main problem is the effect of optical back-scatter, which changes the scene reflectance value in each spectral band and this causes inaccuracy in the spectral information. In this work, a filter is used to detect the level of optical back-scatter in each spectral band from a set of multispectral images. Extraction of an underwater object spectra can be done by subtracting the estimated level of optical back-scatter and scaling the remainder in each spectral band from the captured image in the corresponding band. An experiment is designed to show the performance of the proposed filter for correcting the set of multispectral underwater images and recovering the pixel spectra. The multispectral images are captured in 33 bands in clear and different levels of turbid water. The results of processing experimental images in turbid water show that the effect of optical back-scatter can be mitigated in the image of each spectral band and as a result the spectra of the object can be recovered. However, for very high level of turbid water the recovery is limited because of the effect of extinction.

This research proposes the multispectral image retrieval method by using spectral feature and semantic computing which is not many studies have focused. The main contributions are to enhance the effectiveness and advantageous of global environmental analysis system and realize semantic associative search and analysis. In this work, we study multispectral image retrieval using spectral feature computed in multispectral semantic-image space. The multispectral semantic-image space is supposing to realize the interpretation of substance (materials) on earth surface which can be provided the analyzed results as human-level interpretation. Our essential approach is utilizing the semantic computing to measure the similarity between multispectral image and the meaningful keywords which according to the user's contexts. Our research results found that this method possible to acquire the spectral feature from the multispectral image and could be used in multispectral image retrieval. In this study, a multispectral image is used as the image query according to user's query contexts. Moreover, the method performance of UAV-based multispectral aerial image retrieval using spectral feature and semantic computing is measured based on the queries with three contexts of multispectral image which is indicated by previous study on agricultural monitoring system and semantic interpretation model.

Binary Neural Networks (BNNs) show promising progress in reducing computational and memory costs, but suffer from substantial accuracy degradation compared to their real-valued counterparts on large-scale datasets, e.g., ImageNet. In this work we study existing BNN architectures and revisit the commonly used technique to include scaling factors. We suggest several architectural design principles for BNNs, based on our studies on architectures. Guided by our principles we develop a novel BNN architecture BinaryDenseNet, which is the first architecture specifically created for BNNs to the best of our knowledge. In our experiments, BinaryDenseNet achieves 18.6% and 7.6% relative improvement over the well-known XNOR-Network and the current state-of-the-art Bi-Real Net in terms of top-1 accuracy on ImageNet, respectively. Further, we show the competitiveness of our BinaryDenseNet regarding memory requirements and computational complexity.

During the past decade, there have been significant improvements in remote sensing technologies, which have provided high‐resolution data at shorter time intervals. Considerable effort has been directed towards developing new classification strategies for analysing this imagery, but the use of artificial intelligence‐based analysis techniques has been somewhat limited. The aim of this study was to develop an artificial neural network (ANN)‐based technique for the classification of multispectral aerial images for land use in agricultural and environmental applications. The specific land‐use classes included water, forest, and several types of agricultural fields. Multispectral images at a 1‐m resolution were obtained for the state of Georgia, USA from a Geographic Information Systems (GIS) data clearinghouse. These false‐colour images contained green, red and infrared true‐colour information. Three approaches were used for the preparation of the inputs to the ANN. These included histograms of the pixel intensities, textural parameters extracted from the image, and matrices of the pixels for spatial information. A probabilistic neural network was used. Seven hundred images were used for model development and 175 for independent model evaluation. The overall accuracy for the evaluation data set was 74% for the histogram approach, 71% for the spatial approach and 89% for the textural approach. The evaluation of ANNs based on various combinations of all three approaches did not show an improvement in accuracy. We also found that some approaches could be used selectively for certain classes. For example, the textural approach worked best for forest classes. For future studies, edge detection prior to classification, with more careful selection of each class, should be included for land‐use classification of multispectral images.

S[formula omitted]NN: Time step reduction of spiking surrogate gradients for training energy efficient single-step spiking neural networks

The binarized neural network (BNN) is one of the most promising candidates for low-cost convolutional neural networks (CNNs). This is because of its significant reduction in memory and computational costs, and reasonable classification accuracy. Content-addressable memory (CAM) can perform binarized convolution operations efficiently since the bitwise comparison in CAM matches well with the binarized multiply operation in a BNN. However, a significant design issue in CAM-based BNN accelerators is that the operational reliability is severely degraded by process variations during match-line (ML) sensing operations. In this paper, we proposed a novel ML sensing scheme to reduce the hardware error probability. Most errors occur when the difference between the number of matches in the evaluation ML and the reference ML is small; thus, the proposed hardware identified cases that are vulnerable to process variations using dual references. The proposed dual-reference sensing structure has >49% less ML sensing errors than that of the conventional design, leading to a >1.0% accuracy improvement for Fashion MNIST image classification. In addition, owing to the parallel convolution operation of the CAM-based BNN accelerator, the proposed hardware achieved >34% processing-time improvement compared with that of the digital logic implementation.

An efficient unconstrained facial expression recognition algorithm based on Stack Binarized Auto-encoders and Binarized Neural Networks

Recent research has demonstrated that effective fusion of multispectral images (visible and thermal images) enables robust pedestrian detection under various illumination conditions (e.g., daytime and nighttime). However, there are some open problems such as poor performance in small-sized pedestrian detection and high computational cost of multispectral information fusion. This paper proposes a multilayer fused deconvolutional single-shot detector that contains a two-stream convolutional module (TCM) and a multilayer fused deconvolutional module (MFDM). The TCM is used to extract convolutional features from multispectral input images. Then fusion blocks are incorporated into the MFDM to combine high-level features with rich semantic information and low-level features with detailed information to generate features with strong a representational power for small pedestrian instances. In addition, we fuse multispectral information at multiple deconvolutional layers in the MFDM via fusion blocks. This multilayer fusion strategy adaptively makes the most use of visible and thermal information. In addition, using fusion blocks for multilayer fusion can reduce the extra computational cost and redundant parameters. Empirical experiments show that the proposed approach achieves an 81.82% average precision (AP) on a new small-sized multispectral pedestrian dataset. The proposed method achieves the best performance on two well-known public multispectral datasets. On the KAIST multispectral pedestrian benchmark, for example, our method achieves a 97.36% AP and a 20 fps detection speed, which outperforms the state-of-the-art published method by 6.82% in AP and is three times faster in its detection speed.

Research on LWIR multispectral imaging detection technology carried out in the key national defense laboratory in Tianjin University is introduced in this paper. Firstly, a kind of infrared multispectral image simulation method based on multispectral or hyperspectral images data in the VIS/NIR band is recommended. The combined strategy of unsupervised and supervised classification methods is put forward to efficiently realize auto-matching and labeling of pixels. Then, using the infrared image simulation technology, infrared multispectral simulation images can be generated highly similar to real natural environments, which are valuable to the development of LWIR multispectral spectrometers, as well as research on multispectral detection algorithms. Secondly, the co-image plane imaging detection technique is presented as well as the attempt to make the small LWIR multispectral imaging detector based on such concept. A fourband prototype has been achieved in VIS/NIR band, verifying the feasibility and validity of this technique.

Although deep neural networks are successful for many tasks in the speech domain, the high computational and memory costs of deep neural networks make it difficult to directly deploy highperformance Neural Network systems on low-resource embedded devices. There are several mechanisms to reduce the size of the neural networks i.e. parameter pruning, parameter quantization, etc. This paper focuses on how to apply binary neural networks to the task of speaker verification. The proposed binarization of training parameters can largely maintain the performance while significantly reducing storage space requirements and computational costs. Experiment results show that, after binarizing the Convolutional Neural Network, the ResNet34-based network achieves an EER of around 5% on the Voxceleb1 testing dataset and even outperforms the traditional real number network on the text-dependent dataset: Xiaole while having a 32x memory saving.

Partition of Relative Maximum Region Band Based Image Compression

Uncertainty of soil reflectance retrieval from SPOT and RapidEye multispectral satellite images using a per-pixel bootstrapped empirical line atmospheric correction over an agricultural region