MoBiNet: A Mobile Binary Network for Image Classification

Speaker verification based on deep neural network is one of the most prospective biometric authentication methods. But massive computation and parameters make it difficult to deploy speaker verification systems on memory and power constrained embedded devices. Binary neural networks with weights and activations compressed to 1 bit can reduce computational and memory cost significantly. However, speaker verification system based on binary neural networks suffers performance degradation problem. In this paper, to mitigate this problem and facilitate information flow across layers, we optimize binary neural network structure by adding a shortcut to connect layers. We propose an energy and memory efficient speaker system based on the optimized binary networks. Experimental results on text-dependent and text-independent datasets show that performance of proposed speaker verification system outperforms conventional binary neural networks significantly. Compared with full precision networks, the proposed system can largely maintain the performance while reducing memory and computational costs.

The study presents the results of the development and testing of deep learning neural network architectures, which demonstrate high accuracy rates in classifying neurophysiological data, in particular hemodynamic brain activation patterns obtained by functional near-infrared spectroscopy, during solving mathematical problems on spatial-numerical associations. The analyzed signal represents a multidimensional time series of oxyhemoglobin and deoxyhemoglobin dynamics. Taking the specificity of the fNIRS signal into account, a comparative analysis of 2 types of neural network architectures was carried out: (1) architectures based on recurrent neural networks: recurrent neural network with long short-term memory, recurrent neural network with long short-term memory with fully connected layers, bidirectional recurrent neural network with long short-term memory, convolutional recurrent neural network with long short-term memory; (2) architectures based on convolutional neural networks with 1D convolutions: convolutional neural network, fully convolutional neural network, residual neural network. Trained long short-term memory recurrent neural network architectures showed worse results in accuracy in comparison with 1D convolutional neural network architectures. Residual neural network (model_Resnet) demonstrated the highest accuracy rates in three experimental conditions more than 88% in detecting age-related differences in brain activation during spatial-numerical association tasks considering the individual characteristics of the respondents’ signal.

Field-programmable gate array (FPGA)–specific deep neural network (DNN) architectures using native lookup tables (LUTs) as independently trainable inference operators have been shown to achieve favorable area-accuracy and energy-accuracy trade-offs. The first work in this area, LUTNet, exhibited state-of-the-art performance for standard DNN benchmarks. In this article, we propose the learned optimization of such LUT-based topologies, resulting in higher-efficiency designs than via the direct use of off-the-shelf, hand-designed networks. Existing implementations of this class of architecture require the manual specification of the number of inputs per LUT, K . Choosing appropriate K a priori is challenging. Doing so at even high granularity, for example, per layer, is a time-consuming and error-prone process that leaves FPGAs’ spatial flexibility underexploited. Furthermore, prior works see LUT inputs connected randomly, which does not guarantee a good choice of network topology. To address these issues, we propose logic shrinkage , a fine-grained netlist pruning methodology enabling K to be automatically learned for every LUT in a neural network targeted for FPGA inference. By removing LUT inputs determined to be of low importance, our method increases the efficiency of the resultant accelerators. Our GPU-friendly solution to LUT input removal is capable of processing large topologies during their training with negligible slowdown. With logic shrinkage, we improve the area and energy efficiency of the best-performing LUTNet implementation of the CNV network classifying CIFAR-10 by 1.54× and 1.31×, respectively, while matching its accuracy. This implementation also reaches 2.71× the area efficiency of an equally accurate, heavily pruned binary neural network (BNN). On ImageNet, with the Bi-Real Net architecture, employment of logic shrinkage results in a post-synthesis area reduction of 2.67× vs. LUTNet, allowing for implementation that was previously impossible on today’s largest FPGAs. We validate the benefits of logic shrinkage in the context of real application deployment by implementing a face mask detection DNN using a BNN, LUTNet, and logic-shrunk layers. Our results show that logic shrinkage results in area gains versus LUTNet (up to 1.20×) and equally pruned BNNs (up to 1.08×), along with accuracy improvements.

Binary Neural Networks (BNNs) have received significant attention due to the memory and computation efficiency recently. However, the considerable accuracy gap between BNNs and their full-precision counterparts hinders BNNs to be deployed to resource-constrained platforms. One of the main reasons for the performance gap can be attributed to the frequent weight flip, which is caused by the misleading weight update in BNNs. To address this issue, we propose a state-aware binary neural network (SA-BNN) equipped with the well designed state-aware gradient. Our SA-BNN is inspired by the observation that the frequent weight flip is more likely to occur, when the gradient magnitude for all quantization states {-1,1} is identical. Accordingly, we propose to employ independent gradient coefficients for different states when updating the weights. Furthermore, we also analyze the effectiveness of the state-aware gradient on suppressing the frequent weight flip problem. Experiments on ImageNet show that the proposed SA-BNN outperforms the current state-of-the-arts (e.g., Bi-Real Net) by more than 3% when using a ResNet architecture. Specifically, we achieve 61.7%, 65.5% and 68.7% Top-1 accuracy with ResNet-18, ResNet-34 and ResNet-50 on ImageNet, respectively.

Multilevel image segmentation with adaptive image context based thresholding

Binary neural networks have recently begun to be used as a highly energy- and computation-efficient image processing technique for computer vision tasks. This paper proposes a novel extension of existing binary neural network technology based on the use of a Hadamard transform in the input layer of a binary neural network. Previous state-of-the-art binary neural networks require floating-point arithmetic at several parts of the neural network model computation in order to maintain a sufficient level of accuracy. The Hadamard transform is similar to a Discrete Cosine Transform (used in the popular JPEG image compression method) except that it does not include expensive multiplication operations. In this paper, it is shown that the Hadamard transform can be used to replace the most expensive floating-point arithmetic portion of a binary neural network. In order to test the efficacy of this proposed method, three types of experiments were conducted: application of the proposed method to several state-of-the-art neural network models, verification of its effectiveness in a large image dataset (ImageNet), and experiments to verify the effectiveness of the Hadamard transform by comparing the performance of binary neural networks with and without the Hadamard transform. The results show that the Hadamard transform can be used to implement a highly energy-efficient binary neural network with only a miniscule loss of accuracy.

Deep Neural Networks are resource-intensive learning models. To enable their training and deployment on low-end devices, decreasing their computational requirements has become an attractive research area in recent years. One of the most promising approaches in this field is designing networks which use only binary values for the network weights and activations, i.e., Binary Neural Networks (BNN). However, BNNs have robustness and stability issues, which make their training difficult and BNNs perform considerably worse than their real-valued counterparts. To improve the training and deployment performance of BNNs, in this work, we employ a tree-structured network model, which has been widely used in Deep Neural Networks (DNN). We perform experiments to analyze the accuracy, stability and robustness of the proposed tree-structured BNN model. Tree-structured network model integrates multiple classifiers on a backbone BNN resulting in a multi-branch architecture. Our experiments demonstrates that tree-structures model improves the accuracy, stability and robustness of BNNs.

This work presents an FPGA-based hardware/software design to help the agricultural robot intelligently decide if biological agents need to be applied to the target crops. For target crop recognition, in our global positioning, the selective search integrates with a thresholding scheme to reduce the number of region of interest (ROI) in a captured image. In our local recognition, a binarized neural network (BNN) architecture is presented to help recognize the target crop. Furthermore, an estimation method of pest and disease severity is also presented. Experiments show integrating our presented BNN architecture needs a few extra resources (less than 17% of available FPGA resources in terms of Xilinx Zynq UltraScall+™ MPSoC ZU3EG A484), compared to an existing BNN one. However, the top-1 accuracy rate and the top-5 one can be increased by 32.25% to 32.84% and by 14.99% to 15.17%, respectively. Furthermore, when the presented BNN architecture was also implemented on the ARM Cortex-A53 CPU and the NVIDIA GeForce RTX 2080 GPU, our BNN hardware module on the FPGA can accelerate the frames per second (FPS) by a factor of 3,690.18 and a factor of 1.07, respectively.

Based on the vigorous development of deep learning, image classification nowadays is mainly accomplished by using the Convolutional Neural Networks whose main computation tasks are usually calculated by Graphics Processing Units. Although the computing speed of Graphics Processing Units is faster than Central Processing Units when considering the neural networks, the power consumption of Graphics Processing Units is significantly huge. In this paper, we investigate the hardware acceleration of neural networks by using the Field Programmable Gate Array via High-Level Synthesis. We study the binarized neural networks with binarized weights and activation functions, which is suitable to be implemented by the Field Programmable Gate Array. The proposed design has high computing speed and low power consumption. Our method is applied to a robot vision application to classify robots from human and others. Experiment results show that our design provides good performance for image classification.

Underwater habitats are home to about 34,800 fish species. Understanding the various fish species and their therapeutic properties is crucial. The majority of bodies of water have a large number of fish. They can be found in almost all aquatic settings, from high alpine streams (like char and gudgeon) to the abyssal and even hadal depths of the deepest oceans, despite the fact that no species have yet been discovered in the bottom 25% of the ocean (such as cusk-eels and snailfish). In the research, a model for categorizing the various fish species was devised. To ascertain the fish’s possible health advantages, different fish species are categorized. The transfer learning is used to pretrain models with our data which is then given to convolutional neural networks (CNNs), visual geometry group (VGG), residual neural network (RESNET), and densely connected neural network (DENSENET) with a moderate amount of change in the output layer. Additionally, this dataset is input to a CNN model with five sequential convolutional layers that uses the “Adam optimizer,” “ReLU,” and the “SoftMax” activation function. Furthermore, convolutional neural network is used with sixteen layers and softmax activation at the bottom to create the VGG model. The transfer learning is used for RESNET and DENSENET by changing the last layers. The 121 thick layers of the DENSENET that has used were interconnected forward and used Adam optimizer. The 50-layer RESNET, which employs skip connections with ReLU and softmax activation functions. By using classic CNN, we obtain an accuracy of 98.11%; with the help of VGG, and able to achieve an accuracy of 99%; with RESNET, a 99.56% accuracy; and finally, DENSENET, a 98.78% accuracy. Potential health advantages of fish.

Neural networks have been widely applied in system dynamics modeling. However, traditional neural networks still encounter limitations in capturing long-term dynamics, nonlinear modeling, and interpretability. To address these challenges, this study proposes a novel neural network architecture, Deep Stacked State-observer based Neural Network (DSSO-NN). Firstly, the state-space representation is introduced, integrating discretized state-space equations into the neural network design to leverage both system state information and deep learning capabilities. Subsequently, two optimization measures are employed to enhance the network's nonlinear modeling ability with activation functions and the state observer, respectively. Finally, DSSO-NN is validated using the Case Western Reserve University bearing dataset. Experimental results demonstrate that activation functions provide minimal improvement to model performance. In contrast, the incorporation of the state observer significantly enhances the DSSO-NN's ability to capture system dynamics behaviors and improves modeling accuracy. DSSO-NN exhibits higher precision and greater stability, offering a novel perspective on using the state observer as an alternative to traditional activation functions.

To achieve an advanced Internet of Things (IoT), it is necessary to combine artificial intelligence (AI) with IoT. Compact circuits that can operate AI functions will be useful for this purpose. Therefore, we propose stochastic weights binary neural networks (SWBNN). SWBNNs are more accurate than binary neural networks (BNN) with small circuits. BNNs can be realized with small circuits since binary calculation needs simpler circuits than real number calculation. However, BNNs have lower accuracy than networks with real numbers. Thus, the proposed SWBNNs are BNNs that behave stochastically, which makes them more accurate than BNNs. Moreover, SWBNNs can still be achieved with small circuits since they execute binary calculation. As a result, the accuracy for the test data of SWBNNs is closer to the accuracy for learning data than the accuracy for the test data of BNNs is. Especially when using the CIFAR10 database, the difference in the identification accuracy rate between learning data and test data decreased from 6% for BNNs to 2% for SWBNNs. From results of a field-programmable gate array (FPGA) implementation, circuits of SWBNNs are sufficiently small although they are 10% bigger than those of BNNs. Therefore, SWBNNs are more accurate than BNNs, and the circuit costs ofintroducing stochastic weights are low.

Introduction. The article considers the solution of the problem of detecting fires at technological facilities in automatic mode. To solve this problem, an approach was chosen to create a convolutional neural network capable of operating on a real-time video stream. Aims and Purposes. The aim of this work is to create a neural network capable of detecting flames and smoke in the image from CCTV cameras. The purposes of the work include: selection of the optimal architecture of the neural network in accordance with the latest research in this field; speeding up the chosen architecture using quantization and filter thinning techniques. Methods. Different architectures of convolutional neural networks performing the task of detecting objects in an image are considered. Their performance and quality of work are compared. The YOLOv5 architecture, its target function, training methods and ways of speeding up work are considered. Results and discu ssion. The paper shows the results of training a convolutional neural network of the YOLOv5 architecture for the task of flame and smoke detection, as well as how the results change when applying neural network acceleration methods. It was determined that the use of such acceleration methods such as quantization and filter cleaning can significantly increase up the speed of the neural network, while almost no loss in accuracy of operation. Conclusions. As a result of the conducted work, the architecture of the neural network was determined for performing the task of detecting a fire source. Based on the chosen architecture, a neural network was trained to detect flames and smoke in the image. The speed of its work allows to process video stream in real-time without using graphic accelerator.

<p dir="ltr">Recurrent Neural Networks (RNNs) are a major class of Artificial Neural Networks (ANNs). Suitable network architecture is vital for RNNs to achieve high performance. However, designing the architecture of deep RNNs (DRNNs) is a complicated and time-consuming process. Each layer of a DRNN contains numerous hyper-parameters. It is challenging to optimize the hyper-parameters of all layers jointly to achieve the best possible learning performance. Trial-and-error methods for designing DRNNs have been proven to be laborious and highly costly in practice. Therefore, an efficient automatic architecture search technique is needed to design DRNNs.</p><p dir="ltr">Genetic Algorithm (GA) is a popular evolutionary computation based approach. Existing research works already explored the use of GAs in designing ANNs. But, most of the existing approaches focus on the design of fixed-depth ANNs. However, it is not easy to fix the depth of DRNNs in advance, since different problems need DRNNs of different depths. Hence, the potential of GA has to be further explored to evolve the architecture of DRNNs of varying depths.</p><p dir="ltr">The primary goal of this thesis is to develop advanced GA approaches for designing Long Short Term Memory (LSTM) based DRNN architectures of varying depths effectively and efficiently.</p><p dir="ltr">Firstly, this thesis proposes a GA-based algorithm called Two-Stage Surrogate-Assisted GA (TS-SA-GA) with a progressive incremental strategy to design the architecture of LSTM networks of varying depths. The progressive approach can effectively extend well-designed shallow networks to high performing deep networks. Moreover, the new GA-based algorithm adopts newly designed knowledge-driven crossover and mutation operators to identify and repair LSTM network designs affected by inappropriate use of activation functions, thereby significantly reducing the chances for GA to evolve hard-to-train LSTM network architectures. Furthermore, this thesis proposes a two-stage surrogate method to predict the trainability and fitness of LSTM networks. This improves the efficiency and effectiveness of the GA-based algorithm to evolve LSTM networks.</p><p dir="ltr">Secondly, this thesis proposes a new algorithm called Evolutionary Design of LSTM Ensembles (ED-LSTM-Ensemble) based on multi-objective optimization techniques to evolve LSTM networks and ensembles simultaneously to directly utilize the ensemble performance to drive the evolution of base LSTM networks. Additionally, this thesis proposes a connection weight inheritance strategy to evolve LSTM networks and ensembles efficiently and effectively.</p><p dir="ltr">Thirdly, this thesis proposes a new algorithm called Skip Connections through Evolutionary and Differential Architecture search (SCEDA) to design LSTM networks together with appropriate skip connections automatically. Designing LSTM networks together with skip connections is highly challenging. This thesis hybridize gradient-based DAS and evolutionary architecture search to optimize the architecture of LSTM networks with suitable skip connections in a single evolutionary process to improve the efficiency and effectiveness of the search process.</p><p dir="ltr">Finally, this thesis conducts an in-depth empirical analysis of the impact of activation functions on the trainability of LSTM networks based on the concept of Edge Of Chaos. The analysis reveals the strong interrelation between the activation functions across multiple layers of an LSTM network and the trainability of the network. The analysis also highlights the importance of controlling activation functions to avoid generating untrainable LSTM networks. On the basis of the analysis, this thesis proposes a machine learning model to guide the use of activation functions in LSTM networks.</p>

Compared with traditional images, multispectral images (MSIs) contain more spectral bands and higher data dimensions. The existing MSI classification model has high computational complexity and consumes a lot of computing resources. In this letter, we propose a lightweight multispectral classification method named CABNN based on binary neural networks (BNNs) to effectively have a trade-off between model performance and computational cost. First, we modify and binarize the MobileNetV1 network and add almost computation-free shortcuts to enhance the expressive capability. Secondly, since the BNN is sensitive to the distribution of activation functions, we introduce RPReLU with learnable coefficients to automatically adjust activation distribution at almost no extra cost. Lastly, considering that MSIs have multiple channels, we utilize an efficient channel attention (ECA) module to assign different weights to each channel to concentrate on crucial features and suppress insignificant features. We conduct experiments on four public MSI datasets, including NaSC-TG2, EuroSAT, GID Fine land-cover classification, and UC Merced Land Use. Extensive experiments demonstrate that the proposed CABNN has higher efficiency and better comprehensive performance than the state-of-the-art methods across the board.