Neuromorphic LIF Row-by-Row Multiconvolution Processor for FPGA.

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

Deep (machine) learning in recent years has significantly increased the predictive modeling strengths for applications in the areas of computer vision, speech recognition, and natural language processing. Being heavily data-driven, conventional general-purpose deep learning methods mainly rely on training data to learn task-specific data representations. In this thesis, we focus on leveraging novel learning algorithms in deep learning to tackle visual recognition challenges at the three key levels -- namely pixel, object, and image levels. Throughout the thesis, we demonstrate how each and every of our proposed methods can address a specific visual recognition problem. Firstly, we apply deep learning to improve the saliency detection performance of deep networks. A coarse saliency map of the input image's salient object is initially predicted by a convolutional-deconvolutional network (or equivalently a fully convolutional network). Following that, the coarse saliency map is sequentially refined by a separate network called Recurrent Attentional Convolutional-Deconvolutional Network (RACDNN) that shares its network weights across its sequential steps. At every sequential (temporal) step, the same RACDNN is applied to a newly selected/attended image sub-region and it performs saliency map refinement for that particular sub-region. Despite having many learnable network parameters and no access to additional training data, RADCNN can be trained well and it outperforms the "single-step" (coarse saliency detection) baseline and state-of-the-art saliency detection methods by large margins, on several saliency detection benchmark datasets. Secondly, we introduce auto-encoder weight transfer networks (AE-WTN+), a deep model for scaling up object detection. Conventional WTN transfers object class knowledge from the classification weights of a pretrained large-scale image classification network, to the classification weights of an object detection network. However, they suffer from under-fitting and are not able to preserve the rich object class information of novel classes. In this work, we propose a new WTN method -- AE-WTN+ which makes use of normalization techniques to ease network optimization. Inputs and intermediate features are normalized to improve trainability for reduced under-fitting. More importantly, a novel auto-encoding (reconstruction) loss is proposed to encourage information preservation of all classes in the weight predictions of AE-WTN+. This enables the predicted weights of AE-WTN+ to maintain the class discriminability and neighbourhood relationships contained in the pre-trained large-scale classification weights. Experiments on large-scale detection datasets validate the effectiveness of AE-WTN+ on both seen and novel classes. Thirdly, we improve the efficiency of cross-layer network connections for very deep convolutional network in the context of image classification. This gives rise to a new convolutional network architecture called Deluge Networks (DelugeNets) which pass the output (feature activations) of any preceding composite network layer as direct inputs to all succeeding composite layers. For the incoming feature activations of preceding composite layers, the succeeding composite layers first apply rather lightweight cross-layer depthwise convolutional weights on them for feature aggregation. The heavy lifting is carried out by the last convolutional weight layer in each preceding composite layer that performs cross-channel convolutions. The proposed technique enables unimpeded information flow among many layers in very deep DelugeNets with great efficiency. Experiments on small-scale and large-scale datasets show that DelugeNets perform competitively to state-of-the-art deep convolutional networks at lower computational and parameter costs. Fourthly, we propose Stochastic Downsampling (SDPoint), a method to overcome a major limitation suffered by conventional deep convolutional networks -- their computational costs during inference are fixed and cannot be changed. SDPoint downsamples the feature maps at a randomly selected point (layer index) in the network hierarchy during training. The many downsampling configurations known as SDPoint instances each entail a unique downsampling point and downsampling ratio. The same convolutional network weights are shared across the different SDPoint instances that downsample feature maps differently. During inference, the computational cost of a network trained with SDPoint can be instantaneously adjusted to fit a given computational budget, by manually selecting an appropriate SDPoint instance. Additionally, sharing weights between different feature map scales provides significant regularization benefits, making the networks trained with SDPoint less sensitive to scales and more competent. In image classification experiments, SDPoint demonstrates significant cost-accuracy and computational advantages, over independently-trained (non-weight-shared) baselines with various inference costs.

Algorithms based on deep learning are taking off as solutions to several problems in numerous fields. Numerous deep learning architectures like convolutional neural networks, recurrent neural networks, deep belief networks, artificial neural networks and deep neural networks have been used in fields such as audio recognition, machine vision, processing of natural language, recognition of speech, social network filtering, computer vision, translating languages and bioinformatics. One such application of deep learning architectures is object detection. Object detection in satellite and aerial imagery undertakes an important role in a long list of applications and has allured a lot of attention which resulted in exceptional progress in recent years. It has abundant uses in various fields such as surveillance, military, national security, urban planning, environmental applications, traffic control and forestry. Several issues like varying size, background and orientation of the target object make the automatic detection of target in satellite imagery a challenging issue. Recent research in this field has given better speed and accuracy to the detection of the target object. With the rise in the data being generated by the Earth Observation programs, it is becoming much more imperative to find better and efficient ways to exploit this data by improving upon the existing Deep Learning research. In this paper, we bring forth a wide-ranging analysis of the recent advancements in methods of object detection based on deep learning to recognize the objects in satellite imagery.KeywordsDeep learningObject detectionSatellite imageryRemote sensingYou Only Look Once (YOLO)You Only Look Twice (YOLT)Single Shot Detector (SSD)Convolutional Neural Networks (CNN)Faster Region-based Convolutional Neural Networks (Faster RCNN)Hybrid Deep Convolutional Neural Network (HDNN)Satellite Imagery Multiscale Rapid Detection with Windowed Networks (SIMRDWN)

Computer vision is becoming an increasingly trendy word in the area of image processing. With the emergence of computer vision applications, there is a significant demand to recognize objects automatically. Deep CNN (convolution neural network) has benefited the computer vision community by producing excellent results in video processing, object recognition, picture classification and segmentation, natural language processing, speech recognition, and many other fields. Furthermore, the introduction of large amounts of data and readily available hardware has opened new avenues for CNN study. Several inspirational concepts for the progress of CNN have been investigated, including alternative activation functions, regularization, parameter optimization, and architectural advances. Furthermore, achieving innovations in architecture results in a tremendous enhancement in the capacity of the deep CNN. Significant emphasis has been given to leveraging channel and spatial information, with a depth of architecture and information processing via multi-path. This survey paper focuses mainly on the primary taxonomy and newly released deep CNN architectures, and it divides numerous recent developments in CNN architectures into eight groups. Spatial exploitation, multi-path, depth, breadth, dimension, channel boosting, feature-map exploitation, and attention-based CNN are the eight categories. The main contribution of this manuscript is in comparing various architectural evolutions in CNN by its architectural change, strengths, and weaknesses. Besides, it also includes an explanation of the CNN’s components, the strengths and weaknesses of various CNN variants, research gap or open challenges, CNN applications, and the future research direction.

Designing of VehiNet Using Convolutional Neural Networks and Deep Learning Techniques

Over the past few years, Convolutional Neural Networks (CNNs) have provided major breakthroughs in fields such as computer vision and natural language processing, resulting in a rise in the adoption of CNNs with increased levels of complexity. Consequently, the need for fast and power efficient processing of such networks has become critically important. Conventional hardware solutions, namely CPUs and GPUs, fail to address these requirements as CPUs are not suited for processing massively parallel multiply and accumulate (MAC) operations and GPUs are not power efficient. However, Field Programmable Gate Arrays (FPGAs) have emerged as a promising alternative due to their extensive programmability, ease of executing parallel operations and wide interfacing capabilities. In this paper, we have designed a hardware accelerator for speeding up the inference phase of LeNet–5 to enable faster classification of handwritten digits. We employ software quantization for ease of implementation on FPGAs, and partial pipelining to process the various layers of a typical CNN. Targeting the Xilinx Zynq-7000 SoC, we report a speedup improvement of at least 4.7x and power efficiency improvement of 32x compared to similar works.

Nowadays, deep learning has been widely used in various fields, and great breakthroughs have been made in computer vision, speech recognition and natural language processing. Based on the deep learning algorithm, this paper studies the general optimization technology of OpenCL based on FPGA in detail, and puts forward a targeted hardware design scheme, which has the characteristics of high performance, low power consumption and programmability. By analyzing the algorithm of forward calculation and reverse calculation in convolutional neural network, the hardware acceleration module is designed. Compared with CPU, GPU and other platforms, FPGA has obvious advantages in power consumption, which proves that the scheme proposed in this paper has good feasibility and accuracy.

This paper reviews the history and use of neural networks technology in great details. It starts by demonstrating the range of tasks neural networks can tackle, which are not limited to structural engineering or medical imaging. Then, it proceeds to outline the principles of artificial neural network models such as feedforward, recurrent and convolutional neural networks. The paper focuses on the training of neural networks and provides an account of the development of the backpropagation algorithm, optimization strategies, and regularization approaches. Moreover, a variety of case studies of neural networks applications are presented, for example, in climate modelling, natural language and speech recognition, computer vision and defect diagnosis techniques. In addition, the paper highlights the impact of the latest transformer models on self-supervised learning in natural language processing. The last part tackles ethical issues that are related to the use of neural networks such as fairness, bias and interpretability. All in all, this survey witnesses the growing importance of neural networks as a tool for enhancing the computational capabilities and addressing difficult problems.

In today's digital world, app stores have become an essential part of software distribution, providing customers with a wide range of applications and opportunities for software developers to showcase their work. This study elaborates on the importance of end-user feedback for software evolution. However, in the literature, more emphasis has been given to high-rating & popular software apps while ignoring comparatively low-rating apps. Therefore, the proposed approach focuses on end-user reviews collected from 64 low-rated apps representing 14 categories in the Amazon App Store. We critically analyze feedback from low-rating apps and developed a grounded theory to identify various concepts important for software evolution and improving its quality including user interface (UI) and user experience (UX), functionality and features, compatibility and device-specific, performance and stability, customer support and responsiveness and security and privacy issues. Then, using a grounded theory and content analysis approach, a novel research dataset is curated to evaluate the performance of baseline machine learning (ML), and state-of-the-art deep learning (DL) algorithms in automatically classifying end-user feedback into frequently occurring issues. Various natural language processing and feature engineering techniques are utilized for improving and optimizing the performance of ML and DL classifiers. Also, an experimental study comparing various ML and DL algorithms, including multinomial naive Bayes (MNB), logistic regression (LR), random forest (RF), multi-layer perception (MLP), k-nearest neighbors (KNN), AdaBoost, Voting, convolutional neural network (CNN), long short-term memory (LSTM), bidirectional long short term memory (BiLSTM), gated recurrent unit (GRU), bidirectional gated recurrent unit (BiGRU), and recurrent neural network (RNN) classifiers, achieved satisfactory results in classifying end-user feedback to commonly occurring issues. Whereas, MLP, RF, BiGRU, GRU, CNN, LSTM, and Classifiers achieved average accuracies of 94%, 94%, 92%, 91%, 90%, 89%, and 89%, respectively. We employed the SHAP approach to identify the critical features associated with each issue type to enhance the explainability of the classifiers. This research sheds light on areas needing improvement in low-rated apps and opens up new avenues for developers to improve software quality based on user feedback.

Insect taxonomy lies at the heart of many aspects of ecology, and identification tasks are challenging due to the enormous inter- and intraspecies variation of insects. Conventional methods used to study insect taxonomy are often tedious, time-consuming, labor intensive, and expensive, and recently, computer vision with deep learning algorithms has offered an alternative way to identify and classify insect images into their taxonomic levels. We designed the classification task according to the taxonomic ranks of insects—order, family, and genus—and compared the generalization of four state-of-the-art deep convolutional neural network (DCNN) architectures. The results show that different taxonomic ranks require different deep learning (DL) algorithms to generate high-performance models, which indicates that the design of an automated systematic classification pipeline requires the integration of different algorithms. The InceptionV3 model has advantages over other models due to its high performance in distinguishing insect order and family, which is having F1-score of 0.75 and 0.79, respectively. Referring to the performance per class, Hemiptera (order), Rhiniidae (family), and Lucilia (genus) had the lowest performance, and we discuss the possible rationale and suggest future works to improve the generalization of a DL model for taxonomic rank classification.

Abstract. Building change detection from remote sensing images is vital for many applications such as urban planning and dynamic monitoring, smart city construction, and geographical information census. In recent years, with the improvement of artificial intelligence and computer vision techniques, deep learning algorithms, especially convolutional neural networks (CNN), provide automatic detection and extraction methods. Unlike traditional approaches relying on shallow manual features, CNN can generate the deep semantic features by fusing spatial and spectral information, which is conducive to identifying building change regions. However, most deep CNN models directly fuse different level features and recover spatial details, which probably introduce redundant background information and noise from shallow layers. Considering the building’s multiscale, convolution operation with fixed receptive fields cannot obtain a strong global feature response to changed regions. To address the above problems, we develop a CNN framework for automatic building region change detection using dual-temporal high-resolution remote sensing images. To refine the shallow features, self-attention knowledge distillation strategies are introduced to FCN. Furthermore, we propose the multiscale feature change-aware module to integrate the globally changed information in different decoders. Finally, the model aggregates the multi-scale regional change information and outputs the prediction map. The results of comparative visual analysis and quantitative evaluation in two public datasets demonstrate that the proposed network model can improve the accuracy and efficiency of the automatic building change detection (85.69 IoU, 97.56 OA on the WHU dataset, and 83.72 IoU, 97.64 OA on the LEVIR-CD dataset).

The application of deep learning technology has increased rapidly in recent years. Technologies in deep learning increasingly emulate natural human abilities, such as knowledge learning, problem-solving, and decision-making. In general, deep learning can carry out self-training without repetitive programming by humans. Convolutional neural networks (CNNs) are deep learning algorithms commonly used in wide applications. CNN is often used for image classification, segmentation, object detection, video processing, natural language processing, and speech recognition. CNN has four layers: convolution layer, pooling layer, fully connected layer, and non-linear layer. The convolutional layer uses kernel filters to calculate the convolution of the input image by extracting the fundamental features. The pooling layer combines two successive convolutional layers. The third layer is the fully connected layer, commonly called the convolutional output layer. The activation function defines the output of a neural network, such as 'yes' or 'no'. The most common and popular CNN activation functions are Sigmoid, Tanh, ReLU, Leaky ReLU, Noisy ReLU, and Parametric Linear Units. The organization and function of the visual cortex greatly influence CNN architecture because it is designed to resemble the neuronal connections in the human brain. Some of the popular CNN architectures are LeNet, AlexNet and VGGNet.

Rice is one of the most cultivated cereal in Asian countries and Vietnam in particular. Good seed germination is important for rice seed quality, that impacts the rice production and crop yield. Currently, seed germination evaluation is carried out manually by experienced persons. This is a tedious and time-consuming task. In this paper, we present a system for automatic evaluation of rice seed germination rate based on advanced techniques in computer vision and machine learning. We propose to use U-Net - a convolutional neural network - for segmentation and separation of rice seeds. Further processing such as computing distance transform and thresholding will be applied on the segmented regions for rice seed detection. Finally, ResNet is utilized to classify segmented rice seed regions into two classes: germinated and non- germinated seeds. Our contributions in this paper are three-fold. Firstly, we propose a framework which confirms that convolutional neural networks are better than traditional methods for both segmentation and classification tasks (with F1- scores of 93.38\% and 95.66\% respectively). Secondly, we deploy successfully the automatic tool in a real application for estimating rice germination rate. Finally, we introduce a new dataset of 1276 images of rice seeds from 7 to 8 seed varieties germinated during 6 to 10 days. This dataset is publicly available for research purpose.

Convolutional Neural Network (CNN) is a deep learning algorithm which is widely used in image processing and pattern recognition due to its robustness to feature invariance. However, it is also computation-intensive that results in a bad real-time performance. Field Programmable Gate Arrays (FPGA) has good performance in energy-efficiency, flexibility of parallel processing and pipelined operations. Thus it is expected to be used for accelerating deep learning algorithm. In this research, a FPGA based system is developed to realize the real-time Hand Gesture Recognition. We train a designed CNN model with caffe framework and obtain the model's parameters on PC. Bilinear interpolation algorithm is used to adjust the size of the image captured by camera. Then we use FPGA to implement the inference process of Hand Gesture Recognition with obtained parameters by designing an accelerator using Xilinx SDx tools.

With the increase in computing power and the availability of large amounts of data, deep learning techniques, especially convolutional neural networks (CNNS) and recurrent neural networks (RNNS), have become important tools for processing complex signals. These methods show excellent performance in speech recognition, image processing, natural language processing and so on. In this paper, we explore the application of recurrent neural network (RNN) sequence algorithms in the field of computer digital signal processing, highlighting current artificial intelligence techniques and their capabilities in solving complex signal processing problems. First, the paper reviews the basic principles and development of deep learning and RNN sequence algorithms, highlighting the advances these advanced technologies have made in simulating the way the human brain processes information. The practical application and effect of RNN sequence algorithm in computer digital signal processing are demonstrated through experimental data. By comparing with traditional algorithms, we demonstrate the efficiency and accuracy of RNN in processing complex signals, such as speech recognition in noisy environments and real-time video data processing. The experimental data not only demonstrate the effectiveness of RNNS in this field, but also highlight the unique advantages of deep learning methods when dealing with large and high-dimensional data. Through these empirical studies, this paper aims to provide researchers and engineers with an in-depth understanding of the potential of RNNS in digital signal processing applications, and looks forward to the future development direction of artificial intelligence technology in this field.