GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices Based on Fine-Grained Structured Weight Sparsity.

The study presents the results of the development and testing of deep learning neural network architectures, which demonstrate high accuracy rates in classifying neurophysiological data, in particular hemodynamic brain activation patterns obtained by functional near-infrared spectroscopy, during solving mathematical problems on spatial-numerical associations. The analyzed signal represents a multidimensional time series of oxyhemoglobin and deoxyhemoglobin dynamics. Taking the specificity of the fNIRS signal into account, a comparative analysis of 2 types of neural network architectures was carried out: (1) architectures based on recurrent neural networks: recurrent neural network with long short-term memory, recurrent neural network with long short-term memory with fully connected layers, bidirectional recurrent neural network with long short-term memory, convolutional recurrent neural network with long short-term memory; (2) architectures based on convolutional neural networks with 1D convolutions: convolutional neural network, fully convolutional neural network, residual neural network. Trained long short-term memory recurrent neural network architectures showed worse results in accuracy in comparison with 1D convolutional neural network architectures. Residual neural network (model_Resnet) demonstrated the highest accuracy rates in three experimental conditions more than 88% in detecting age-related differences in brain activation during spatial-numerical association tasks considering the individual characteristics of the respondents’ signal.

Mobile devices are becoming an important carrier for deep learning tasks, as they are being equipped with powerful, high-end mobile CPUs and GPUs. However, it is still a challenging task to execute 3D Convolutional Neural Networks (CNNs) targeting for real-time performance, besides high inference accuracy. The reason is more complex model structure and higher model dimensionality overwhelm the available computation/storage resources on mobile devices. A natural way may be turning to deep learning weight pruning techniques. However, the direct generalization of existing 2D CNN weight pruning methods to 3D CNNs is not ideal for fully exploiting mobile parallelism while achieving high inference accuracy. This paper proposes RT3D, a model compression and mobile acceleration framework for 3D CNNs, seamlessly integrating neural network weight pruning and compiler code generation techniques. We propose and investigate two structured sparsity schemes i.e., the vanilla structured sparsity and kernel group structured (KGS) sparsity that are mobile acceleration friendly. The vanilla sparsity removes whole kernel groups, while KGS sparsity is a more fine-grained structured sparsity that enjoys higher flexibility while exploiting full on-device parallelism. We propose a reweighted regularization pruning algorithm to achieve the proposed sparsity schemes. The inference time speedup due to sparsity is approaching the pruning rate of the whole model FLOPs (floating point operations). RT3D demonstrates up to 29.1x speedup in end-to-end inference time comparing with current mobile frameworks supporting 3D CNNs, with moderate 1%~1.5% accuracy loss. The end-to-end inference time for 16 video frames could be within 150 ms, when executing representative C3D and R(2+1)D models on a cellphone. For the first time, real-time execution of 3D CNNs is achieved on off-the-shelf mobiles.

Recently the development of deep learning has been propelling the sheer growth of vision and speech applications on lightweight embedded and mobile systems. However, the limitation of computation resource and power delivery capability in embedded platforms is recognized as a significant bottleneck that prevents the systems from providing real-time deep learning ability, since the inference of deep convolutional neural networks (CNNs) and recurrent neural networks (RNNs) involves large quantities of weights and operations. Particularly, how to provide quality-of-services (QoS)-guaranteed neural network inference ability in the multitask execution environment of multicore SoCs is even more complicated due to the existence of resource contention. In this article, we present a novel deep neural network architecture, MV-Net, which provides performance elasticity and contention-aware self-scheduling ability for QoS enhancement in mobile computing systems. When the constraints of QoS, output accuracy, and resource contention status of the system change, MV-Net can dynamically reconfigure the corresponding neural network propagation paths and thus achieves an effective tradeoff between neural network computational complexity and prediction accuracy via approximate computing. The experimental results show that (1) MV-Net significantly improves the performance flexibility of current CNN models and makes it possible to provide always-guaranteed QoS in a multitask environment, and (2) it satisfies the quality-of-results (QoR) requirement, outperforming the baseline implementation significantly, and improves the system energy efficiency at the same time.

Feature data processing: Making medical data fit deep neural networks

In this paper, alternative direction finding methods have been proposed using deep learning techniques. Firstly, Regeression and Classification models have created by using Convolutional Neural Networks (CNNs). In the second Convolutional Neural Networks and Recurrent Neural Networks (RNNs) have been utilized in the proposed methods. Despite having vast amount of direction finding studies, utilization of neural networks is scarce in literature and past works mostly only includes usage of CNNs. In this study, direction finding is performed by learning signals reaching multiple antenna arrays by networks. Created neural networks have been fed with different data formats and their performances against noisy and no-noise data have been shown. In addition, comparative analysis of the developed models were made in the similar Signal-to-Noise Ratio (SNR) range with the subspace based MUSIC algorithm, which is frequently used in direction finding.

This paper studies the problem of Chinese license plate recognition (CLPR) in complex natural scenes. Aiming at the problem that the license plate is difficult to be segmented due to its tilt and distortion caused by the process of license plate photographing, an improved Convolutional Recurrent Neural Network (CRNN) deep neural network model for license plate recognition was proposed, which combined with Deep Convolutional Neural Networks (DCNN), Recurrent Neural Networks (RNN), Spatial Transformer Networks (STN) and Connectionist Temporal Classification (CTC) model. The recognition process does not need to be segmented, which avoids the error caused by plate segmentation and affects the recognition accuracy. The convolutional layers selects DenseNet, the latest CNN network architecture, to improve the feature extraction accuracy. The space conversion layers adds STN to affine the license plate, which improves the recognition accuracy. In order to evaluate the performance, Chinese City Parking Dataset (CCPD) is used to test, and the validity of the method is proved.

The natural language processing (NLP) area has been magically transformed by the Deep Neural Network (DNN). The two variations of Neural Networks, Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) can handle the different NLP tasks effectively. CNN is based on feature extraction using N-grams at higher levels. The RNN can be effectively used to model the sequential information. Choosing a neural technique for the various NLP tasks is always a challenge. The paper focuses on the evaluating the performance of the two DNN techniques CNN and RNN for the various NLP tasks, so that appropriate technique can be selected.

This paper focuses on the design and comparison of different deep neural networks for the real-time prediction of locomotor intentions by using data from inertial measurement units. The deep neural network architectures are convolutional neural networks, recurrent neural networks, and convolutional recurrent neural networks. The input to the architectures are features in the time domain, which have been derived either from one inertial measurement unit placed on the upper right leg of ten healthy subjects, or two inertial measurement units placed on both the upper and lower right leg of ten healthy subjects. The study shows that a WaveNet, i.e., a full convolutional neural network, achieves a peak F1-score of 87.17% in the case of one IMU, and a peak of 97.88% in the case of two IMUs, with a 5-fold cross-validation.

This paper proposes a novel approach for scanned document categorization using a deep neural network architecture. The proposed approach leverages the strengths of both convolutional neural networks (CNNs) and recurrent neural networks (RNNs) to extract features from the scanned documents and model the dependencies between words in the documents. The pre-processed documents are first fed into a CNN, which learns and extracts features from the images. The extracted features are then passed through an RNN, which models the sequential nature of the text. The RNN produces a probability distribution over the predefined categories, and the document is classified into the category with the highest probability. The proposed approach is evaluated on a dataset of scanned documents, where each document is categorized into one of four predefined categories. The experimental results demonstrate that the proposed approach achieves high accuracy and outperforms existing methods. The proposed approach achieves an overall accuracy of 97.3%, which is significantly higher than the existing methods' accuracy. Additionally, the proposed approach's performance was robust to variations in the quality of the scanned documents and the OCR accuracy. The contributions of this paper are twofold. Firstly, it proposes a novel approach for scanned document categorization using deep neural networks that leverages the strengths of CNNs and RNNs. Secondly; it demonstrates the effectiveness of the proposed approach on a dataset of scanned documents, highlighting its potential applications in various domains, such as information retrieval, data mining, and document management. The proposed approach can help organizations manage and analyze large volumes of data efficiently.

Maxout neurons for deep convolutional and LSTM neural networks in speech recognition

In this digital era, there is an exponential growth of text-based content in the electronic world. Data as texts exist in the form of documents, social media posts on Facebook, Twitter, etc., logs, sensor data, and emails. Twitter is a social platform where users express their views on various aspects in a day to day life. Twitter produces over 500 million tweets daily that is 6000 tweets per second. Twitter data is, by definition, very noisy and unstructured in nature. Text classifications based on the machine learning techniques have problems like poor generalization ability and sparsity dimension explosion. Classifiers based on deep learning techniques are implemented to improve accuracy to overcome shortcomings of machine learning techniques and to avoid feature extraction processes and have high prediction accuracy and strong learning ability. In this work, the classification of tweets is performed on Covid-19 dataset by implementing deep learning techniques namely Convolution Neural Network (CNN), Recurrent Neural Network (RNN), Recurrent Convolution Neural Network (RCNN), Recurrent Neural Network with Long Short Term Memory (RNN+LSTM), and Bidirectional Long Short Term Memory with Attention (BI-LSTM + Attention). The algorithms are implemented using two-word embedding techniques namely Global Vectors for Word Representation (GloVe) and Word2Vec. RNN with Bidirectional LSTM model has performed better than all the classifiers considered. It has classified the text with an accuracy of 93% and above when used with GloVe and Word2Vec.

A floating foundation vibration reduction system with an air spring as a vibration isolation element has been widely used in the foundation of large ultra-precision instruments. With the excellent vibration isolation performance of the air spring, its complex dynamic nonlinear behavior is always a research difficulty. In this paper, a dynamic model of a floating foundation vibration reduction system based on the restoring force of air spring is derived. The structure of the recurrent convolution neural network (RCNN) is proposed based on combining the working characteristics of a convolution neural network and a long short-term memory neural network, and the dynamic model of a floating foundation vibration reduction system is established with the restoring force of the air spring calculated by the RCNN as the input. Finally, a test experiment was designed to compare the dynamic characteristics of the traditional numerical model and three deep neural network models in the floating foundation. The results show that convolution neural network, long short-term memory, and RCNN models could predict the vibration response of floating foundation vibration reduction system, and the RCNN model had better performance for a floating foundation.

This research aims to develop artificial intelligence (AI) algorithms in the context of facial recognition with a focus on increasing accuracy in difficult environmental conditions. Although facial recognition technology has made great progress, challenges such as poor lighting, variations in facial expressions, and head rotation are still problems that must be overcome. The research methodology involved collecting a wide dataset covering a wide variety of faces under various environmental conditions. This data is then processed and its features are extracted using computer image processing techniques. Furthermore, several deep neural network architectures, such as Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), were developed, trained, and evaluated for face recognition tasks. The expected result is the development of an AI algorithm that is able to overcome challenges in facial recognition with higher accuracy than existing methods. In particular, significant improvements in facial recognition accuracy are expected especially under low lighting conditions and variations in facial expressions. This research has a major impact in a variety of security applications, such as border surveillance, building access control, and corporate security. With higher facial recognition accuracy, security risks can be significantly reduced, resulting in safer and more efficient security solutions. In conclusion, this research aims to bring innovation in facial recognition technology through advanced AI approaches, with the potential to improve security in various contexts.

paper, we focus on exploring the relationship between weave patterns and their mechanical properties in woven fiber composites through Machine Learning. Specifically, we explore the interactions between woven architectures and in-plane stiffness properties through Deep Convolutional Neural Network (DCNN) and Generative Adversarial Network (GAN). Our research is important for exploring how woven composite’s pattern is related to its mechanical properties and accelerating woven composite design as well as optimization. We focus on two tasks: (1) Stiffness prediction: Predicting in-plane stiffness properties for given weave patterns. Our DCNN extracts high-level features through several convolutional and fully connected layers to determine the final predictions. (2) Weave pattern prediction: Predicting weave patterns for target stiffness properties, which can be treated as the reverse task of the first one. Due to many-to-one mapping between weave patterns and the composite properties, we utilize a Decoder Neural Network as our baseline model and compare its performance with GAN and Genetic Algorithm. We represent the weave patterns as 2D checkerboard models and use finite element analysis (FEA) to determine in-plane stiffness properties, which serve as input data for our ML framework. We show that: (1) for stiffness prediction, DCNN can predict stiffness values for a given weave pattern with relatively high accuracy (above 93%); (2) for weave pattern prediction, the GAN model gives the best prediction accuracy (above 92%) while Decoder Neural Network has the best time efficiency. HAOTIAN FENG

Text recognition in natural scene images is a challenging problem in computer vision. Different than the optical character recognition (OCR), text recognition in natural scene images is more complex due to variations in text size, colors, fonts, orientations, complex backgrounds, occlusion, illuminations and uneven lighting conditions. In this paper, we propose a segmentation-free method based on a deep convolutional recurrent neural network to solve the problem of cursive text recognition, particularly focusing on Urdu text in natural scenes. Compared to the non-cursive scripts, Urdu text recognition is more complex due to variations in the writing styles, several shapes of the same character, connected text, ligature overlapping, stretched, diagonal and condensed text. The proposed model gets a whole word image as an input without pre-segmenting into individual characters, and then transforms into the sequence of the relevant features. Our model is based on three components: a deep convolutional neural network (CNN) with shortcut connections to extract and encode the features, a recurrent neural network (RNN) to decode the convolutional features, and a connectionist temporal classification (CTC) to map the predicted sequences into the target labels. To increase the text recognition accuracy further, we explore deeper CNN architectures like VGG-16, VGG-19, ResNet-18 and ResNet-34 to extract more appropriate Urdu text features, and compare the recognition results. To conduct the experiments, a new large-scale benchmark dataset of cropped Urdu word images in natural scenes is developed. The experimental results show that the proposed deep CRNN network with shortcut connections outperform than other network architectures. The dataset is publicly available and can be downloaded from <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://data.mendeley.com/datasets/k5fz57zd9z/1</uri> .