Beyond Computational Formalism or, Architecture Matters

Aim: This study set out to determine how well AI approaches like Artificial Neural Networks (ANNs) and Deep Learning Neural Networks (DLNNs) might be used to forecast rainfall (DNN). These methods of weather prediction were tested and ranked in terms of their efficiency. Substances and Techniques: Group 1 uses a Deep Learning Neural Network (DNN) for analysis, whereas Group 2 uses an Artificial Neural Network (ANN) with a pre-test power (G-power) of 80% and an alpha error rate (Err) of 0.05 for sample test analysis. In all, 20 samples were identified; 10 from each category. According to the outcomes of the MATLAB simulations, the accuracy of the Deep Learning Neural Network (DNN) is 92.59%, while that of the Artificial Neural Network (ANN) is 95.68%. By means of SPSS statistical program, we find that the achieved accuracy ratio is 0.034 (p 0.05). This study concludes that the unique Artificial Neural Network (ANN) algorithm outperforms the Deep Learning Neural Network (DNN) method in predicting rainfall in the context of cutting-edge weather forecasting.

This study presents a fault diagnosis system for vehicle heating, ventilation and air conditioning (HVAC) acoustic signal with various feature extractions in deep learning neural network. Traditionally, sound used for fault diagnosis or signal classification is observed the difference of energy in time or frequency domains. Unfortunately, the frequency smearing effect often arises in some critical conditions. In the present study, discrete wavelet transform (DWT) and wavelet packet transform (WPT) are proposed in fault diagnosis. Meanwhile, when using mechanical learning methods, the data are relatively large, in order to reduce the amount of data, DWT and WPT low-frequency decomposition could be used to improve the performance. Furthermore, the signal characteristics more comprehensive, this study attempts to use the feature extraction method of wavelet packet conversion to improve the signal characteristics. In the experiment process, the operation state of the blade blower in the vehicle air conditioner, four different faults were designed, test database was established through sound to classify, and identify the data using deep neural networks to achieve the purpose of blower fault diagnosis. In data analysis, the original signal is presented through wavelet packet decomposition and discrete packet conversion technology, compared with traditional time and frequency domain signals to explore the identification rate, identification speed and related issues. Experimental results show that using WPT combined with deep neural networks have good fault diagnosis and discrimination capabilities, training, and identification time is shorter than time-frequency domain signals.

Over the last decade, the deep neural networks are a hot topic in machine learning. It is breakthrough technology in processing images, video, speech, text and audio. Deep neural network permits us to overcome some limitations of a shallow neural network due to its deep architecture. In this paper we investigate the nature of unsupervised learning in restricted Boltzmann machine. We have proved that maximization of the log-likelihood input data distribution of restricted Boltzmann machine is equivalent to minimizing the cross-entropy and to special case of minimizing the mean squared error. Thus the nature of unsupervised learning is invariant to different training criteria. As a result we propose a new technique called "REBA" for the unsupervised training of deep neural networks. In contrast to Hinton's conventional approach to the learning of restricted Boltzmann machine, which is based on linear nature of training rule, the proposed technique is founded on nonlinear training rule. We have shown that the classical equations for RBM learning are a special case of the proposed technique. As a result the proposed approach is more universal in contrast to the traditional energy-based model. We demonstrate the performance of the REBA technique using wellknown benchmark problem. The main contribution of this paper is a novel view and new understanding of an unsupervised learning in deep neural networks.

The article deals with the problem of path planning in a two-dimensional environment based on deep learning neural networks. Deep neural networks require large amounts of data and place high computational requirements on computing tools. The lack of sufficient data leads to a decrease in the accuracy of the neural network, and high computational requirements at the learning stage limit the use of this technology in engineering practice. In this paper, the forms of representation of the environment for the input of a neural network are studied. Vector form allows to reduce the amount of information supplied to the input of a neural network, but it leads to the need to use more complex neural networks. In this article, a combined form of representation is proposed, including a vector global and local map layout. The vector part of the map includes the position of the robot, the position of the target point and a description of obstacles. The local raster map describes the area closest to the robot. Using numerical research, the effectiveness of this form of data representation for a precise neural network is shown, compared with the raster representation of the map. In this article, two structures of neural networks are studied, one of which uses 8 possible directions of movement, and the other uses 3 possible directions of movement. It is shown that when using 3 possible directions, the cycling of trajectories planned by the neural network is eliminated, which leads to an increase in accuracy.

To construct the risk prediction model of acute exacerbation of chronic obstructive pulmonary disease (AECOPD) and verify its effectiveness based on deep learning and back propagation algorithm neural network (BP neural network). Based on the relevant data of 1 326 patients with chronic obstructive pulmonary disease (COPD) in the team's previous clinical study, the acute exacerbation, and its risk factors during the stable period and 6 months of follow-up were recorded and analyzed. Combined with previous clinical research data and expert questionnaire results, the independent risk factors of AECOPD after screening and optimization by multivariate Logistic regression including gender, body mass index (BMI) classification, number of acute exacerbation, duration of acute exacerbation and forced expiratory volume in one second (FEV1) were used to build the BP neural network by Python 3.6 programming language and Tensorflow 1.12 deep learning framework. The patients were randomly selected according to the ratio of 4:1 to generate the training group and the test group, of which, the training group had 1 061 sample data while the test group had 265 pieces of sample data. The training group was used to establish the prediction model of neural network, and the test group was used for back-substitution test. When using the training group data to construct the neural network model, the training group was randomly divided into training set and verification set according to the ratio of 4:1. There were 849 training samples in the training set and 212 verification samples in the verification set. The optimal model was screened by adjusting the parameters of the neural network and combining the area under the receiver operator characteristic curve (AUC), and the sample data of the test group was substituted into the model for verification. The independent risk factors including gender, BMI classification, number of acute exacerbation, duration of acute exacerbation and FEV1 were collected from the team's previous clinical research, and the AECOPD risk prediction model was constructed based on deep learning and BP neural network. After 10 000 training sessions, the accuracy of the AECOPD risk prediction model in the validation set of the training group was 83.09%. When the number of training times reached 8 000, the accuracy basically tended to be stable and the prediction ability reached the upper limit. The AECOPD risk prediction model trained for 10 000 times was used to predict the risk of the validation set data, and the receiver operator characteristic curve (ROC curve) analysis showed that the AUC was 0.803. When using this model to predict the risk of the data of the test group, the accuracy rate was 81.69%. The risk prediction model based on deep learning and BP neural network has a medium level of prediction efficiency for acute exacerbation within 6 months in COPD patients, which can evaluate the risk of AECOPD and assist the clinic in making accurate treatment decisions.

In recent years, growing interest in deep learning neural networks has raised a question on how they can be used for effective processing of high-dimensional datasets produced by hyperspectral imaging (HSI). HSI, traditionally viewed as being within the scope of remote sensing, is used in non-invasive substance classification. One of the areas of potential application is forensic science, where substance classification on the scenes is important. An example problem from that area—blood stain classification—is a case study for the evaluation of methods that process hyperspectral data. To investigate the deep learning classification performance for this problem we have performed experiments on a dataset which has not been previously tested using this kind of model. This dataset consists of several images with blood and blood-like substances like ketchup, tomato concentrate, artificial blood, etc. To test both the classic approach to hyperspectral classification and a more realistic application-oriented scenario, we have prepared two different sets of experiments. In the first one, Hyperspectral Transductive Classification (HTC), both a training and a test set come from the same image. In the second one, Hyperspectral Inductive Classification (HIC), a test set is derived from a different image, which is more challenging for classifiers but more useful from the point of view of forensic investigators. We conducted the study using several architectures like 1D, 2D and 3D convolutional neural networks (CNN), a recurrent neural network (RNN) and a multilayer perceptron (MLP). The performance of the models was compared with baseline results of Support Vector Machine (SVM). We have also presented a model evaluation method based on t-SNE and confusion matrix analysis that allows us to detect and eliminate some cases of model undertraining. Our results show that in the transductive case, all models, including the MLP and the SVM, have comparative performance, with no clear advantage of deep learning models. The Overall Accuracy range across all models is 98–100% for the easier image set, and 74–94% for the more difficult one. However, in a more challenging inductive case, selected deep learning architectures offer a significant advantage; their best Overall Accuracy is in the range of 57–71%, improving the baseline set by the non-deep models by up to 9 percentage points. We have presented a detailed analysis of results and a discussion, including a summary of conclusions for each tested architecture. An analysis of per-class errors shows that the score for each class is highly model-dependent. Considering this and the fact that the best performing models come from two different architecture families (3D CNN and RNN), our results suggest that tailoring the deep neural network architecture to hyperspectral data is still an open problem.

A point cloud deep neural network metamodel method for aerodynamic prediction

<span>One of the most significant and daunting activities in today's world is temperature prediction. The meteorologists traditionally predict temperature via some statistical models aimed to forecast the fluctuations that might have happened to atmospheric parameters such as temperature, humidity, etc. The main objective of this paper is to build an intelligent temperature prediction model of Erbil city in KRG/ Iraq based on a historical dataset from 1992 to 2016 in each year there are twelve months’ average temperature readings from (January to December). Hence to resolve this prediction problem an up-to-date deep learning neural network has been used, the network model is based on <span id="docs-internal-guid-850bd062-7fff-c6c8-9146-ba6427eb24e0" style="font-size: 9pt; font-family: 'Times New Roman'; color: #000000; background-color: transparent; font-weight: 400; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">long short-term memory</span> (LSTM) as an artificial recurrent neural network (RNN) architecture which employed to estimate the future average temperature. The implementing model uses the dataset from real-time 30 weather stations deployed in the area of the city. The prediction performance of the proposed recurrent neural network model has been compared with some state of art algorithms like Adeline neural network, Autoregressive neural network (NAR), and <span id="docs-internal-guid-14d37b98-7fff-0f76-848f-ad9f89224f77" style="font-size: 9pt; font-family: 'Times New Roman'; color: #000000; background-color: transparent; font-weight: 400; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> generalized regression neural network</span> (GRNN). The results show that the proposed model based on deep learning gives minimum prediction error.</span>

Aim. To study various network architecture options for implementing an electronic differential system in an electric vehicle. Materials and Methods. The study primarily used comparative analysis to identify the most rational neural network (NN) architecture for processing numerical data structured as arrays. Results. The analysis revealed that a deep learning neural network is the most effective choice. For future developments, and after experimental confirmation, a recurrent neural network could also be a viable option. Conclusion. The study confirmed that achieving the desired goal is not feasible using convolutional neural networks, large language models, random vector functional communication networks and radial-basis functional NNs.

In order to explore the application of the image recognition model based on multi-stage convolutional neural network (MS-CNN) in the deep learning neural network in the intelligent recognition of commodity images and the recognition performance of the method, in the study, the features of color, shape, and texture of commodity images are first analyzed, and the basic structure of deep convolutional neural network (CNN) model is analyzed. Then, 50,000 pictures containing different commodities are constructed to verify the recognition effect of the model. Finally, the MS-CNN model is taken as the research object for improvement to explore the influence of label errors (p = 0.03, 0.05, 0.07, 0.09, 0.12) with different parameter settings and different probabilities (size of convolutional kernel, Dropout rate) on the recognition accuracy of MS-CNN model, at the same time, a CIR system platform based on MS-CNN model is built, and the recognition performance of salt and pepper noise images with different SNR (0, 0.03, 0.05, 0.07, 0.1) was compared, then the performance of the algorithm in the actual image recognition test was compared. The results show that the recognition accuracy is the highest (97.8%) when the convolution kernel size in the MS-CNN model is 2*2 and 3*3, and the average recognition accuracy is the highest (97.8%) when the dropout rate is 0.1; when the error probability of picture label is 12%, the recognition accuracy of the model constructed in this study is above 96%. Finally, the commodity image database constructed in this study is used to identify and verify the model. The recognition accuracy of the algorithm in this study is significantly higher than that of the Minitch stochastic gradient descent algorithm under different SNR conditions, and the recognition accuracy is the highest when SNR = 0 (99.3%). The test results show that the model proposed in this study has good recognition effect in the identification of commodity images in scenes of local occlusion, different perspectives, different backgrounds, and different light intensity, and the recognition accuracy is 97.1%. To sum up, the CIR platform based on MS-CNN model constructed in this study has high recognition accuracy and robustness, which can lay a foundation for the realization of subsequent intelligent commodity recognition technology.

Background: Retinal fundus images, obtained from routine eye screenings, contain important information, not only concerning microvascular lesions of the retina, but potentially also of the progression and manifestation of various other diabetes complications. Recent advances in the field of machine learning, so-called deep learning neural networks (DNN), make it possible to automatically detect retinal lesions from fundus photos. Our overall aim is to develop a DNN to detect retinal vascular lesions. In this abstract, we describe a pre-processing step consisting of application of a DNN. Methods: For each eye, 5 retinal images of parts of the eye are taken, and collected into a joint mosaic image, used for image grading. There is no automated labelling of the images to identify left or right eyes. To distinguish left- from right eye images, we used a set of 1,916 unique, high-definition retina mosaic fundus images from diabetes patients followed at Steno Diabetes Center Copenhagen (SDCC) between 2003-2017. We trained and validated a DNN, using 964 left-eye (training set: 713, validation: 251) and 952 right-eye images (training set: 706, validation: 246). A DNN is a method that finds the optimal algorithm for relating a large set of input (mosaic fundus images) to output (left or right eye). The DNN used was based on the InceptionV3 architecture and run for 100 epochs with 100 gradient descent steps pr. epoch. Results: The DNN classification obtained a validation AUC-ROC of 0.92 and a validation accuracy of 0.86. Conclusions: The DNN successfully distinguished left-eye from right-eye images. We plan to use the DNN to identify right- and left-eye images in the SDCC image database comprising ∼ 80,000 left-eye and right-eye images. We will subsequently develop a DNN for automated detection of referable diabetic retinopathy of retinal fundus images, for use in the clinic. Disclosure L. Díaz: None. D. Vistisen: Stock/Shareholder; Self; Novo Nordisk A/S. M.E. Jørgensen: Research Support; Self; Amgen, AstraZeneca, Boehringer Ingelheim Pharmaceuticals, Inc., Sanofi-Aventis. Stock/Shareholder; Self; Novo Nordisk A/S. M. Valerius: None. J.N. Hajari: None. H. Lund-Andersen: None. S. Byberg: None.

Predicting aerodynamic forces on bluff bodies remains to be a challenging task due to the unpredictable flow behavior, specifically at higher Reynolds numbers. Experimental approaches to determine aerodynamic coefficients could be costly and time consuming. In the meantime, use of numerical techniques could also require a considerable computational cost and time depending on complexity of the flow behavior. The research focusses on developing an effective deep learning technique to predict aerodynamic force coefficients acting on elliptical bluff bodies for a given aspect ratio and given flow condition. Collecting data for drag and lift coefficients of several aspect ratios for flow conditions starting from onset of vortex shredding to verge of subcritical region is conducted by an accurate full order model. The specified region will provide a transient flow behavior and thus lift coefficient will be represented in terms of root mean square value and drag coefficient in terms of a mean value. With variations in flow behavior and vortex shredding frequencies, it requires to select an appropriate turbulence model, optimum discretization of fluid domain and time step to obtain an accurate result. Flow simulations are conducted primarily using Unsteady Reynolds Averaged Navier-Stokes Equations (URANS) model and Detached Eddy Simulations (DES) model. Effectiveness in using different turbulence models for specified flow regimes are also explored in comparison to available experimental results. At lower Reynolds numbers, aerodynamic force coefficients for a specified body will only depend on Reynolds number. But after a certain specific Reynolds number, aerodynamic forces are dependent on the Mach number in addition to Reynolds number. Therefore, for higher Reynolds numbers, aerodynamic force coefficients are recorded for multiple Mach numbers with same Reynolds number and will be fed to the neural network. With the development of the machine learning and neural network modelling, many of the fields have nourished and created effective and efficient technologies to ease complex functions and activities. Our goal is to ease the complexity in the computational fluid dynamic field with a deep neural network tool created to predict drag and lift coefficient of elliptical bluff bodies for a given aspect ratio with an acceptable accuracy level. Researchers have developed deep neural network tools to predict various flow conditions and have succeeded with sufficient accuracy and a satisfying reduction of computational cost. In our proposed deep learning neural network, we have chosen to model the network with inputs as the geometry setup and the flow conditions with validated drag and lift coefficients. The model will extract the necessary flow features into filters with the convolution operation performed on the inputs. Our main directive is to create a deep learned neural network tool to predict the target values within an acceptable range of accuracy while minimizing the computation cost.

At present, deep learning is becoming one of the most popular approach to creation of the artificial intelligences systems such as speech recognition, natural language processing, computer vision and so on. The paper presents a historical overview of deep learning in neural networks. The model of the artificial neural network is described as well as the learning algorithms for neural networks including the error backpropagation algorithm, which is used to train deep neural networks. The development of neural networks architectures is presented including neocognitron, autoencoders, convolutional neural networks, restricted Boltzmann machine, deep belief networks, long short-term memory, gated recurrent neural networks, and residual networks. Training deep neural networks with many hidden layers is impeded by the vanishing gradient problem. The paper describes the approaches to solve this problem that provide the ability to train neural networks with more than hundred layers. An overview of popular deep learning libraries is presented. Nowadays, for computer vision tasks convolutional neural networks are utilized, while for sequence processing, including natural language processing, recurrent networks are preferred solution, primarily long short-term memory networks and gated recurrent neural networks.

Collaborative filtering and deep learning based recommendation system for cold start items

Traffic prediction plays an important role in the intelligent transportation system (ITS), because it can increase people’s travel convenience. Despite the deep neural network has been widely used in the field of traffic prediction, literature surveys of such methods and data categories are rare. In this paper, we have a summary of traffic forecasting from data, methods and challenges. Firstly, we are according to the difference of in spatio-temporal dimensions, divide the data into three types, including the spatio-temporal static data, spatial static time dynamic data, and spatio-temporal dynamic data. Secondly, we explore three significant neural networks of deep learning in traffic prediction, including the convolutional neural network (CNN), the recurrent neural network (RNN), and the hybrid neural networks models. These methods are used in many aspects of traffic prediction, including road traffic accidents forecast, road traffic flow prediction, road traffic speed forecast, and road traffic congestion forecast introduced. Finally, we provide a discussion of some current challenges and development prospects. KeywordsDeep neural networkTraffic forecastingSpatio-temporal data