Toward accurate platform-aware performance modeling for deep neural networks

Taiwan, being located on a path in the west Pacific Ocean where typhoons often strike, is often affected by typhoons. The accompanying strong winds and torrential rains make typhoons particularly damaging in Taiwan. Therefore, we aimed to establish an accurate wind speed prediction model for future typhoons, allowing for better preparation to mitigate a typhoon’s toll on life and property. For more accurate wind speed predictions during a typhoon episode, we used cutting-edge machine learning techniques to construct a wind speed prediction model. To ensure model accuracy, we used, as variable input, simulated values from the Weather Research and Forecasting model of the numerical weather prediction system in addition to adopting deeper neural networks that can deepen neural network structures in the construction of estimation models. Our deeper neural networks comprise multilayer perceptron (MLP), deep recurrent neural networks (DRNNs), and stacked long short-term memory (LSTM). These three model-structure types differ by their memory capacity: MLPs are model networks with no memory capacity, whereas DRNNs and stacked LSTM are model networks with memory capacity. A model structure with memory capacity can analyze time-series data and continue memorizing and learning along the time axis. The study area is northeastern Taiwan. Results showed that MLP, DRNN, and stacked LSTM prediction error rates increased with prediction time (1–6 hours). Comparing the three models revealed that model networks with memory capacity (DRNN and stacked LSTM) were more accurate than those without memory capacity. A further comparison of model networks with memory capacity revealed that stacked LSTM yielded slightly more accurate results than did DRNN. Additionally, we determined that in the construction of the wind speed prediction model, the use of numerically simulated values reduced the error rate approximately by 30%. These results indicate that the inclusion of numerically simulated values in wind speed prediction models enhanced their prediction accuracy.

Ellipsoidal basis function(EBF) can make the partition and limitary of input space. Compared with the Guassian function of radial basis function(RBF) neural network, the EBF can make the partition of input space more specific, which has the higher capability of pattern recognition. However, the neural network has a common problem of training the weight and threshold. The evolution of genetic algorithm(GA) can maximumly optimize the training time of neural network. In this paper, a new method based on GA-EBF neural network was proposed. The simulation experiment shows that the proposed method has a higher rate of fault diagnosis than that of RBF neural network. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>

In the deregulated power systems, the available transfer capability (ATC) should be known for secure and reliable operation. ATC mainly depends on load for a particular transaction. Due to complex nature of load, it is better if the ATC estimator is able to handle this complex nature. This paper presents fully complex-valued radial basis function (FC-RBF) neural network approach for ATC estimation for bilateral transaction under normal condition. The training patterns for neural network are generated using differential evolution algorithm (DEA). The important feature of the proposed method is the use of input reduction techniques to improve the performance of the developed network. Differential evolution feature selection (DEFS) technique is proposed to reduce the complexity and training time of neural network. The proposed method is tested on IEEE 118 bus system, and results are compared with DEA and repeated power flow (RPF) results. The test results show that the proposed method reduces the training time and it is suitable for online application.

Nowadays , nonlinear time series and artificial neural networks (ANN) models are used for forecasting in the field of business, agriculture and soon. Recent studies have shown, ANN have been successfully used for forecasting of financial and agriculture data series The classical methods used for time series prediction like Box-Jenkins or ARIMA assumes that there is a linear relationship between inputs and outputs. ANN have more advantages that can approximate to model both linear and nonlinear structures in time series, they are not able to handling both structures equally well. The autoregressive integrated moving average (ARIMA) model and two ANN models namely, Radial basis function neural networks (RBFNN), and Elman recurrent neural networks (ERNN) methods were applied to Hyderabad airport traffic data. The data obtained for 15 years from 2002–2003 to 2016–2017 about domestic and international passenger of International Airport of Hyderabad, India. In this research paper, we compared the performances of ARIMA, RBFNN and ERNN were based on three measures: mean absolute error (MAE), mean absolute percentage error (MAPE), and root mean square error (RMSE). The results showed that RBFNN obtained the smallest MAE, MAPE and RMSE in both the modeling and forecasting processes. The performances of the three models ranked in ascending order were: ARIMA, ERNN and the RBFNN model. Keywords: T ime series, forecasting, artificial neural networks, ARIMA models, radial basis function neural networks, and Elman recurrent neural networks Cite this Article R. Ramakrishna, Berhe Aregay, Tewodros Gebregergs. The Comparison in Time Series Forecasting of Air Traffic Data by Autoregressive Integrated Moving Average Model, Radial Basis Function and Elman Recurrent Neural Networks. Research & Reviews: Journal of Statistics . 2018; 7(3): 75–90p.

The work is devoted to resolving the contradiction between the accuracy of modelling nonlinear dynamics and the speed of model construction under conditions of limited computing resources. The purposeof the work is to reduce the time for building time delay neural networks while ensuring a given accuracy in the tasks of identifying nonlinear dynamic objects with continuous characteristics. This goal is achieved by developing a method for pre-training neural networks that reflect the basic characteristics of the subject area. The scientific novelty of the work is the development of a method for identifying nonlinear dynamic objectsin the form of time delay neural networks based on the use of a set of basic pre-trained neural networks that reflect the typical properties of the subject area. In contrast to the traditional approach to pre-training, the developed method allows building models of lower complexity.A formal criterion is proposed for determining the moment of termination of the neural network pre-training, the use of which allows avoiding retraining of the base model and ensuring a significant reduction in the model training time on the target data set. The practical utility of the work lies in the development of an algorithm for the method of pre-training time delay neural networks in the tasks of identifying nonlinear dynamic objects with continuous characteristics, which allows to significantlyreduce the training time of neural networks without losing model accuracy. The value of this study is todetermine the area of effective use of the proposed method, namely, when the general and target datasets do not have significant differences and the target dataset is of sufficient size to reflect the properties of the research object.

The purposes are to analyze the safety and real-time performance of intelligent transportation under big data and solve urban transportation safety problems. The existing problems of complex traffic data collection and single factor analysis are analyzed. Guangzhou target road section and its historical traffic flow are taken as the data source, the big data analytics (BDA) platform is constructed after preprocessing, and then the deep belief network (DBN) is introduced to build the intelligent transportation model based on BDA. Finally, the model is simulated to analyze its performance. The results reveal that the Root Mean Square Error (RMSE), Mean Absolute Error (MAE), and Mean Absolute Percentage Error (MAPE) of the proposed algorithm are 4.42%, 6.21%, and 8.03%, respectively compared to Recurrent Neural Network (RNN), Convolutional Neural Network (CNN), Neuro fuzzy c-means model (FCM), Deep Neural Network (DNN), and neural network (NN). The mean absolute error (MAE) and mean absolute percentage error (MAPE) are 4.42%, 6.21% and 8.03% respectively. The prediction accuracy is significantly better than that of other algorithms. The analysis of Accuracy, Precision, Recall and F1 suggests that the safety prediction accuracy of the proposed algorithm can reach up to 88.57%, which is at least 3.19% higher than that of other algorithms. Meantime, the proposed algorithm can effectively inhibit the spread of congestion, and achieve the effect of timely evacuation for traffic congestion. Regarding the data transmission performance, the delay is less than 40ms and the packet loss rate is less than 0.1, which is significantly better than other model algorithms in real-time performance. Hence, the constructed intelligent transportation algorithm based on BDA has high precision and excellent congestion evacuation performance, which can provide an experimental basis for constructing intelligent and safe transportation in the future.

The training time of classifier based on neural network is very long using the conventional normalization when the distances between samples of different classes are too small. To overcome the disadvantage, the normalization method based on rough set theory is proposed. By normalizing samples using rough ser theory, the samples which are near but belong to different classes are taken apart. The normalized samples are used to train neural network. The method is applied into neural network based fault line detection for distribution network. The simulation results show that the training time of neural network with processed samples is shorter markedly.

The main challenge in using supervised neural networks in data mining applications is to get explicit knowledge from these models. For this purpose, a study on knowledge acquisition from supervised neural networks employed for classification problems is presented. An algorithm for rule extraction from neural networks, based on the RX algorithm is developed. This algorithm, named modified RX, is experimentally evaluated in two different domains: Iris Plants Database and Pima Indians Diabetes Database. The results are compared to those obtained by classification trees. As far as the efficacy is concerned, one observes that the successful application of the algorithm mainly depends on the knowledge representation acquired by the connectionist model, whereas the efficiency only depends on the neural network training time.

Experiments have proven that traditional prediction research methods have limitations in practice. Proposing countermeasures for environmental changes is the key to optimal control of the cold chain environment and reducing the lag of control effects. In this paper, a cold chain transportation environment prediction method, combining k-means++ and a long short-term memory (LSTM) neural network, is proposed according to the characteristics of the cold chain transportation environment of agricultural products. The proposed prediction model can predict the trend of cold chain environment changes in the next ten minutes, which allows cold chain vehicle managers to issue control instructions to the environmental control equipment in advance. The fusion process for temperature and humidity data measured by multiple data sensors is performed with the k-means++ algorithm, and then the fused data are fed into an LSTM neural network for prediction based on time series. The prediction error of the prediction model proposed in this paper is very satisfactory, with a root-mean-square error (RMSE), mean absolute error (MAE), mean squared error (MSE), mean absolute percentage error (MAPE) and R-squared of 0.5707, 0.2484, 0.3258, 0.0312 and 0.9660, respectively, for temperature prediction, and with an RMSE, MAE, MSE, mean absolute percentage error and R-squared of 1.6015, 1.1770, 2.5648, 0.2736 and 0.9702, respectively, for humidity prediction. Finally, the LSTM neural network and back propagation (BP) neural network are compared in order to enhance the reliability of the results. In terms of the prediction effect of the temperature and humidity in cold chain vehicles transporting agricultural products, the proposed model has a higher prediction accuracy than that of existing models and can provide strategic support for the fine management and regulation of the cold chain transportation environment.

The paper is devoted to resolving the contradiction between the accuracy of modeling nonlinear dynamics and the speed of model construction under conditions of limited computing resources. The aim of the work is to reduce the time for building models of nonlinear dynamic objects with continuous characteristics in the form of neural networks while ensuring a given modeling accuracy. This goal is achieved by developing a new method for synthesizing intelligent systems based on the superposition of pre-trained (support) models in the form of neural networks that reflect the basic properties of the object. The scientific novelty of the work lies in the development of a method for identifying nonlinear dynamic objects in the form of neural networks with time delays based on a set of pre-trained neural network models that reflect the basic properties of the subject area. In contrast to the traditional approach to pre-training, the developed method allows building models of lower complexity and with shorter training time while ensuring the required accuracy. To determine the initial parameters of the model, expressions based on the superposition of reference models in the form of neural networks are proposed. The practical utility of the work lies in the development of an algorithm for the method of reference models for training neural networks with time delays, which can significantly reduce the training time of neural networks without losing model accuracy

In this paper studies a kind of information matching method based on neural network, and the method combines first-order logic and semantic similarity to complete the table matching, using SOM+ to generate feedback neural network to complete the field matching. The method can effectively reduce the matching time complexity. And by using history match, it effectively reduces the training time of neural network, improving the accuracy of matching.

In this research, two methods for crack detection in structures are presented and compared. The considered structure is a cantilever beam with rectangular cross section. In order to find cracks, firstly, a new technique based on wavelet analysis and finite element method (FEM) is applied. The advantage of this technique is that the crack detection process is more clear and comfortable than previous works. Then the process of crack detection is performed using FEM and combination of two types of artificial neural network (ANN) including radial basis function (RBF) and back-error propagation (BEP) neural networks. For crack identification in the proposed method, firstly, a RBF neural network is used to detect the number of cracks of structure. Then a BEP neural network is trained to detect the locations of cracks. Training of neural networks is performed using obtained data from FEM. Finally obtained results from two methods are compared with each other.

As for the problem of too long training time of convolution neural network (CNN), this paper proposes a fast training method for CNN in SAR automatic target recognition (ATR). The CNN is divided into two parts: one that contains all the convolution layers and sub-sampling layers is considered as convolutional auto-encoder (CAE) for unsupervised training to extract high-level features; the other that contains fully connected layers is regarded as shallow neural network (SNN) to work as a classifier. The experiment based on MSATR database shows that the proposed method can tremendously reduce the training time with little loss of recognition rate.

As an essential water quality parameter in aquaculture ponds, dissolved oxygen (DO) affects the growth and development of aquatic animals and their feeding and absorption. However, DO is easily influenced by external factors. It is not easy to make scientific and accurate predictions of DO concentration trends, especially in long-term predictions. This paper uses a one-dimensional convolutional neural network to extract the features of multidimensional input data. Bidirectional long and short-term memory neural network propagated forward and backward twice and thoroughly mined the before and after attribute relationship of each data of dissolved oxygen sequence. The attention mechanism focuses the model on the time series prediction step to improve long-term prediction accuracy. Finally, we built an integrated prediction model based on convolutional neural network (CNN), bidirectional long and short-term memory neural network (BiLSTM) and attention mechanism (AM), which is called CNN-BiLSTM-AM model. To determine the accuracy of the CNN-BiLSTM-AM model, we conducted short-term (30 minutes, one hour) and long-term (6 hours, 12 hours) experimental validation on real datasets monitored at two aquaculture farms in Yantai City, Shandong Province, China. Meanwhile, the performance was compared and visualized with support vector regression, recurrent neural network, long short-term memory neural network, CNN-LSTM model and CNN-BiLSTM model. The results show that compared with other comparative models, the proposed CNN-BiLSTM-AM model has an excellent performance in mean absolute error, root means square error, mean absolute percentage error and determination coefficient.

این مطالعه به منظور بررسی پیش بینی شاخص های عملکرد تولیدی در مرغان تخم گذار با استفاده از شبکه های عصبی مصنوعی و رگرسیون غیرخطی چندگانه انجام شد. بررسی بر روی اطلاعات چهار دوره متوالی پرورش در یک واحد پرورش مرغ تخم گذار صورت گرفت. روش های داده-کاوی شامل رگرسیون خطی و غیر خطی، شبکه عصبی پرسپترون سه لایه، شبکه عصبی پرسپترون چهار لایه و شبکه عصبی با تابع پایه ای شعاعی بود. در این مدل ها از متغیرهای سن گله، میزان خوراک مصرفی و فصل تولید به عنوان متغیر پیشگو و شاخص های عملکرد تولیدی شامل درصد تخم-گذاری، وزن توده ای تخم مرغ تولیدی و ضریب تبدیل غذایی به عنوان متغیر پاسخ استفاده شد. نتایج نهایی رگرسیون های خطی نشان داد که برای تمامی متغیرهای وابسته مورد مطالعه متغیر مستقل سن گله معنی دار می باشد. بنابراین رگرسیون غیر خطی شاخص های عملکرد تولیدی در مقابل سن برای مقایسه با شبکه های عصبی مختلف مورد بررسی قرار گرفت و برای مقایسه کلیه مدل ها از ضریب تعیین (R2) و میانگین قدر مطلق خطا (MAE) استفاده شد. نتایج نشان داد بین شبکه های عصبی مصنوعی مختلف مورد مطالعه، شبکه با تابع پایه ای شعاعی بهتر از سایر مدل های در پیش‌بینی شاخص های عملکرد تولیدی مرغان تخم گذار عمل می کند.