RETRACTED: An Improved Dynamic Process Neural Network Prediction Model Identification Method

A time series prediction scheme based on multiresolution-based bilinear recurrent neural network (MBLRNN) is proposed in this paper. The proposed predictor is based on the BLRNN that has been proven to have robust abilities in modeling and predicting time series. The learning process is further improved by using a multiresolution-based learning algorithm for training the BLRNN so as to make it more robust for long-term prediction of the time series. The proposed MBLRNN-based predictor is applied to the long-term prediction of time series. Experiments and results on the Mackey-Glass Series data and Sunspot Series data show that the proposed MBLRNN outperforms both the traditional multilayer perceptron type neural network (MLPNN) and the BLRNN in terms of the normalized mean square error (NMSE).

The Sixth International Symposium on Neural Networks (ISNN 2009)

Modeling of MSMPR crystallizer dynamics – time series prediction by neural network

Purpose In order to analyze the text complexity of Chinese and foreign academic English writings, the artificial neural network (ANN) under deep learning (DL) is applied to the study of text complexity. Firstly, the research status and existing problems of text complexity are introduced based on DL. Secondly, based on Back Propagation Neural Network (BPNN) algorithm, analyzation is made on the text complexity of Chinese and foreign academic English writings. And the research establishes a BPNN syntactic complexity evaluation system. Thirdly, MATLAB2013b is used for simulation analysis of the model. The proposed model algorithm BPANN is compared with other classical algorithms, and the weight value of each index and the model training effect are further analyzed by statistical methods. Finally, L2 Syntactic Complexity Analyzer (L2SCA) is used to calculate the syntactic complexity of the two libraries, and Mann–Whitney U test is used to compare the syntactic complexity of Chinese English learners and native English speakers. The experimental results show that compared with the shallow neural network, the deep neural network algorithm has more hidden layers and richer features, and better performance of feature extraction. BPNN algorithm shows excellent performance in the training process, and the actual output value is very close to the expected value. Meantime, the error of sample test is analyzed, and it is found that the evaluation error of BPNN algorithm is less than 1.8%, of high accuracy. However, there are significant differences in grammatical complexity among students with different English writing proficiency. Some measurement methods cannot effectively reflect the types and characteristics of written language, or may have a negative relationship with writing quality. In addition, the research also finds that the measurement of syntactic complexity is more sensitive to the language ability of writing. Therefore, BPNN algorithm can effectively analyze the text complexity of academic English writing. The results of the research provide reference for improving the evaluation system of text complexity of academic paper writing. Design/methodology/approach In order to analyze the text complexity of Chinese and foreign academic English writings, the artificial neural network (ANN) under deep learning (DL) is applied to the study of text complexity. Firstly, the research status and existing problems of text complexity are introduced based on DL. Secondly, based on Back Propagation Neural Network (BPNN) algorithm, analyzation is made on the text complexity of Chinese and foreign academic English writings. And the research establishes a BPNN syntactic complexity evaluation system. Thirdly, MATLAB2013b is used for simulation analysis of the model. The proposed model algorithm BPANN is compared with other classical algorithms, and the weight value of each index and the model training effect are further analyzed by statistical methods. Finally, L2 Syntactic Complexity Analyzer (L2SCA) is used to calculate the syntactic complexity of the two libraries, and Mann–Whitney U test is used to compare the syntactic complexity of Chinese English learners and native English speakers. The experimental results show that compared with the shallow neural network, the deep neural network algorithm has more hidden layers and richer features, and better performance of feature extraction. BPNN algorithm shows excellent performance in the training process, and the actual output value is very close to the expected value. Meantime, the error of sample test is analyzed, and it is found that the evaluation error of BPNN algorithm is less than 1.8%, of high accuracy. However, there are significant differences in grammatical complexity among students with different English writing proficiency. Some measurement methods cannot effectively reflect the types and characteristics of written language, or may have a negative relationship with writing quality. In addition, the research also finds that the measurement of syntactic complexity is more sensitive to the language ability of writing. Therefore, BPNN algorithm can effectively analyze the text complexity of academic English writing. The results of the research provide reference for improving the evaluation system of text complexity of academic paper writing. Findings In order to analyze the text complexity of Chinese and foreign academic English writings, the artificial neural network (ANN) under deep learning (DL) is applied to the study of text complexity. Firstly, the research status and existing problems of text complexity are introduced based on DL. Secondly, based on Back Propagation Neural Network (BPNN) algorithm, analyzation is made on the text complexity of Chinese and foreign academic English writings. And the research establishes a BPNN syntactic complexity evaluation system. Thirdly, MATLAB2013b is used for simulation analysis of the model. The proposed model algorithm BPANN is compared with other classical algorithms, and the weight value of each index and the model training effect are further analyzed by statistical methods. Finally, L2 Syntactic Complexity Analyzer (L2SCA) is used to calculate the syntactic complexity of the two libraries, and Mann–Whitney U test is used to compare the syntactic complexity of Chinese English learners and native English speakers. The experimental results show that compared with the shallow neural network, the deep neural network algorithm has more hidden layers and richer features, and better performance of feature extraction. BPNN algorithm shows excellent performance in the training process, and the actual output value is very close to the expected value. Meantime, the error of sample test is analyzed, and it is found that the evaluation error of BPNN algorithm is less than 1.8%, of high accuracy. However, there are significant differences in grammatical complexity among students with different English writing proficiency. Some measurement methods cannot effectively reflect the types and characteristics of written language, or may have a negative relationship with writing quality. In addition, the research also finds that the measurement of syntactic complexity is more sensitive to the language ability of writing. Therefore, BPNN algorithm can effectively analyze the text complexity of academic English writing. The results of the research provide reference for improving the evaluation system of text complexity of academic paper writing. Originality/value In order to analyze the text complexity of Chinese and foreign academic English writings, the artificial neural network (ANN) under deep learning (DL) is applied to the study of text complexity. Firstly, the research status and existing problems of text complexity are introduced based on DL. Secondly, based on Back Propagation Neural Network (BPNN) algorithm, analyzation is made on the text complexity of Chinese and foreign academic English writings. And the research establishes a BPNN syntactic complexity evaluation system. Thirdly, MATLAB2013b is used for simulation analysis of the model. The proposed model algorithm BPANN is compared with other classical algorithms, and the weight value of each index and the model training effect are further analyzed by statistical methods. Finally, L2 Syntactic Complexity Analyzer (L2SCA) is used to calculate the syntactic complexity of the two libraries, and Mann–Whitney U test is used to compare the syntactic complexity of Chinese English learners and native English speakers. The experimental results show that compared with the shallow neural network, the deep neural network algorithm has more hidden layers and richer features, and better performance of feature extraction. BPNN algorithm shows excellent performance in the training process, and the actual output value is very close to the expected value. Meantime, the error of sample test is analyzed, and it is found that the evaluation error of BPNN algorithm is less than 1.8%, of high accuracy. However, there are significant differences in grammatical complexity among students with different English writing proficiency. Some measurement methods cannot effectively reflect the types and characteristics of written language, or may have a negative relationship with writing quality. In addition, the research also finds that the measurement of syntactic complexity is more sensitive to the language ability of writing. Therefore, BPNN algorithm can effectively analyze the text complexity of academic English writing. The results of the research provide reference for improving the evaluation system of text complexity of academic paper writing.

Historically, machine learning techniques and particularly neural networks have been applied in the domain of Geophysics to discern rock properties through a quantitative approach from seismic data and associated seismic attributes. However, the success or failure of neural network is dependent on amount of available data. In general, more labelled data during training of the neural network results in accurate predictions. Therefore, seismic based rock property prediction models and their calibration needs to be fed with labelled data coming from well logs in adequate amount. The fact that the number of wells are often limited can be a huge challenge in adopting neural networks predictions. Downton and Hampson (2019) developed a Hybrid Theory-guided Data Science (TGDS) technique to improve the training of the network by augmenting the amount of data used for training. This technique aims at overcoming the issue of scarcity of the well log data. By using the outputs of theory based components like Rock Physics theory as input to the data science component of augmenting the amount of the data. In this method, Rock Physics is applied to model the elastic response due to the changes in rock and fluid properties of the available well control to create a huge dataset of pseudo wells. Then, synthetic seismic gathers are modeled at the pseudo well locations to train a Deep Feed-Forwards Neural Network (DFNN). The parameters estimated during DFNN training with synthetic seismic gathers are then applied on a real seismic dataset. The final predicted P-impedance from this formerly described method was compared to conventional inverted Pre-Stack Elastic Inversion. This paper shows the application of the method of TGDS based synthetic seismic catalog generation followed by DFNN applied on real seismic dataset for a commercially oil producing field in North Sea. The reservoirs are injectite sands which are difficult to image but can be key pay zones with quite high porosity and permeability. Well log curves e.g. P-Impedance, Porosity and Water Saturation serve as guidelines to create the synthetic seismic data. Additionally, with the added advantage of increased synthetic data, pseudo wells were broken into training and validation datasets. So, the DFNN training is carried out on a portion of the training data subset. During the process of training the DFNN, the seismic data remains blind. The estimated DFNN non-linear operator is applied to the seismic data, however, it needs to be scaled to the real data as the training was performed on the synthetic seismic data. The final result of the data science and machine learning approach driven derived P-impedance showed higher frequency and better lateral continuity than the conventional theory based approach of Pre-Stack Elastic Inversion. Augmenting the amount of training data by generating synthetic data based on the statistics of from nearby well control and rock physics relationships is prime differentiator of this workflow. By using rock physics theory a large number of pseudo wells can be generated that gives a range of expected geologic scenarios. Key wells from nearby fields, that do not necessarily tie the seismic under consideration, can be incorporated into the analysis and also, if required additional wells could be incorporated into the analysis as drilled.

Neural network (NN) training is the optimization process by which the relation between the NN input and output is established. A new formulation for the NN training is presented where an NN model is reconstructed such that it produces predicted output data optimally fitting the observed ones. The optimal level of fit is determined by minimization of the generalized cross-validation function, which is integrated in the training. The training process is fully automated, does not require the user to set aside data for validation, and enables objective testing and evaluation of the predictions. Results are demonstrated and discussed using synthetic data produced by Lorenz’s low-order circulation model and on real data from the equatorial Pacific.

Many real-world applications reveal difficulties in learning classifiers from\nimbalanced data. The rising big data era has been witnessing more\nclassification tasks with large-scale but extremely imbalance and low-quality\ndatasets. Most of existing learning methods suffer from poor performance or low\ncomputation efficiency under such a scenario. To tackle this problem, we\nconduct deep investigations into the nature of class imbalance, which reveals\nthat not only the disproportion between classes, but also other difficulties\nembedded in the nature of data, especially, noises and class overlapping,\nprevent us from learning effective classifiers. Taking those factors into\nconsideration, we propose a novel framework for imbalance classification that\naims to generate a strong ensemble by self-paced harmonizing data hardness via\nunder-sampling. Extensive experiments have shown that this new framework, while\nbeing very computationally efficient, can lead to robust performance even under\nhighly overlapping classes and extremely skewed distribution. Note that, our\nmethods can be easily adapted to most of existing learning methods (e.g., C4.5,\nSVM, GBDT and Neural Network) to boost their performance on imbalanced data.\n

Efficient self-organizing multilayer neural network for nonlinear system modeling

Feedforward neural network prediction is the most commonly used method in time series prediction. In view of the low prediction accuracy of the conventional BPNN model when the time series data contain a certain linear relationship, this paper describes a neural network approach for time series prediction, that is BPNN–DIOC (back-propagation neural network with direct input-to-output connections). Eight different datasets were used to verify the validity of BPNN–DIOC model in time series prediction. In this paper, the BPNN was extended to four variants based on the presence or absence of output layer bias and input-to-output connections firstly, and the prediction accuracy of eight datasets are analyzed by statistic method secondly. Finally, the experimental results demonstrate that the BPNN–DIOC has better prediction accuracy compared to the conventional BPNN while the output layer bias has no significant effect. Therefore, the input-to-output connections can significantly improve the prediction ability of time series.

Passive sensing applications are limited by their inability to determine who is using a device, a critical concern in child mobile device use research, where devices are often shared between siblings or between a child and their parent. Our previous work leveraged behavioral biometrics to identify a target child user; however, it is unknown what type of training data is necessary for optimal model performance. This study evaluated model performance across different characteristics of training data. Thirty-six children (11.3 ± 0.9 years, 56% female) self-selected a video or a game on iPads for 10 min while laying and for another 5 min while sitting. The SensorLog application captured iPad accelerometer and gyroscope data while the child interacted with the device. Machine learning algorithms including Neural Network (NN), Random Forest (RF), k-Nearest Neighbors (k-NN), and SwipeFormer were applied to determine the most important aspects of training data to optimize model performance. The aspects of training data evaluated included (1) varying the length (i.e., seconds of training data), (2) varying the user position (i.e., sitting, laying), and (3) varying the time proximity between training and testing data. F1 score was used to evaluate model performance. The SwipeFormer F1 scores were lowest when the training data was further from the test data (0 when training data was 11 min away from test data) and highest when training data was close to test data (0.91 when training data was the minute preceding test data). The SwipeFormer F1 scores were highest when predicting the user laying from laying (0.97) and sitting from sitting (0.94), and lowest when predicting the user sitting from laying (0) and laying from sitting (0). The length of training data had little impact on performance, with a SwipeFormer F1 score of 0.91 when training on one minute of data and a SwipeFormer F1 score of 0.94 when training on twelve minutes of data. Because researchers would likely be predicting users at different timepoints than their training data, research should focus on improving model performance for identifying users independent of time proximity for training and test data.

SummaryAccurate prediction of the solubility of chemical substances in solvents remains a challenge. The sparsity of high-quality solubility data is recognized as the biggest hurdle in the development of robust data-driven methods for practical use. Nonetheless, the effects of the quality and quantity of data on aqueous solubility predictions have not yet been scrutinized. In this study, the roles of the size and the quality of data sets on the performances of the solubility prediction models are unraveled, and the concepts of actual and observed performances are introduced. In an effort to curtail the gap between actual and observed performances, a quality-oriented data selection method, which evaluates the quality of data and extracts the most accurate part of it through statistical validation, is designed. Applying this method on the largest publicly available solubility database and using a consensus machine learning approach, a top-performing solubility prediction model is achieved.

The neural network has attracted a large number of researchers in the field of network intrusion detection systems (NIDS) due to its powerful performance. However, most attention was paid to the improvement of the performance of the target system while few to the security issues of the neural network itself. The online learning network uses the constant network traffic as the training data to achieve lower memory usage, better adaptation to the environment over time, and an infinite amount of training data. Due to the existence of network attacks specifically designed for neural networks such as poison attacks and evasion attacks, as well as the existence of the false positives of the model itself, suspicious traffic will be inevitably put into training, resulting in performance drops or even collapse. Furthermore, autoencoder based anomaly detection needs pure normal data for train, which is more dangerous in the situation mentioned above. This article proposes a novel idea to improve the training process of autoencoder based NIDS under online learning. Since the training data in online learning can be regarded as infinite, we can randomly throw away a part of the data that is about to enter training. Experiments were implemented by the DAE-based anomaly detection algorithm and KDD99 datasets. It can be known that performance deduction can be significantly alleviated by tuning the leakage ratio and accuracy decrease can be suppressed effectively.

Mark Watson Morgan Kaufmann Publishers, San Francisco, 1997, 377 pp. ISBN 1-55860-420-0, $44.95 This book is well written and organized to cover how to apply AI (Artificial Intelligence) techniques for practical Internet and Intranet tools by using the Java programming language. In the book, the author uses neural network and genetic algorithms to develop distributed systems that are applied for the Internet and Intranets, such as client/server model and information retrieval tool and so on. Part 1 begins with a good general introduction to AI. The history and survey of Artificial Intelligence research are given briefly. In addition, symbolic processing versus neural networks approaches are discussed. This author opts to use neural network to develop example Internet/Intranet systems in the book. Part 2 consists of Java programs developed for AI, neural networks, genetic algorithms, natural language processing, and expert systems. Chapter 2 illustrates Graphical User Interface (GUI) class that is derived from Applet and Canvas classes in Java, which is used in most example programs in the book. Chapter 3 introduces a frame data structure that is used as a message in distributed systems in the later chapters. Chapter 4 shows neural network classes developed in Java and its theory and implementation. Chapter 5 explains the genetic algorithms and the Java genetic algorithm classes. Chapter 6 describes how to develop and implement natural language processing classes using Java. Chapter 7 shows how to share data objects in distributed systems. For example, distributed AI agent classes are developed to share data objects, WWW data collection agent classes are shown to retrieve documents in the Internet, and small POP (Post Office Protocol) mail classes are developed to send and receive mails. Chapter 8 illustrates how to develop expert systems by using Java Jess, which is based on OPS5/CLIPS expert systems. Part 3 explains how to generate and implement the projects using the Java classes that are developed in Part 2. Chapter 9 shows the project that demonstrates the use of genetic algorithms in chapter 5 to control enemy ships in a scrolling arcade game. Chapter 10 describes the real-time handwriting recognition system developed by using the neural network classes in chapter 4. Chapter 11 illustrates a greedy algorithm to optimize neural networks that is based on the author’s hand-tuning neural training data. Chapter 12 improves neural network training data by applying genetic algorithms, to discard bad training data and to avoid discarding good training data. Chapter 13 describes the distributed natural language processing system that uses natural language processing classes in chapter 6. The data structures in chapter 2 are used to store historical data in a separate server application on any other computer in the Internet or Intranets. The final chapter provides the design and implementation of a complete agent-based information retrieval tool for the Internet. The retrieved documents are stored in ASCII or HTML format. This book is a very good reference book for anyone interested in developing systems using Java in distributed environments, especially applying AI techniques. Also, the author provides the CD-ROM that includes the source code for Java classes developed in the book. Haklin Kimm, University of Tennessee

A neural network (NN) aided model is proposed for the filtered reaction rate in moderate or intense low-oxygen dilution (MILD) combustion. The framework of the present model is based on the partially stirred reactor (PaSR) approach, and the fraction of the reactive structure appearing in the PaSR is predicted using different NN’s, to consider both premixed and non-premixed conditions while allowing the use of imbalanced training data between premixed and non-premixed combustion direct numerical simulation (DNS) data. The key ingredient in the present model is the use of local combustion mode prediction performed by using another NN, which is developed in a previous study. The trained model was then assessed by using two unknown combustion DNS cases, which yields much higher dilution level (more intense MILD condition) and higher Karlovitz number than the DNS cases used as training data. The model performance assessment has been carried out by means of the Pearson’s correlation coefficient and mean squared error. For both the present model and zeroth-order approximated reaction rate, the correlation coefficient with the target values shows relatively high values, suggesting that the trend of predicted field, by the present model and zeroth-order approximation, is well correlated with the actual reaction rate field. This suggests that the use of PaSR equation is promising if the fraction of the reactive structure is appropriately predicted, which is the objective in the present study. On the other hand, substantially lower mean squared error is observed for a range of filter sizes for the present model than that for the zeroth-order approximation. This suggests that the present filtered reaction rate model can account for the SGS contribution reasonably well.

The problem of fault classification in industry has been studied extensively. Most classification algorithms are modeled on the premise of data balance. However, the difficulty of collecting industrial data in different modes is quite different. This inevitably leads to data imbalance, which will adversely affect the fault classification performance. This article proposes a novel data augmentation classifier (DAC) for imbalanced fault classification. Data augmentation based on generative adversarial networks (GANs) is an effective way to solve the problem of unbalanced classification. However, the randomness of the GAN generation process restricts the effect of data enhancement. DAC proposes a data selection strategy based on data filtering and data purification in model training to solve this problem. In addition, DAC combines supervised learning and data generation processes to obtain an end-to-end model. Meanwhile, multigenerator structure of DAC (MDAC) is proposed to solve the problem of incomplete learning of a single generator when data imbalances get complicated. The proposed DAC and MDAC are applied in two fault classification cases of the Tennessee Eastman (TE) benchmark process, results of which show superiority of DAC and MDAC compared to existing methods. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —Data imbalances are common in fault classification and affect the effectiveness of modeling in industry. As a generative model, generative adversarial networks (GANs) provide new ideas for small-class data augmentation. However, the instability of its training process and the randomness of data generation affect the results of data augmentation. In this article, the GAN generation process is analyzed in detail. The results of the visualization indicate that no data generation was perfect at any one time. Based on the rules of GAN data generation, we propose a data selection strategy during training. High-quality data are selected for data augmentation through data filtering and data purification. Apart from this, we combine the training process of GAN and classification model for imbalanced data to reduce modeling time. Through industrial examples, we have evaluated the effectiveness of this method.