Data processing using neural networks

The application of artificial neural networks to the estimation of bioprocess variables will be discussed. In fermentation processes, direct on-line measurements of primary process variables usually are unavailable. The state of the cultivation, therefore, has to be inferred from measurements of secondary variables and any previous knowledge of process dynamics. This research investigates the learning, recall and generalization characteristics of neural networks trained to model the nonlinear behavior of a fermentation process. Two different neural network methodologies are discussed, namely, feed-forward and recurrent neural networks, which differ in their treatment of time dependence. The neural networks are trained by backpropagation using a conjugate gradient technique, which provides a dramatic improvement in the convergence speed. The objective is to use environmental and physiological information available from on-line sensors to estimate concentrations of species in a bioreactor. Results of the neural network estimators are presented, based on experimental data available from the ethanol production by Zymomonas mobilis fermentation. The feed-forward and recurrent neural network methodologies are demonstrated to perform suitably as unmeasurable state estimators. Both networks offer comparable abilities of recall, but recurrent networks perform better than feed-forward networks in generalization.

A multilayer feedforward neural network with a modification to the standard backpropagation training of neural nets is considered. Information on the gradients of target outputs with respect to network inputs is used in this modification. This modified neural network is then applied for approximating design requirements of aerospace composite components, such as an aircraft engine guide vane and a satellite reflector assembly. This neural network functional approximator is then used with an optimization algorithm for determining the optimal parameters for component design requirements. The modified neural network provides for faster convergence of learning process and requires a smaller number of hidden layer nodes for convergence to similar training error levels. HE application of neural networks as universal approximators has been the focus of tremendous amount of activity in the past few years.16 Neural networks have been used for approximating and modeling both memoryless systems and dynamical systems with memory. This paper does not go into the various neural net architectures and training schemes, and the interested reader could refer to standard texts on this subject matter.1'2 This paper considers static neural networks (systems without memory) formed by interconnections without loops, called feedforward networks.2 In addition, this work considers a backpropagation learning scheme with adaptation of learning rate. There have been many additions to, or adaptations of, the basic backpropagation scheme, such as adaptive learning rate, learning with momentum, and the like,1'2 for improving convergence of the learning scheme, and these are not addressed in this paper. This paper explores a modification of the backpropagation learning scheme using information on the gradient of the target outputs with respect to the inputs of the network. The performance of the modified learning scheme is compared with that of the standard backpropagation learning scheme.2 A feedforward network with a single hidden layer trained with this modified learning scheme is applied in structural optimization problems. Figure 1 shows a feedforward net with a single hidden layer. The restriction to a single hidden layer is not a limitation of the modified learning scheme, and the new training scheme can be used on networks with multiple hidden layers also. The use of neural networks for structural response approximation has been reported by several authors.46 The primary reason for investigating such properties is to assess the applicability of trained neural networks for quick and accurate approximation of response functions during structural design optimization. Structural optimization, utilizing either gradient-based numerical search or genetic search algorithms, requires repeated function (response) evaluations during the search process. A single finite element analysis of large-scale structures requires a significant amount of computational time and resource and hence the need for robust approxima

This paper proposes a new method for the design, through simulated evolution, of biologically inspired receptive fields in feedforward neural networks (NNs). The method is intended to enhance pattern recognition performance by creating new neural architectures specifically tuned for a particular pattern recognition problem. It proposes a combined neural architecture composed of two networks in cascade: a feature extraction network (FEN) followed by a neural classifier. The FEN is composed of several layers with receptive fields constructed by additive superposition of excitatory and inhibitory fields. A genetic algorithm (GA) is used to select receptive field parameters to improve classification performance. The parameters are receptive field size, orientation, and bias as well as the number of different receptive fields in each layer. Based on a random initial population where each individual represents a different neural architecture, the GA creates new enhanced individuals. The method is applied to handwritten digit classification and face recognition. In both problems, results show strong dependency between NN classification performance and receptive field architecture. GA selected parameters of the receptive fields produced improvements in the classification performance on the test set up to 90.8% for the problem of handwritten digit classification and up to 84.2% for the face recognition problem. On the same test sets, results were compared advantageously to standard feedforward multilayer perceptron (MLP) NNs where receptive fields are not explicitly defined. The MLP reached a maximum classification performance of 84.9% and 77.5% in both problems, respectively.

Single-hidden-layer feed-forward quantum neural network based on Grover learning

The Sixth International Symposium on Neural Networks (ISNN 2009)

On-line Estimation of Bioreactors using Recurrent Neural Networks

Arabic language can be used by native and non- native speakers; due to Arabic is the language of the holy book of Muslims. In this paper, Arabic phoneme recognition system is proposed based on Malay speakers. This system consists of three main stages. The first stage is noise reduction and it aims to enhance the phoneme signals by excluding the unvoiced signals and keep only the voiced signal. Wiener filter is adapted to accomplish this task. The second stage is based on Mel-Frequency Cepstral Coefficients method to extract a vector of features to represent each phoneme signal. Eventually, pattern recognition neural network is designed as recognizer. The proposed system produces sufficient outcomes with 20 hidden neurons. Keywords—Arabic; Malay; pattern recognition; Wiener; Mel- Frequency Cepstral Coefficients I. INTRODUCTION It is well known the wide range of Artificial Neural Networks (ANN) application in speech recognition, financial, telecommunications, electronics and medical. As for speech recognition system for Arabic language, (1) & (2) have addressed the implementation of recognition system based on Arabic native speakers. In general, neural networks (NNs) for phonemes recognition are divided into several categories for instance Very Large Scale Integration (VLSI) NN that can be divided into digital and analogue with digital NNs being more compatible with feed-forward neural networks while analogue NNs are found to be more successful with recurrent NNs (3). From Automatic Speech System (ASR) point of view, NNs for recognition purpose can be divided into two main category namely conventional neural networks such as Multilayer Perceptron (MLP) and Radial Basis Function (RBF) and secondly Recurrent Neural Networks (RNN). The first category of NNs was implemented as pattern classifiers and proven capable for recognition purpose but not at par as compared to Hidden Markov Model (HMM) performance (2). At present, HMM has been proven to be the most successful approach in ASR research area until recently the possibility to hybrid both HMM and ANN (4) & (5). In general, speech recognition systems consist of three stages specifically noise reduction, feature extraction and classification. The aim of noise reduction is to reduce the level of noise in the speech signal and make it more compatible for the ASR. As for feature extraction, this process attempted to extract the most relevant features related to the phoneme signal. The extracted features acted as input features of the NN during training and testing. Finally, the purpose of classification process is to arrange the most similar and related features in categories based on the training phase. Basically, networks training database are being observed and updated during training session of the networks, involving the weights and biases arguments. Networks with lower errors can be considered as better networks (6). Hence, this paper deems to explore further the ability of NN for Arabic phonemes recognition spoken by Malay individuals. This paper is arranged as following; related work is discussed in section II followed by section III that discussed method to be implemented based on Wiener filter, Mel-Frequency Cepstral Coefficients and pattern recognition neural network. Section IV elaborated experimental analysis and results and followed by Conclusion section.

Application of radial basis function and feedforward artificial neural networks to the Escherichia coli fermentation process

An increasing number of applications of dynamic neural networks has been developed for digital signal processing (DSP), dynamic neural networks are feedforward neural networks with commonly used scalar synapses replaced by linear filters. This provides feedforward neural networks with the capability of performing dynamic mappings, which depend on past input values, dynamic neural networks are suitable for time series prediction, nonlinear system identification, and signal processing applications. Their most popular types are Finite Impulse Response (FIR) neural networks, which are obtained by replacing synapses with finite impulse response filters. Due to their guaranteed stability characteristic and easy to minimize error surface they have been used with great success in many applications such as signal enhancement, noise cancellation, classification of input patterns, system identification, prediction, and control.. Most of the works on system identification using neural networks are based on multilayer feedforward neural networks with backpropagation learning or more efficient variations of this algorithm, an elegant method for training layered networks. This paper is based on work in a Dynamic System Modeling (DYSMO) and as an application for speed control of DC motor drive.

The capability of artificial neural networks to use and combine simple learning rules to master different complex tasks is very motivating and has sufficed on lots of challenging pattern recognition problems. Thus, in this paper, we selected a pattern recognition task that can be tough even for humans. The aim is to investigate the capability of neural networks to learn and recognize patterns describing some segmented chest organs in binary segmented images. The geometric descriptions in addition to the entropy of those images have been leveraged on in this work for classification purposes. Two learning algorithms are used for training both feedforward and competitive neural networks which have been considered for the pattern recognition task. Overall, we show that a competitive neural network that falls into unsupervised learning category outperformed the feedforward network which is considered a supervised learning network. This outperformance is in terms of time, error rate, cost, and accuracy.KeywordsArtificial neural networksPattern recognitionSegmented chest organsCompetitive neural networksUnsupervised learningSupervised learning

The paper presents the screening of various feedforward neural networks (FANN) and wavelet-feedforward neural networks (WFANN) applied to time series of ground-level ozone (O3), nitrogen dioxide (NO2), and particulate matter (PM10 and PM2.5 fractions) recorded at four monitoring stations located in various urban areas of Romania, to identify common configurations with optimal generalization performance. Two distinct model runs were performed as follows: data processing using hourly-recorded time series of airborne pollutants during cold months (O3, NO2, and PM10), when residential heating increases the local emissions, and data processing using 24-h daily averaged concentrations (PM2.5) recorded between 2009 and 2012. Dataset variability was assessed using statistical analysis. Time series were passed through various FANNs. Each time series was decomposed in four time-scale components using three-level wavelets, which have been passed also through FANN, and recomposed into a single time series. The agreement between observed and modelled output was evaluated based on the statistical significance (r coefficient and correlation between errors and data). Daubechies db3 wavelet-Rprop FANN (6-4-1) utilization gave positive results for O3 time series optimizing the exclusive use of the FANN for hourly-recorded time series. NO2 was difficult to model due to time series specificity, but wavelet integration improved FANN performances. Daubechies db3 wavelet did not improve the FANN outputs for PM10 time series. Both models (FANN/WFANN) overestimated PM2.5 forecasted values in the last quarter of time series. A potential improvement of the forecasted values could be the integration of a smoothing algorithm to adjust the PM2.5 model outputs.

The paper deals with the applications of artificial neural networks in the control of the DC drive. In the paper three control structures are discussed. The first control structure uses a conventional PI controller. The second structure uses a neural network predictive control. The last structure is a sensorless control of the DC drive using feedforward neural network. The DC drives were simulated in program Matlab with Simulink toolbox. The main goal was to find the simplest neural network structures with minimum number of neurons, but simultaneously good control characteristics are required. Despite used neural networks, which are very simple, it was achieved satisfactory results.KeywordsArtificial Neural NetworkArtificial Neural Network ModelModel Predictive ControlDirect Torque ControlInduction Motor DriveThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

In present days, real time navigation depends on Kalman filter to fuse data from Global Positioning System (GPS) and Inertial Navigation System (INS). But there exist drawbacks like long design time, requirement of prior knowledge in fusing data from GPS and INS using Kalman Filter. In order to overcome the drawbacks of Kalman filter, GPS and INS integration are done using Artificial Neural Networks (ANN) like FFNNs. But still there exists certain drawbacks like less accuracy in FFNNs for GPS and INS data integration. Therefore, this paper introduces Higher Order Neural Networks (HONNs) approach for INS and GPS data integration to overcome the drawback. A higher order feed forward neural network architecture with optimum number of nodes is used. In the given architectures, the replacement of summation at each node by multiplication results in more powerful mapping, because of its capability of processing higher-order information from training data. Performance comparison of HONNs with FFNNs shows that the proposed architecture provides satisfactory results in terms of error rate and number of epochs. The HONNs like Multiplicative Neural Network (MNN) module and Sigma-Pi-ANN module and Traditional Feed Forward Neural Networks(FFNNs) like Radial Basis Function Neural Network (RBF) module and Back propagation Neural Network (BPN) module are trained to predict the INS position error and provide accurate position of the moving aircrafts. Sigma-Pi ANN is found to be the best in terms of accuracy, number of epochs and execution time among the two HONNs.

An experimental application of Artificial Neural Networks to Eating Disorders is presented. The sample, composed of 172 cases (all women) collected at the Centre for the Diagnosis and Treatment of Eating Disorders of the 1st Medical Division of the St. Eugenio Hospital of Rome, was subdivided, on the basis of the diagnosis made by the specialist of the St. Eugenio, into four classes: Anorexia Nervosa (AN), Nervous Bulimia (NB), Binge Eating Disorders (BED) and Psychogenic Eating Disorders that are Not Otherwise Specified (PED-NOS). The data base was composed of 124 different variables: generic information, alimentary behavior, eventual treatment and hospitalization, substance use, menstrual cycles, weight and height, hematochemical and instrumental examinations, psychodiagnostic tests, etc. The goal of this experiment was to verify the accuracy of the Neural Networks in recognising anorexic and bulimic patients. This article describes 6 experiments, using a Feed Forward Neural Network, each one using different variables. Starting from only the generic variables (life styles, family environment, etc.) and hematoclinical and instrumental examinations, a Neural Networks provided 86.94% of the prediction precision. This work is meant to be a first contribution to creating diagnostic procedures for Eating Disorders, that would be simple and easy-to-use by professionals who are neither psychologists nor psychiatrists nor psychotherapists but who are, however, among the first to meet these patients and who are therefore called upon to give such patients the very first pieces of advice on seeking proper treatment.

Ultrasound (US) imaging is the initial phase in the preliminary diagnosis for the treatment of kidney diseases, particularly to estimate kidney size, shape and position, to give information about kidney function, and to help in diagnosis of abnormalities like cysts, stones, junctional parenchyma and tumors which is shown in Figs. 7–9. This study proposes Grey Level Co-occurrence Matrix (GLCM)-based Probabilistic Principal Component Analysis (PPCA) and Artificial Neural Network (ANN) method for the classification of kidney images. Grey Wolf Optimization (GWO) is used to update the current positions of abnormal kidney images in the discrete searching space, thus getting the optimal feature subset for better classification purposes based on Feed Forward Neural Network (FFNN). The scanned image is pre-processed and the required features are extracted by GLCM, among those, some features are selected by PPCA. Feed Forward Back propagation Neural Network (FFBN) is used to classify the normalities and abnormalities in the part of kidney images. The proposed methodology is implemented in MATLAB platform and the analyzed result produces 98% accuracy using GWO-FFBN technique.