Neural Network Model: Application to Automatic Analysis of Human Sleep

This paper presents an application of the neural network (NN) model for forecasting and filtering problems. Instead of using the gradient descent method as used by the standard back-propagation network, the present study adopted the Levenberg-Marquardt algorithm. It was shown that the Levenberg-Marquardt algorithm tends to reach termination criteria more quickly making the searches employing the full model less time consuming. In the first part of forecasting problem, multilayer feedforward (MLFF) network was constructed by trials to forecast the tidal-level variations at the Chao Phraya river mouth in Thailand. Unlike the well-known conventional harmonic analysis, the neural network model uses a set of previous data for learning and then forecasting directly the time series of tidal levels. It was found that lead time of 1 to 24 hourly tidal levels can be successfully predicted using only a short-time hourly learning data.In the second part of filtering problem, the MLFF network was also used to establish a stage-discharge relationship for tidal river. Comparisons were made among conventional methods, i.e. by using multivariate functions (Linear and power models) and the NN model. The NN model was found to be able to model the transfer function too much higher accuracy than multiple regression analysis. The root mean square errors (RMSE) and the mean relative error (MRE) for the NN model are about 8 cm and 6.4% whereas the same errors for multiple regression analysis are 16 cm and 12%, respectively. The efficiency index for the linear, power, and NN models is 0.4, 0.7, and 0.9, respectively. In addition, the obtained stage-discharge relationship by the NN model can indicate reasonably important behavior of the tidal influences, i.e. there are stronger influences of tide during the dry period than the flood period.

Remote sensing continues to be an invaluable tool in earthquake damage assessments and emergency response. This study evaluates the effectiveness of multilayer feedforward neural networks, radial basis neural networks, and Random Forests in detecting earthquake damage caused by the 2010 Port-au-Prince, Haiti 7.0 moment magnitude (Mw) event. Additionally, textural and structural features including entropy, dissimilarity, Laplacian of Gaussian, and rectangular fit are investigated as key variables for high spatial resolution imagery classification. Our findings show that each of the algorithms achieved nearly a 90% kernel density match using the United Nations Operational Satellite Applications Programme (UNITAR/UNOSAT) dataset as validation. The multilayer feedforward network was able to achieve an error rate below 40% in detecting damaged buildings. Spatial features of texture and structure were far more important in algorithmic classification than spectral information, highlighting the potential for future implementation of machine learning algorithms which use panchromatic or pansharpened imagery alone.

Worst-Case Identification of Uncertain Nonlinear Systems Using Neural Network Models

Background: Data mining (DM) is an approach used in extracting valuable information from environmental processes. This research depicts a DM approach used in extracting some information from influent and effluent wastewater characteristic data of a waste stabilization pond (WSP) in Birjand, a city in Eastern Iran. Methods: Multiple regression (MR) and neural network (NN) models were examined using influent characteristics (pH, Biochemical oxygen demand [BOD 5 ], temperature, chemical oxygen demand [COD], total suspended solids [TSS], total dissolved solid [TDS], electrical conductivity [EC] and turbidity) as the regression input vectors. Models were adjusted to input attributes, effluent BOD 5 (BODout) and COD (CODout). The models performances were estimated by 10-fold external cross-validation. An internal 5-fold cross-validation was also used for the training data set in NN model. The models were compared using regression error characteristic (REC) plot and other statistical measures such as relative absolute error (RAE). Sensitivity analysis was also applied to extract useful knowledge from NN model. Results: NN models (with RAE = 78.71 ± 1.16 for BODout and 83.67 ± 1.35 for CODout) and MR models (with RAE = 84.40% ± 1.07 for BODout and 88.07 ± 0.80 for CODout) indicate different performances and the former was better (P < 0.05) for the prediction of both effluent BOD 5 and COD parameters. For the prediction of CODout the NN model with hidden layer size (H) = 4 and decay factor = 0.75 ± 0.03 presented the best predictive results. For BODout the H and decay factor were found to be 4 and 0.73 ± 0.03, respectively. TDS was found as the most descriptive influent wastewater characteristics for the prediction of the WSP performance. The REC plots confirmed the NN model performance superiority for both BOD and COD effluent prediction. Conclusion: Modeling the performance of WSP systems using NN models along with sensitivity analysis can offer better understanding on exploring the most significant parameters for the prediction of system performance. The findings of this study could build the foundation for prospective work on the characterization of WSP operations and optimization of their performances with a view to conducting statistical approaches.

We demonstrate a method to obtain maximum likelihood weight estimates for a multi-layered feedforward neural network using least squares. The proposed method uses the Fisher's information matrix instead of the Hessian matrix to compute the search direction. Since this matrix is formulated as an inner product, it is guaranteed to be positive definite. The formulation used by the method also provides an interesting way of highlighting the multicollinearity problem in multilayered feedforward networks.

Successful implementation of optimization-driven control techniques, such as model predictive control (MPC), is highly dependent on an accurate and detailed model of the process. As complexity in the system increases, linear approximation used in MPC may result in poor performance since a critical operating point is valid in only a small neighborhood of operation. To address this problem, this paper proposes a collaborative approach that combines linear and data-based models to predict state variables individually. The outputs of these models, along with constraints, are then incorporated into the MPC algorithm. For data-based process model, a multi-layered feed-forward network is used. Additionally, the offset-free technique is applied to eliminate steady-state errors resulting from model-process mismatch. To demonstrate the results, a binary distillation column process which is multivariable and inherently nonlinear is chosen as testbed. We compare the performance of the proposed method to MPC using the full nonlinear model and also to single-model MPC methods for both the linear model and neural network model. We show that the proposed method is only slightly suboptimal with respect to the best available performance and greatly improves over individual methods. In addition, the computational load is reduced when compared to the full nonlinear MPC.

This paper investigates the application of neural networks to frequency line tracking. Recently, hidden Markov models (HMM's) have been successfully applied to this problem, and here, we study a neural network architecture called Mnet, which is based on an underlying Markov model representation. A supervised learning algorithm is developed for Mnet, and a method of analytically deriving the connection weights for the Mnet is also mentioned. Two more conventional neural networks are also studied; a multilayer feedforward network and a multilayer network with feedback. The simulation results show that all three neural networks are comparable in performance to a hidden Markov model when applied to the frequency line tracking problem.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

The multilayer feedforward network has often been used for learning a nonlinear mapping based on a set of examples of the input-output data. In this paper, we present a novel use of the network, in which the example data are not explicitly given. We consider the problem of shape from shading in computer vision, where the input (image coordinates) and the output (surface depth) satisfy only a known differential equation. We use the feedforward network as a parametric representation of the object surface and reformulate the shape from shading problem as the minimization of an error function over the network weights. The stochastic gradient and conjugate gradient methods are used for the minimization. Boundary conditions for either surface depth or surface normal (or both) can be imposed by adjusting the same network at different levels. It is further shown that the light source direction can be estimated, based on an initial guess, by integrating the source estimation with the surface estimation. Extensions of the method to a wider class of problems are discussed. The efficiency of the method is verified by examples of both synthetic and real images.

This paper compares two methods of blind equalization for non-minimum phase systems. Higher order statistics is used to estimate the unknown channel and neural networks are used as equalizers to recover the transmitted signal. Two kinds of neural network equalizers are realized; one is the multilayer feedforward network (MFN) based on the backpropagation algorithm, the other is the minimal resource allocation network (MRAN) which is a radial basis function network. It is shown that the MRAN equalizer performs better and converges faster than MFN equalizer.

Stereology and stochastic geometry can be used as auxiliary tools for diagnostic purposes in tumour pathology. Whether first-order parameters or stochastic-geometric functions are more important for the classification of the texture of biological tissues is not known. In the present study, volume and surface area per unit reference volume, the pair correlation function and the centred quadratic contact density function of epithelium were estimated in three case series of benign and malignant lesions of glandular tissues. The information provided by the latter functions was summarized by the total absolute areas between the estimated curves and their horizontal reference lines. These areas are considered as indicators of deviation of the tissue texture from a completely uncorrelated volume process and from the Boolean model with convex grains, respectively. We used both areas and the first-order parameters for the classification of cases using artificial neural networks (ANNs). Learning vector quantization and multilayer feedforward networks with backpropagation were applied as neural paradigms. Applications included distinction between mastopathy and mammary cancer (40 cases), between benign prostatic hyperplasia and prostatic cancer (70 cases) and between chronic pancreatitis and pancreatic cancer (60 cases). The same data sets were also classified with linear discriminant analysis. The stereological estimates in combination with ANNs or discriminant analysis provided high accuracy in the classification of individual cases. The question of which category of estimator is the most informative cannot be answered globally, but must be explored empirically for each specific data set. Using learning vector quantization, better results could often be obtained than by multilayer feedforward networks with backpropagation.

Friction has been shown to be one of the major contributing factors for problem associated with accuracy in motion control systems. Apart from making the system response slow, it causes steady state error or limit cycles near the reference position for the motion control system. In order to alleviate these problems, various control methods have been introduced and proposed for compensating the friction effect. Among the successful method is model-based friction compensation. Many sophisticated friction models have been proposed by researchers. Unfortunately, selecting and developing accurate models for friction compensation for a particular application has been historically challenging and trouble some due to complexity of parametrically modeling of the friction nonlinearities. Motivated by the need for simple and at the same time effective friction compensation in motion control system, AI-based friction model using multilayer feedforward network (MFN) is proposed and developed to estimate and compensate the frictional dynamics of a DC motor driven motion control system. The effectiveness of the developed MFN-based friction model to compensate the friction is evaluated experimentally on a rotary experimental motion system, and compared with mathematical model approach. The experimental results show that the friction compensation using the MFN-based friction model is more effectively to compensate for the effect of the friction than that on mathematical model of friction.

EACMFS Prizes and Awards

A new approach for detecting multiple faults in digital circuits is presented in this paper, which uses neural network technique to generate the test vectors and to diagnose the multiple faults. First of all, the optimal neural network model corresponding to an arbitrary digital circuit is used. For a logic circuit with m inputs and n outputs, the optimal neural networks having m+n neurons can represent the logic function of the circuit. Secondly, the test vectors of multiple faults in the circuit are obtained by using evolutionary strategies to compute the minimum energy states of the optimal neural network. Thirdly, the multi-layer feed forward network and back propagation algorithm are used to diagnose the multiple faults in the circuit. The design of both network configurations and samples patterns are given in detail. Experimental results show that the approaches proposed in this paper are effective for detecting and diagnosing multiple faults in digital circuits

The hypnogram is a schematic representation of sleep dynamics as a sequence of stages along the night. The authors have shown, in previous work, the adequacy of syntactic modeling in automatic sleep analysis. According to this approach, the hypnogram is viewed as expression of a language whose temporal organization is described by stochastic context-free grammars. Here, the authors propose a hybrid model combining syntactic and semantic information, where stage duration is taken into account in terms of attributed grammars. This model enables hypnogram simulation, by means of language generation, and provides improvement in classification accuracy. The method is applied in automatic hypnogram analysis of a control group and a population of dysthymic patients.

The photocatalytic degradation of carbamazepine and prediction by artificial neural networks