Training of Multilayer Perceptron Neural Networks by Using Cellular Genetic Algorithms

The need for an accurate reasoning algorithm is usually necessitated by the sensitivity of domain of (medicine as example) application of such algorithms. Most reasoning algorithms for medical diagnosis are either limited by their techniques or accuracy and efficiency. Even the Select and Test (ST) algorithm which is considered a more approximate reasoning algorithm is also limited by its approach of using bipartite graph in modeling domain knowledge and making inference through the use of orthogonal vector projection for estimating likelihood of diagnosis at the clinical decision stage (induction). While the bipartite graph knowledge base lacks n-ary use of predicate on concepts, orthogonal vector projection on the other hand has high computation for the inference process. The aim of this paper is to enhance ST algorithm for improved performance and accuracy. First, we propose the use of ontologies and semantic web based rule for knowledge representation so as to provide support for inference making. Furthermore, three major improvements were added to ST algorithm to aid the improvement of its approximation. Secondly, we designed an inference making procedure to enable interaction with the knowledge base mentioned earlier. Thirdly, we model Hill’s Criteria of Causation into clinical decision stage of ST to overcome the limitation of orthogonal vector projection. Lastly, the improved ST algorithm was largely represented and described using set notations (though implemented as linked-list and queues) and mathematical notations. The result of the improved ST algorithm revealed a sensitivity of 0.81 and 0.89 and specificity of 0.82 and 1.0 in the Wisconsin Breast Cancer Database (WBCD) and Wisconsin Diagnostic Breast Cancer (WDBC) datasets respectively. In addition, the accuracy obtained from the proposed algorithm was 86.0% and 88.72% for the Wisconsin Breast Cancer Database (WBCD) and Wisconsin Diagnostic Breast Cancer (WDBC) datasets respectively. This enhancement in accuracy was obtained at a slowdown time due to the reasoning process and ontology parsing task added to the enhanced system. However, there was an improvement in the accuracy and inference power of the resulting system.

This work intends to an integration of implementing an automated diagnostic systems for breast cancer detection using Artificial Neural Network (ANN) in FPGA. In the world, breast cancer is the fifth most common cause of cancer death. So better classification system is needed for diagnosing breast cancer disease. In this work, the training and testing of the Multilayer Perceptron Neural Network (MLPNN) with Back Propagation Network (BPN) is done with the attributes of the record of the Wisconsin Breast Cancer Database (WBCD). The neural network lacks the flexibility during off line training. In order to overcome the flexibility, it is necessary to train and test the network on on-chip neural network using FPGA. The purpose is to determine the cancer of patients either having benign or malignant through an FPGA based implementation of smart instrument. In order to implement the hardware, VERILOG coding is done for ANN and synthesized by Xilinx family XC5VLX50TFFT1136 FPGA Virtex 5 board using XILINX ISE tool to get the netlist of ANN. Finally the netlist is mapped to FPGA and the hardware functionality is verified. The correct classification rate of proposed system is 90.83%.

Optimal breast cancer classification using Gauss–Newton representation based algorithm

Error Correcting Output Codes, ECOC, is an output representation method capable of discovering some of the errors produced in classification tasks. This paper describes the application of ECOC to the training of feed forward neural networks, FFNN, for improving the overall accuracy of classification systems. Indeed, to improve the generalization of FFNN classifiers, this paper proposes an ECOC-Based training method for Neural Networks that use ECOC as the output representation, and adopts the traditional Back-Propagation algorithm, BP, to adjust weights of the network. Experimental results for face recognition problem on Yale database demonstrate the effectiveness of our method. With a rejection scheme defined by a simple robustness rate, high reliability is achieved in this application.

ADANOISE: Training neural networks with adaptive noise for imbalanced data classification

Deep Learning is a new area of Machine Learning research that deals with learning different levels of representation and abstraction in order to move Machine Learning closer to Artificial Intelligence. Artificial Metaplasticity are Artificial Learning Algorithms based on modelling higher level properties of biological plasticity: the plasticity of plasticity itself, so called Biological Metaplasticity. Artificial Metaplasticity aims to obtain general improvements in Machine Learning based on the experts generally accepted hypothesis that the Metaplasticity of neurons in Biological Brains is of high relevance in Biological Learning. This paper presents and discuss the results of applying different Artificial Metaplasticity implementations in Multilayer Perceptrons at artificial neuron learning level. To illustrate their potential, a relevant application that is the objective of state-of-the-art research has been chosen: the diagnosis of breast cancer data from the Wisconsin Breast Cancer Database. It then concludes that Artificial Metaplasticity also may play a high relevant role in Deep Learning.

In this paper we introduce an advanced supervised training method for neural networks. It is based on Jacobian rank deficiency and it is formulated, in some sense, in the spirit of the Gauss-Newton algorithm. The Levenberg-Marquardt algorithm, as a modified Gauss-Newton, has been used successfully in solving nonlinear least squares problems including neural-network training. It outperforms (in terms of training accuracy, convergence properties, overall training time, etc.) the basic backpropagation and its variations with variable learning rate significantly, however, with higher computation and memory complexities within each iteration. The new method developed in this paper is aiming at improving convergence properties, while reducing the memory and computation complexities in supervised training of neural networks. Extensive simulation results are provided to demonstrate the superior performance of the new algorithm over the Levenberg-Marquardt algorithm.

A homotopy method for training neural networks

Quantitative structure–activity relationships by evolved neural networks for the inhibition of dihydrofolate reductase by pyrimidines

With the popularity of smart wearable systems, sensor signal processing poses more challenges to machine learning in embedded scenarios. For example, traditional machine-learning methods for data classification, especially in real time, are computationally intensive. The deployment of Artificial Intelligence algorithms on embedded hardware for fast data classification and accurate fall detection poses a huge challenge in achieving power-efficient embedded systems. Therefore, by exploiting the associative memory feature of Hopfield Neural Network, a hardware module has been designed to simulate the Neural Network algorithm which uses sensor data integration and data classification for recognizing the fall. By adopting the Hebbian learning method for training neural networks, weights of human activity features are obtained and implemented/embedded into the hardware design. Here, the neural network weight of fall activity is achieved through data preprocessing, and then the weight is mapped to the amplification factor setting in the hardware. The designs are checked with validation scenarios, and the experiment is completed with a Hopfield neural network in the analog module. Through simulations, the classification accuracy of the fall data reached 88.9% which compares well with some other results achieved by the software-based machine-learning algorithms, which verify the feasibility of our hardware design. The designed system performs the complex signal calculations of the hardware’s feedback signal, replacing the software-based method. A straightforward circuit design is used to meet the weight setting from the Hopfield neural network, which is maximizing the reusability and flexibility of the circuit design.

Automated retrosynthetic planning algorithms are a research area of increasing importance. Automated reaction-template extraction from large datasets, in conjunction with neural-network-enhanced tree-search algorithms, can find plausible routes to target compounds in seconds. However, the current method for training neural networks to predict suitable templates for a given target product leads to many predictions that are not applicable in silico. Most templates in the top 50 suggested templates cannot be applied to the target molecule to perform the virtual reaction. Here, we describe how to generate data and train a neural network policy that predicts whether templates are applicable or not. First, we generate a massive training dataset by applying each retrosynthetic template to each product from our reaction database. Second, we train a neural network to perform near-perfect prediction of the applicability labels on a held-out test set. The trained network is then joined with a policy model trained to predict and prioritize templates using the labels from the original dataset. The combined model was found to outperform the policy model in a route-finding task using 1700 compounds from our internal drug-discovery projects.

There are numerous advantages of deep neural network surrogate modeling for response time-history prediction. However, due to the high cost of refined numerical simulations and actual experiments, the lack of data has become an unavoidable bottleneck in practical applications. An iterative self-transfer learning method for training neural networks based on small datasets is proposed in this study. A new mapping-based transfer learning network, named as deep adaptation network with three branches for regression (DAN-TR), is proposed. A general iterative network training strategy is developed by coupling DAN-TR and the pseudo-label strategy, and the establishment of corresponding datasets is also discussed. Finally, a complex component is selected as a case study. The results show that the proposed method can improve the model performance by near an order of magnitude on small datasets without the need of external labeled samples, well behaved pre-trained models, additional artificial labeling, and complex physical/mathematical analysis.

The paper presents a novel architecture and method for training neural networks to produce synthesized speech in a particular voice and speaking style, based on a small quantity of target speaker/style training data. The method is based on neural network embedding, i.e. mapping of discrete variables into continuous vectors in a low-dimensional space, which has been shown to be a very successful universal deep learning technique. In this particular case, different speaker/style combinations are mapped into different points in a low-dimensional space, which enables the network to capture the similarities and differences between speakers and speaking styles more efficiently. The initial model from which speaker/style adaptation was carried out was a multi-speaker/multi-style model based on 8.5 hours of American English speech data which corresponds to 16 different speaker/style combinations. The results of the experiments show that both versions of the obtained system, one using 10 minutes and the other as little as 30 seconds of target data, outperform the state of the art in parametric speaker/style-dependent speech synthesis. This opens a wide range of application of speaker/style dependent speech synthesis based on small quantities of training data, in domains ranging from customer interaction in call centers to robot-assisted medical therapy.

The use of passive 118-GHz O/sub 2/ observations of rain cells for precipitation cell-top altitude estimation is demonstrated by using a multilayer feedforward neural network retrieval system. Rain cell observations at 118 GHz were compared with estimates of the cell-top altitude obtained by optical stereoscopy. The observations were made with 2-4-km horizontal spatial resolution by using the millimeter-wave temperature sounder (MTS) scanning spectrometer aboard the NASA ER-2 research aircraft during the Genesis of Atlantic Lows Experiment (GALE) and the Cooperative Huntsville Meteorological Experiment (COHMEX) in 1986. The neural network estimator applied to MTS spectral differences between clouds, and nearby clear air yielded an rms discrepancy of 1.76 km for a combined cumulus, mature, and dissipating cell set and 1.44 km for the cumulus-only set. An improvement in rms discrepancy to 1.36 km was achieved by including additional MTS information on the absolute atmospheric temperature profile. An incremental method for training neural networks was developed that yielded robust results, despite the use of as few as 56 training spectra. Comparison of these results with a nonlinear statistical estimator shows that superior results can be obtained with a neural network retrieval system. Imagery of estimated cell-top altitudes was created from 118-GHz spectral imagery gathered from CAMEX, September through October 1993, and from cyclone Oliver, February 7, 1993.

Building selective ensembles of Randomization Based Neural Networks with the successive projections algorithm