Artificial applicability labels for improving policies in retrosynthesis prediction

Error Correcting Output Codes, ECOC, is an output representation method capable of discovering some of the errors produced in classification tasks. This paper describes the application of ECOC to the training of feed forward neural networks, FFNN, for improving the overall accuracy of classification systems. Indeed, to improve the generalization of FFNN classifiers, this paper proposes an ECOC-Based training method for Neural Networks that use ECOC as the output representation, and adopts the traditional Back-Propagation algorithm, BP, to adjust weights of the network. Experimental results for face recognition problem on Yale database demonstrate the effectiveness of our method. With a rejection scheme defined by a simple robustness rate, high reliability is achieved in this application.

ADANOISE: Training neural networks with adaptive noise for imbalanced data classification

In this paper we introduce an advanced supervised training method for neural networks. It is based on Jacobian rank deficiency and it is formulated, in some sense, in the spirit of the Gauss-Newton algorithm. The Levenberg-Marquardt algorithm, as a modified Gauss-Newton, has been used successfully in solving nonlinear least squares problems including neural-network training. It outperforms (in terms of training accuracy, convergence properties, overall training time, etc.) the basic backpropagation and its variations with variable learning rate significantly, however, with higher computation and memory complexities within each iteration. The new method developed in this paper is aiming at improving convergence properties, while reducing the memory and computation complexities in supervised training of neural networks. Extensive simulation results are provided to demonstrate the superior performance of the new algorithm over the Levenberg-Marquardt algorithm.

Building selective ensembles of Randomization Based Neural Networks with the successive projections algorithm

This study proposes a model that can be used for monitoring and identifying consumer electrical loads using Neural Network and Feed-Forward Backpropagation training methods. The modeling design is made in Simulink-Matlab software. The neural network training error obtained is 2.729%, then this is used for the identification process of electrical loads, which in this study used 21 load combinations. While the error test results obtained are 1.325476%. In this study, a new model is proposed for monitoring and identifying consumer electricity consumption by applying the concept of Non-Intrusive Load Monitoring (NILM) combined with the Artificial Neural Network (ANN) method.

Radio signal-based (indoor) localisation technique is important for IoT applications such as smart factory and warehouse. Through machine learning, especially neural networks methods, more accurate mapping from signal features to target positions can be achieved. However, different radio protocols, such as WiFi, Bluetooth, etc., have different features in the transmitted signals that can be exploited for localisation purposes. Also, neural networks methods often rely on carefully configured models and extensive training processes to obtain satisfactory performance in individual localisation scenarios. The above poses a major challenge in the process of determining neural network model structure, or hyperparameters, as well as the selection of training features from the available data. This paper proposes a neural network model hyperparameter tuning and training method based on Bayesian optimisation. Adaptive selection of model hyperparameters and training features can be realised with minimal need for manual model training design. With the proposed technique, the training process is optimised in a more automatic and efficient way, enhancing the applicability of neural networks in localisation.

In many real-world scenarios, labeled data for a specific machine learning task is costly to obtain. Semi-supervised training methods make use of abundantly available unlabeled data and a smaller number of labeled examples. We propose a new framework for semi-supervised training of deep neural networks inspired by learning in humans. Associations are made from embeddings of labeled samples to those of unlabeled ones and back. The optimization schedule encourages correct association cycles that end up at the same class from which the association was started and penalizes wrong associations ending at a different class. The implementation is easy to use and can be added to any existing end-to-end training setup. We demonstrate the capabilities of learning by association on several data sets and show that it can improve performance on classification tasks tremendously by making use of additionally available unlabeled data. In particular, for cases with few labeled data, our training scheme outperforms the current state of the art on SVHN.

In this study, performances comparison to discriminate five mental states of five artificial neural network (ANN) training methods were investigated. Wavelet Packet Transform (WPT) was used for feature extraction of the relevant frequency bands from raw electroencephalogram (EEG) signals. The five ANN training methods used were (a) Gradient Descent Back Propagation (b) Levenberg-Marquardt (c) Resilient Back Propagation (d) Conjugate Learning Gradient Back Propagation and (e) Gradient Descent Back Propagation with movementum.

This paper deals with a method for training neural networks by using cellular genetic algorithms (CGA). This method was implemented as software, CGANN-Trainer, which was used to generate binary classifiers for recognition of patterns associated with breast cancer images in a multi-objective optimization problem. The results reached by the CGA with the Wisconsin Breast Cancer Database, and the Wisconsin Diagnostic Breast Cancer Database, were compared with some other methods previously reported using the same databases, proving to be an interesting alternative.KeywordsNeural networksgenetic algorithmscellular automatamulti-objective classification

Artificial neural networks (ANN) have been widely used in the field of data classification. Normally, training of neural network is applied with the traditional back propagation technique. As, this approach has various drawbacks, training of neural network is done with Particle Swarm Optimization (PSO). PSO has been widely used to solve the diverse kind of optimization problems. Population initialization performs a significant role in meta-heuristic algorithms. This paper describes a new initialization population approach Log Logistic termed as PSOLL-NN to create the initialization of the swarm. The proposed algorithm has been tested for weight optimization of feed forward neural network; and compared with back propagation Algorithm (BPA), standard PSO (PSO-NN), PSO initialized with Halton Sequence (PSOH-NN), Torus sequence (PSOT-NN) and Sobol sequence (PSOS-NN). The experimental results show that the proposed technique performed exceptionally better than the other traditional techniques. Moreover, the outcome of our work presents a foresight that how the proposed initialization technique can be used as an efficient alternative to standard training approaches for the data classification problems.

Logarithmic barrier methods are developed for training supervised neural networks. Two algorithms are presented: a deterministic logarithmic barrier and a stochastic logarithmic barrier. Specifically, we consider neural network training as a nonlinear programming problem and use barrier methods to find the optimal weights. Furthermore, we put constraints on the weights to avoid network paralysis. The search direction is derived using a recursive prediction error method (RPEM) that approximates the inverse of the Hessian of a logarithmic error function iteratively. The weights move on a center trajectory in the interior of the feasible weight space and have good convergence properties. In the case of the stochastic version, random fluctuations are added to the weights in order to escape local minima. Ill-conditioning problems are discussed and computational experiments are provided.Keywordsneural networksdeterministic and stochastic logarithmic barrier algorithmsinterior point methodsNewton type methodsneural network trainingANN training parallelization

There are numerous advantages of deep neural network surrogate modeling for response time-history prediction. However, due to the high cost of refined numerical simulations and actual experiments, the lack of data has become an unavoidable bottleneck in practical applications. An iterative self-transfer learning method for training neural networks based on small datasets is proposed in this study. A new mapping-based transfer learning network, named as deep adaptation network with three branches for regression (DAN-TR), is proposed. A general iterative network training strategy is developed by coupling DAN-TR and the pseudo-label strategy, and the establishment of corresponding datasets is also discussed. Finally, a complex component is selected as a case study. The results show that the proposed method can improve the model performance by near an order of magnitude on small datasets without the need of external labeled samples, well behaved pre-trained models, additional artificial labeling, and complex physical/mathematical analysis.

This paper addresses the correct choice and combination of confidence measures in large vocabulary speech recognition tasks. We classify single words within continuous as well as large vocabulary utterances into two categories: utterances within the vocabulary which are recognized correctly, and other utterances, namely misrecognized utterances or (less frequent) out-of-vocabulary (OOV). To this end, we investigate the classification error rate (CER) of several classes of confidence measures and transformations. In particular, we employed data-independent and data-dependent measures. The transformations we investigated include mapping to single confidence measures and linear combinations of these measures. These combinations are computed by means of neural networks trained with Bayes-optimal, and with Gardner-Derrida-optimal criteria. Compared to a recognition system without confidence measures, the selection of (various combinations of) confidence measures, the selection of suitable neural network architectures and training methods, continuously improves the CER.

In the context of command-and-control applications, we exploit confidence measures in order to classify single-word utterances into two categories: utterances within the vocabulary which are recognized correctly, and other utterances, namely out-ofvocabulary (OOV) or misrecognized utterances. To this end, we investigate the classification error rate (CER) of several classes of confidence measures and transformations. In particular, we employed data-independent and data-dependent measures. The transformations we investigated include mapping to single confidence measures, LDA-transformed measures, and other linear combinations of these measures. These combinations are computed by means of neural networks trained with Bayesoptimal, and with Gardner-Derrida-optimal criteria. Compared to a recognition system without confidence measures, the selection of (various combinations of) confidence measures, the selection of suitable neural network architectures and training methods, continuously improves the CER. Additionally, we found that a linear perceptron generalizes better than a non-linear backpropagation network.

Generation of the human anti-MUC1 peptide through neural network training and monomeric design method. Analyzing 9-mer peptide potential computationally for treatment of HER2-positive breast cancer. With the advancements of cancer genome atlas project (TCGA), cancer dependancy project (DepMap) and human protein atlas (HPA), large-scale datasets are generated for oncology studies. However, after development of redefined breast cancer drug targets, there are key issues in successful breast cancer treatments that needed to be pursued which paved the pathway for new approaches or strategies. In that respect, our research data aimed to represent a new aspect of breast cancer drug development studies. Extract human MUC1 sequences from various databases. Perform neural networking method for novel peptides sequences. Analyze the potentiality of generated heteroclitic peptide sequences for suitable vaccine candidate for breast cancer treatment. Input scaffolds of protein database (PDB) files for human MUC1 were retrieved and loaded into Evo design server with monomeric based design option. Further, neural network training approaches were followed and other computational tools were used for alignment-independent prediction of protective antigens and subunit vaccines potency of designed heteroclitic peptides. Study findings revealed two human anti-MUC1 heteroclitic peptides of 9mers (WAVWTYVSV, FMSFYIMNL), which showed the lowest energy cluster and sequence identity, normalized relative error rate of secondary structure, solvent accessibility, backbone torsion angles for neural networking and RMSD values in evolutionary profiling, and online MHCPred IC50 interaction values. VaxiGen v2.0 server revealed subunit vaccine potency values of in-silico designed two heteroclitic peptides were 0.1551 (WAVWTYVSV) and 0.3508 (FMSFYIMNL) with a threshold value of 0.5 followed by AllerTOP v2.0 for their allergenicity nature in immunogenic reactions. Computationally designed heteroclitic peptide WAVWTYVSV indicated promising values which can be utilised as drug delivery or tumour marker candidate in the treatment of human breast cancer by eliciting lyse of tumor cells.