Efficient sequential and batch learning artificial neural network methods for classification problems

Survival analysis (SA) prediction involves the prediction of the time until an event of interest occurs (TTE), based on input attributes. The main challenge of SA is instances where the event is not observed (censored), typically through an alternative (censoring) event. Most SA prediction methods suffer from drawbacks limiting the usage of advanced machine learning methods: Ignoring the input of the censored samples, no separation between model and loss, and typical small datasets and high input dimensions. We propose a loss function, denoted suRvival Analysis lefT barrIer lOss (RATIO), that explicitly incorporates the censored samples input in the prediction. RATIO accounts for the difference between censored and uncensored samples, by only considering censoring events occurring after the predicted, and through a linear term on the uncensored data event time. RATIO can be used with any prediction model. We further propose FIESTA a data augmentation method, combining the TTE of uncensored samples with the input of censored samples. We show that RATIO drastically improves the precision and reduces the bias of SA prediction in both models and real-life SA problems, and FIESTA allows for the inclusion of high-dimension data in SA methods even with a small number of uncensored samples.

With the increase in the number of electronic medical records, precise classification of medical records can be use to accurately diagnose diseases. In medical texts, there are differences in the number of texts of different diseases, especially for specific diseases and a small number of samples have brought great challenges to medical text classification. In response to this problem, this paper proposes an accurate text classification model base on transfer learning combined with attention mechanism neural network, and finally obtains accurate category text through two-step classification. The model first uses the attention mechanism long-term and short-term cyclic neural network to extract the overall tumor sample from the mass of unbalanced medical record texts through the common characteristics of the tumor medical record, and uses the similarity between diseases to combine with the convolutional neural network through migration learning. The characteristics of each type of tumor disease are migrate to achieve the effect of precise classification training. The precise training model is used to finally realize the precise classification of tumor diseases. The model was tested on a private tumor medical data set, and the results showed that the accuracy of the tumor data set (precision) was increased by 5% compare to the average of the baseline optimal classification model. The increase in accuracy is important for accurate classification significance.

Sequential learning navigation method and general correction model for Autonomous Underwater Vehicle

Sequential and Reinforcement Learning for Neurocontrol

Given the thousands of genes and the small number of samples, gene selection has emerged as an important research problem in microarray data analysis. Support Vector Machine-Recursive Feature Elimination (SVM-RFE) is one of a group of recently described algorithms which represent the stat-of-the-art for gene selection. Just like SVM itself, SVM-RFE was originally designed to solve binary gene selection problems. Several groups have extended SVM-RFE to solve multiclass problems using one-versus-all techniques. However, the genes selected from one binary gene selection problem may reduce the classification performance in other binary problems. In the present study, we propose a family of four extensions to SVM-RFE (called MSVM-RFE) to solve the multiclass gene selection problem, based on different frameworks of multiclass SVMs. By simultaneously considering all classes during the gene selection stages, our proposed extensions identify genes leading to more accurate classification.

Gaussian process (GP) regression or kriging has been extensively applied in the engineering literature for the purposes of building a cheap-to-evaluate surrogate, within the contexts of multi-fidelity modeling, model calibration, and design optimization. With the ongoing automation of manufacturing and industrial practices as a part of Industry 4.0, there has been a greater need for advancing GP regression techniques to handle challenges such as high input dimensionality, data paucity or big data problems, these consist primarily of proposing efficient design of experiments, optimal data acquisition strategies, sparsifying covariance kernels, and other mathematical tricks. In this work, our attention is focused on the challenges of efficiently training a GP model, which, to the authors opinion, has attracted very little attention and is to-date poorly addressed. The performance of widely used training approaches such as maximum likelihood estimation and Markov Chain Monte Carlo (MCMC) sampling can deteriorate significantly in high-dimensional and big data problems and can lead to cost deficient implementations of critical importance to many industrial applications. Here, we compare an Adaptive Sequential Monte Carlo (ASMC) sampling algorithm to classic MCMC sampling strategies and we demonstrate the effectiveness of our implementation on several mathematical problems and challenging industry applications of varying complexity. The computational time savings of the ASMC approach manifest in large-scale problems helping us to push the boundaries of applicability and scalability of GPs for model calibration in various domains of the industry, including but not limited to design automation, design engineering, smart manufacturing, predictive maintenance, and supply chain manufacturing.

Multi-sensor real-time monitoring of dam behavior using self-adaptive online sequential learning

In statistical pattern recognition, it is important to estimate true distribution of patterns precisely to obtain high recognition accuracy. Normal mixtures are sometimes used for representing distributions. However, precise estimation of the parameters of normal mixtures requires a great number of sample patterns, especially for high dimensional vectors. For some pattern recognition problems, such as face recognition, very high dimensional feature vectors are necessary and there are always not enough training samples compared with the dimensionality. We present a method to estimate the distributions based on normal mixtures with small number of samples. The proposed algorithm is applied to face recognition problem which requires high dimensional feature vectors. Experimental results show the effectiveness of the proposed algorithm.

It is widely assumed that language in some form or other originated by piggybacking on pre-existing learning mechanism not dedicated to language. Using evolutionary connectionist simulations, we explore the implications of such assumptions by determining the effect of constraints derived from an earlier evolved mechanism for sequential learning on the interaction between biological and linguistic adaptation across generations of language learners. Artificial neural networks were initially allowed to evolve “biologically” to improve their sequential learning abilities, after which language was introduced into the population. We compared the relative contribution of biological and linguistic adaptation by allowing both networks and language to change over time. The simulation results support two main conclusions: First, over generations, a consistent head-ordering emerged due to linguistic adaptation. This is consistent with previous studies suggesting that some apparently arbitrary aspects of linguistic structure may arise from cognitive constraints on sequential learning. Second, when networks were selected to maintain a good level of performance on the sequential learning task, language learnability is significantly improved by linguistic adaptation but not by biological adaptation. Indeed, the pressure toward maintaining a high level of sequential learning performance prevented biological assimilation of linguistic-specific knowledge from occurring.

Surrogate modeling is essential in reducing computational costs for history-matching applications. Yet, traditional deep learning-based surrogate models cannot cope with high dimensional input parameters, such as the permeability field. This work introduces a robust method to automate the history matching process utilizing the Bayesian inversion assisted by a hybrid convolutional neural network and long short-term memory (CNN-LSTM) model and principal component analysis (PCA) method. The method includes five main steps. Step 1: Generate a high-spatial permeability field using a geostatistical approach. Step 2: use the PCA to reduce the dimensionality of the permeability fields, followed by using PCA to generate permeability fields and perform simulations. Step 3: construct the CNNLSTM to map the nonlinear relationship between the extracted features from PCA and the sequential outputs, such as the pressure response. Here, Bayesian optimization is employed to automate hyperparameter tuning. Step 4: perform the Bayesian inversion to inverse the high dimensional inputs, e.g., permeability field, in which the CNN-LSTM serves as the forward model to reduce the computational cost. The inversed PCA features are then fed into the PCA to recover the high dimensional inputs. Step 5: check convergence and if the errors are significant between the inversed high dimensional permeability field and the ground truth, revisit the construction of the CNN-BiLSTM and the prior information for the uncertainty parameters. A 2D reservoir model demonstrates the proposed history-matching method. We can inverse the high dimensional inputs (e.g., permeability field) with minor errors between the prediction and ground truth. We propose a Bayesian inversion assisted by a hybrid CNN-LSTM model and PCA method for high-dimensional parameter inversion, which is superior to the traditional models regarding accuracy and efficiency. This method enables us to perform history matching for reservoir simulation with high dimensional inputs and significant uncertainties.

A sequential subspace learning method and its application to dynamic texture analysis

Plants have an important role in human life. Several plants can be used for daily life, namely as food, in the health sector, to become a main ingredient for the industry. Plant type classification techniques using data mining methods become one of the efforts to help humans produce more accurate and consistent classifications. The learning process in the classification method requires good dataset quality, where a small number of datasets will affect the results of the classification. The main objective of this research is to test the Artificial Neural Network (ANN) method for classifying plant species in a relatively small dataset. Three stages are proposed, namely preprocessing using image segmentation thresholding methods and morphological operations, and the extraction of metric and eccentricity features. Based on the results of testing the ANN method can also work well with relatively small datasets, which results in accuracy reaching 96% with the number of training data 125 and testing data 25.

High dimensional inputs -e.g.,

Convolutional neural networks (CNN) have recently seen tremendous success in various computer vision tasks. However, their application to problems with high dimensional input and output, such as high-resolution image and video segmentation or 3D medical imaging, has been limited by various factors. Primarily, in the training stage, it is necessary to store network activations for back-propagation. In these settings, the memory requirements associated with storing activations can exceed what is feasible with current hardware, especially for problems in 3D. Motivated by the propagation of signals over physical networks, that are governed by the hyperbolic Telegraph equation, in this work we introduce a fully conservative hyperbolic network for problems with high-dimensional input and output. We introduce a coarsening operation that allows completely reversible CNNs by using a learnable discrete wavelet transform and its inverse to both coarsen and interpolate the network state and change the number of channels. We show that fully reversible networks are able to achieve results comparable to the state of the art in 4D time-lapse hyper-spectral image segmentation and full 3D video segmentation, with a much lower memory footprint that is a constant independent of the network depth. We also extend the use of such networks to variational auto-encoders, where optimization begins from an exact recovery and we discover the level of compression through optimization.

At present, machine learning techniques is frequently used to extraction and/or prediction some information from enormous collected data. Traditional machine learning is performed batch learning like a popular one, Artificial Neural Network (ANN). That means it can’t integrate new information into already trained model but it is retrained from scratch. Online Sequential Extreme Learning Machine (OS-ELM) is a one of online incremental machine learning techniques that can learn and update model from new receive data without to retrain the model. This paper shows the result of comparison experiment between ANN and OS-ELM and found that, the OS-ELM has acceptable performance in situation of few initial data for training and/or statistic properties of data is changing while it working.