TFFN: Two hidden layer feed forward network using the randomness of extreme learning machine

Evolutionary extreme learning machine

The single-hidden-layer feedforward neural networks (SLFNs) are frequently used in machine learning due to their ability which can form boundaries with arbitrary shapes if the activation function of hidden units is chosen properly. Most learning algorithms for the neural networks based on gradient descent are still slow because of the many learning steps. Recently, a learning algorithm called extreme learning machine (ELM) has been proposed for training SLFNs to overcome this problem. It randomly chooses the input weights and hidden-layer biases, and analytically determines the output weights by the matrix inverse operation. This algorithm can achieve good generalization performance with high learning speed in many applications. However, this algorithm often requires a large number of hidden units and takes long time for classification of new observations. In this paper, a new approach for training SLFNs called least-squares extreme learning machine (LS-ELM) is proposed. Unlike the gradient descent-based algorithms and the ELM, our approach analytically determines the input weights, hidden-layer biases and output weights based on linear models. For training with a large number of input patterns, an online training scheme with sub-blocks of the training set is also introduced. Experimental results for real applications show that our proposed algorithm offers high classification accuracy with a smaller number of hidden units and extremely high speed in both learning and testing.

Extreme Learning Machine (ELM) is a learning algorithm proposed recently to train single hidden layer feed forward networks (SLFN). It has many attractive properties that include better generalization performance and very fast learning. ELM starts by assigning random values to the input weights and hidden biases and then in one step it determines the output weights using Moore-Penrose generalized inverse. Despite the aforementioned advantages, ELM performance might be affected by the random initialization of weights and biases or by the large generated network which might contain unnecessary number of neurons. In order to increase the generalization performance and to produce more compact networks, a hybrid model that combines ELM with competitive swarm optimizer (CSO) is proposed in this paper. The proposed model (CSONN-ELM) optimizes the weights and biases and dynamically determines the most appropriate number of neurons. To evaluate the effectiveness of the CSONN-ELM, it is experimented using 23 benchmark datasets, and compared to a set of static rules extracted from literature that are used to determine the number of neurons of SLFN. Moreover, it is compared to two dynamic methods that are used to enhance the performance of ELM, that are Optimally pruned ELM (OP-ELM) and metaheuristic based ELMs (Particle Swarm Optimization-ELM and Differential Evolution-ELM). The obtained results show that the proposed method enhances the generalization performance of ELM and overcomes the static and dynamic methods.

Structures with fixed-base will produce high accelerations and inter-storey drifts and move laterally during earthquake. The presence of base isolation devices between ground and the structure, reduces the structural vibrations. To maintain the seismic response of structures within safety, service and comfort limits, the combination of base isolators and feedback controllers have been proposed in recent years. This paper proposes a discrete direct adaptive extreme learning machine (ELM) controller for the active control of non linear base isolated building with hysteretic isolation system. The controller is constructed based on a single hidden layer feed forward network and the parameters of the network are adapted using extreme learning machine (ELM) algorithm. In this work, different from the original ELM algorithm the output weights of the network are updated using Lyapunov stability approach, to guarantee the stability of the structure. The performance of the proposed controller is verified on a non-linear three dimensional benchmark base-isolated structure by exciting the structure with three earthquake samples. The result shows that the proposed controller is effective in reducing the seismic responses of the isolation members as well as the superstructure.

Extreme learning machine (ELM) is an easy-to use and effective learning algorithm of single-hidden layer feed-forward neural networks (SLFNs). The classical learning algorithm in neural network, e. g. backpropagation, requires setting several user-defined parameters and may get into local minimum. However, ELM only requires setting the number of hidden neurons and the activation function. It does not require adjusting the input weights and hidden layer biases during the implementation of the algorithm, and it produces only one optimal solution. Therefore, ELM has the advantages of fast learning speed and good generalization performance. In this paper, ELM is introduced in predicting reservoir permeability. By comparing to SVM, we analyze its feasibility and advantages in reservoir permeability prediction. The experimental results show that ELM has similar accuracy compared to SVR, but it has obvious advantages in parameter selection and learning speed.

Multiple faults are easily confused with single faults. In order to identify multiple faults more accurately, a highly efficient learning method is proposed based on a double parallel two-hidden-layer extreme learning machine, called DPT-ELM. The DPT-ELM method is a variant of an extreme learning machine (ELM). There are some issues with ELM. First, achieving a high accuracy requires too many hidden nodes; second, the direct connection between the input layer and the output layer is ignored. Accordingly, to deal with the above-mentioned problems, DPT-ELM extends the single-hidden-layer ELM to a two-hidden-layer ELM, which can achieve a desired performance with fewer hidden nodes. In addition, a direct connection is built between the input layer and the output layer. Since the input layer weights and the thresholds of the two hidden layers are determined randomly, this simplifies the improved model and shortens the calculation time. Additionally, to improve the signal to noise ratio (SNR), an adaptive waveform decomposition (AWD) algorithm is used to denoise the vibration signal. Then, the denoised signal is used to extract the eigenvalues by the time-domain and frequency-domain methods. Finally, the eigenvalues are input to the DPT-ELM classifier. In this paper, two groups of rolling bearing data at different speeds, which were collected from a real experimental platform, are used to test the method. Each set of data includes three single fault states, two complex fault states and a healthy state. The experimental results demonstrate that the DPT-ELM method achieves fast learning speed and a high accuracy. Moreover, based on 10-fold cross-validation, it proves to be an effective method to improve the accuracy with fewer hidden nodes.

Dynamic adjustment of hidden layer structure for convex incremental extreme learning machine

Pulsed Nd:YAG laser has high intensity and high quality beam characteristics, which can be used to produce micro-grooves and micro-turning surface on advanced engineering ceramics. The present research attempts to develop mathematical models by using response surface methodology approach for correlating the machining process parameters and the process responses during laser micro-turning of aluminum oxide (Al2O3) ceramics. The process parameters such as laser average power, pulse frequency, workpiece rotating speed, assist air pressure and Y feed rate were varied during experimentation. The rotatable central composite design experimental planning has been used to design the experimentation. The performance measures considered are surface roughness (Ra) and micro-turning depth deviation. Multi-objective optimization has been carried out for achieving the desired surface roughness as well as minimum depth deviation during laser micro-turning operation. Further, an artificial neural network (ANN) model has been developed to predict the process criteria. Levenberg-Marquadt training algorithm is used for multilayer feed forward backpropagation neural network. The developed ANN model has 5-10-2 feed forward network. There are 5 neurons in the input layer, 10 neurons in the hidden layer and 2 neurons in the output layers corresponding to two output responses, respectively. The developed ANN model has been validated using data obtained by conducting additional set of experiments. It was found that the developed ANN model can predict the process criteria more accurately than response surface methodology (RSM) based developed models.

Space-Division Multiple Access-OFDM based wireless communication has the potential to significantly increase the spectral efficiency, system performance and number of users. Research in the development of efficient multiuser signal detection algorithms for such systems has generated much interest in recent years. This research proposes MUD schemes utilizing three neural network (NN) models like Feed Forward NN (FNN) without any hidden layer, FNN with a single hidden layer and Recurrent Neural Network (RNN) as possible alternatives to existing Genetic Algorithm (GA) based Minimum Bit Error Rate (MBER) MUD. Further, these techniques offer low complexity. Extensive simulation based performance study is carried out to prove the efficiency of the proposed techniques. Better performance of FNN with hidden layer and RNN compared to FNN without hidden layer is clearly observed in rank deficient MIMO scenario, where number of users exceeds the number of receiving antennas. The Bit Error Rate (BER) and complexity analysis of the proposed neural MUD schemes are close to optimal ML and show improvement over the previous implemented technique.

Research community has recently put more attention to the Extreme Learning Machines (ELMs) algorithm in Neural Network (NN) area. The ELMs are much faster than the traditional gradient-descent-based learning algorithms due to its analytical determination of output weights with the random choice of input weights and hidden layer bias. However, since the input weights and bias are randomly assigned and not adjusted, the ELMs model shows an instability if we repeat the experiments many times. Such instability makes the ELMs less reliable than other computational intelligence models. In our investigation, we try to solve this problem by using the Random Production in the first layer of the ELMs. Thus, we can reduce the chance of using random weight assignment in ELMs by removing the bias in the hidden layer. Experiment son different data sets demonstrate that the proposed model has higher stability and reliability than the classical ELMs.

Artificial intelligent provides diverse solutions for the complex problems in agriculture research. The study aimed to use three models of artificial neural networks (Feed Forward Neural Network (FFNN), Generalized Regression Neural Network (GRNN) and Radial-Basis Neural Network (RBNN)) in the field of wheat yield prediction. 27-year data for the period (1986-2012) were utilized to improve the models and four-year data (2013 and 2016) were used to estimate the models, to compare their outputs with the measured data. Prediction data was not entered in the process of building neural network models. The results showed that the optimal configuration of the FFNN model consists of 40 neurons in the hidden layer (8-40-1). The Tan Sigmoid activation function was used in both the hidden layer and the output layer using all of these models (anterior neural feeding network and the regression neural network and radial base neural network) in the 4-year wheat yield forecast field for production (2013-2016) by applying 8 input parameters that were result of NMMS (8.6%, 7.6% and 15.7% resp.), To find that FFNN and GRNN provide the best result from BRNN because while the information set was large or in a wide range, then the range data ranges from -1 to +1 (normalization data) , GRNN gives better outcomes after the information or sample data were in large range.

The theory and implementation of extreme learning machine (ELM) prove that it is a simple, efficient, and accurate machine learning method. Compared with other single hidden layer feedforward neural network algorithms, ELM is characterized by simpler parameter selection rules, faster convergence speed, and less human intervention. The multiple hidden layer regularized extreme learning machine (MRELM) inherits these advantages of ELM and has higher prediction accuracy. In the MRELM model, the number of hidden layers is randomly initiated and fixed, and there is no iterative tuning process. However, the optimal number of hidden layers is the key factor to determine the generalization ability of MRELM. Given this situation, it is obviously unreasonable to determine this number by trial and random initialization. In this paper, an incremental MRELM training algorithm (FC-IMRELM) based on forced positive-definite Cholesky factorization is put forward to solve the network structure design problem of MRELM. First, an MRELM-based prediction model with one hidden layer is constructed, and then a new hidden layer is added to the prediction model in each training step until the generalization performance of the prediction model reaches its peak value. Thus, the optimal network structure of the prediction model is determined. In the training procedure, forced positive-definite Cholesky factorization is used to calculate the output weights of MRELM, which avoids the calculation of the inverse matrix and Moore-Penrose generalized inverse of matrix involved in the training process of hidden layer parameters. Therefore, FC-IMRELM prediction model can effectively reduce the computational cost brought by the process of increasing the number of hidden layers. Experiments on classification and regression problems indicate that the algorithm can be effectively used to determine the optimal network structure of MRELM, and the prediction model training by the algorithm has excellent performance in prediction accuracy and computational cost.

Artificial intelligence model for predicting geomechanical characteristics using easy-to-acquire offset logs without deploying logging tools

Extreme learning machine (ELM) was extended from the generalized single hidden layer feedforward networks where the input weights of the hidden layer nodes can be assigned randomly. It has been widely used for its much faster learning speed and less manual works. Considering the field of multi-label text classification, in this paper, we propose an ELM based algorithm combined with \(L_{21}\)-norm minimization of the output weights matrix called \(L_{21}\)-norm Minimization ELM, which not only fully inherits the merits of ELM but also facilitates group sparsity and reduces complexity of the learning model. Extensive experiments on several benchmark data sets show a more desirable performance compared with other common multi-label classification algorithms.

Feedforward neural networks (FNN) are most heavily used to identify the relation between a given set of input and desired output patterns. By the universal approximation theorem, it is clear that a single-hidden layer FNN is suffcient for the outputs to approximate the corresponding desired outputs arbitrarily close and so we consider a single-hidden layer FNN. In practice, we set up an error function so as to measure the performance of the FNN. As the error function is nonlinear, we define an iterative process, learning algorithm, to obtain the optimal choice of the connection weights and thus set up a numerical optimization problem. In this paper, we consider a new error function defined on the hidden layer We propose a new learning algorithm based on the least square methods converges rapidly. We discuss our method with the classic learning algorithms and the convergence for these algorithms.