A general insight into the effect of neuron structure on classification

The optimization performance of transiently chaotic neural network (TCNN) is affected by various factors such as chaotic characteristic, model parameters, and annealing function, and its capacity of global optimization is limited. It is demonstrated that the non-monotonic activation function can generate richer chaotic characteristic than the monotonic activation function in the TCNN model. Besides, the activation function involving neurobiological mechanism can not only reflect the rich brain activity in brain waves, but also enhance the non-linear dynamic characteristic, which may further improve the global optimization ability. Hence, a novel chaotic neuron model is proposed with the non-monotonic activation function based on the neurobiological mechanisms from the electroencephalogram. The electroencephalogram consists of five brain waves (i.e., , , , , and waves) which are defined by the quality and intensity of brain waves with different frequency bands ranging from 0.5 Hz to 100 Hz. The brain wave with a higher frequency and a lower amplitude represents a more active brain. Researches demonstrate that the five brain waves can be simplified into sinusoidal waves with different frequencies. Hence, a frequency conversion sinusoidal (FCS) function which has the consistent frequency range and features with brain waves is designed based on the above neurobiological mechanisms. Then a novel chaotic neuron model with non-monotonic activation function which is composed of the FCS function and sigmoid function, is proposed for richer chaotic dynamic characteristic. The reversed bifurcation and the Lyapunov exponent of the chaotic neuron are given and the dynamic system is analyzed, indicating that the proposed FCS neuron model owns richer chaotic dynamic characteristic than transiently chaotic neuron model due to its special non-monotonic activation function. Based on the neuron model, a novel transiently-chaotic neural networkfrequency conversion sinusoidal chaotic neural network (FCSCNN) is constructed and the basis of model parameter selection is provided as well. To validate the effectiveness of the proposed model, the FCSCNN is applied to nonlinear function optimization and 10-city, 30-city, 75-city traveling salesman problem. The experimental results show that 1) the FCSCNN has a good performance under the condition of moderate a, smaller cA(0) and 2(0); 2) on the basis of the appropriate model parameters, the FCSCNN has better global optimization ability and optimization accuracy than Hopfield neural network, TCNN, improved-TCNN due to its richer chaotic characteristic in complicated combinational optimization problem, especially in middle and large scale problem.

The authors investigate the convergence and pruning performance of multilayer feedforward neural networks with different types of neuronal activation functions in solving various problems. Three types of activation functions are adopted in the network, namely, the traditional sigmoid function, the sinusoidal function and a periodic function that can be considered as a combination of the first two functions. To speed up the learning, as well as to reduce the network size, the extended Kalman filter (EKF) algorithm conjunct with a pruning method is used to train the network. The corresponding networks are applied to solve five typical problems, namely, 4-point XOR logic function, parity generation, handwritten digit recognition, piecewise linear function approximation and sunspot series prediction. Simulation results show that periodic activation functions perform better than monotonic ones in solving multicluster classification problems. Moreover, the combined periodic activation function is found to possess the fast convergence and multicluster classification capabilities of the sinusoidal activation function while keeping the robustness property of the sigmoid function required in the modelling of unknown systems.

This research mainly based on multilayer perceptron (MLP) neural networks technique of data mining to forecast the wheat crop yield at the district level. There are many statistical and simulation models available, but the proposed algorithm with new activation function provides promising results in a shorter time with more accuracy. Sigmoid and hyperbolic tangent activation functions are widely used in the neural network. The activation functions play an important role in the neural network learning algorithm. The main objective of the proposed work is to develop an amended MLP neural network with new activation function and revised random weights and bias values for crop yield estimation by using the different weather parameter datasets. MLP model has been tested by existing activation functions and newly created activation functions with different cases including weights and bias values. In this research study, we evaluate the result of different activation functions and recommend some new simple activation functions, named DharaSig, DharaSigm and SHBSig, to improve the performance of neural networks and accurate results. Also, three new activation functions created with little variations in the DharaSig function named DharaSig1, DharaSig2 and DharaSig3. In this research study, variable numbers of hidden layers are tested with the variable number of neurons per hidden layer for the agriculture dataset. Variable values of momentum, seed and learning rate are also used in this study. Experiments show that newly created activation functions provide better results compared to ‘sigmoid’ default neural network activation function for agriculture datasets.

The objective of this study is to show how a multi-layer perceptron (MLP) neural network can be used to model a CMM measuring process. To date, most MLP-based process models have been established for process mean only. An innovative approach is proposed to model simultaneously the mean and the variation of a CMM process using one integrated MLP architecture. Therefore, the MLP-based model obtained captures not only the process mean but also the process variation information. Selected issues related to neural network training are also discussed. Specifically, the guideline that was proposed by Mirchandani and Cao (1989) for selecting a number of hidden neurons is tested to determine the effects of the number of hidden neurons. The performances of two different learning algorithms - back-propagation with momentum factor (BPM) and the Fletcher-Reeves (FR) algorithm - are studied in terms of CPU time, training error, and generalization error.

In the multilayer perceptron (MLP), there was a theorem about the maximum number of separable regions (M) given the number of hidden nodes (H) in the input d-dimensional space. We propose a recurrence relation in the high dimensional space and prove the theorem using the expansion of recurrence relation instead of proof by induction. The MLP model has input layer, one hidden layer, and output layer. We use different MLP models on the well log data inversion to test the number of hidden nodes determined by the theorem. The inputs are the first order, second order, and third order features. Higher order neural network (HONN) has the property of more nonlinear mapping. In the experiments, we have 31 simulated well log data. 25 well log data are used for training, and 6 are for testing. The experimental results can support the number of hidden nodes determined by the theorem.

Multi Layer Perceptron (MLP), one of the deep learning algorithms, has been widely used in classification problem research because it has advantages over other conventional classification methods. This study takes 15 articles that have been published in research journals regarding the application of the multi-layer perceptron algorithm to prediction and classification problems. From the results of the analysis carried out, the results show that the lowest performance value of this algorithm is 62.89%, the highest performance value of this algorithm is 100% and the average performance value of this algorithm is 91.98%. From these values, it can be concluded that the multi layer perceptron algorithm is very good and feasible to be used in solving prediction and classification problems

Monkeypox virus is gaining attention due to its severity and spread among people. This study sheds light on the modeling and forecasting of new monkeypox cases. Knowledge about the future situation of the virus using a more accurate time series and stochastic models is required for future actions and plans to cope with the challenge. We conduct a side-by-side comparison of the machine learning approach with the traditional time series model. The multilayer perceptron model (MLP), a machine learning technique, and the Box-Jenkins methodology, also known as the ARIMA model, are used for classical modeling. Both methods are applied to the Monkeypox cumulative data set and compared using different model selection criteria such as root mean square error, mean square error, mean absolute error, and mean absolute percentage error. With a root mean square error of 150.78, the monkeypox series follows the ARIMA (7,1,7) model among the other potential models. Comparatively, we use the multilayer perceptron (MLP) model, which employs the sigmoid activation function and has a different number of hidden neurons in a single hidden layer. The root mean square error of the MLP model, which uses a single input and ten hidden neurons, is 54.40, significantly lower than that of the ARIMA model. The actual confirmed cases versus estimated or fitted plots also demonstrate that the multilayer perceptron model has a better fit for the monkeypox data than the ARIMA model. When it comes to predicting monkeypox, the machine learning method outperforms the traditional time series. A better match can be achieved in future studies by applying the extreme learning machine model (ELM), support vector machine (SVM), and some other methods with various activation functions. It is thus concluded that the selected data provide a real picture of the virus. If the situations remain the same, governments and other stockholders should ensure the follow-up of Standard Operating Procedures (SOPs) among the masses, as the trends will continue rising in the upcoming 10 days. However, governments should take some serious interventions to cope with the virus. In the ARIMA models selected for forecasting, we did not incorporate the effect of covariates such as the effect of net migration of monkeypox virus patients, government interventions, etc.

Fly ash and phosphogypsum are industrial by-products requiring a cost to get rid of. Their potential use in the synthesis of geopolymer bricks provides great benefits such as the saving use of natural resources and the solid by-product waste management. Compressive strength is the most important parameter for geopolymer bricks design. In this study, two artificial neural networks, namely the multilayer perceptron (MLP) and the radial basis function (RBF) networks, have been investigated to predict the compressive strength. While developing the MLP or RBF models, 99 experimental observations were used for training and testing. Two evaluation steps were performed: The first step determined the effective number of hidden layers and neurons in each hidden layer as well as the appropriate activation function in predicting the compressive strength. The second evaluation step evaluated the accuracy with which the model would predict the compressive strength of geopolymers. The MLP neural network with two hidden layers having 8 and 10 neurons and the hyperbolic tangent activation function was the best model for predicting the compressive strength. Artificial neural networks can be used as a reliable and accurate technique for estimating the parameters of geopolymer materials.

In this paper, nonlinear functions generated by randomly initialized multilayer perceptrons (MLPs) and simultaneous recurrent neural networks (SRNs) are learned by MLPs and SRNs. Training SRNs is a challenging task and a new learning algorithm - DEPSO is introduced. DEPSO is a standard particle swarm optimization (PSO) algorithm with the addition of a differential evolution step to aid in swarm convergence. The results from DEPSO are compared with the standard backpropagation (BP) and PSO algorithms. It is further verified that functions generated by SRNs are harder to learn than those generated by MLPs but DEPSO provides better learning capabilities for the functions generated by MLPs and SRNs as compared to BP and PSO. These three algorithms are also trained on several benchmark functions to confirm results.

We report a study on learning ability of a deterministic Boltzmann machine (DBM) with neurons which have a nonmonotonic activation function. We use an end-cut-off-type function with a threshold parameter '/spl theta/' as the nonmonotonic function. Numerical simulations of nonlinear problems, such as the 2-parity problem and the 4-parity problem, show that the DBM network with nonmonotonic neurons has higher learning ability compared to the network with monotonic neurons.

Artificial neural networks (ANN) and evolutionary algorithms are two relatively young research areas that were subject to a steadily growing interest during the past years. This paper examines the use of different evolutionary algorithms, imperialist competitive algorithm (ICA), genetic algorithm (GA), ICA-GA and recursive ICA-GA (R-ICA-GA) to train a classification problem on a multi layer perceptron (MLP) neural network. All of named evolutionary training algorithms are compared together in this paper. The first goal of the paper is to apply new evolutionary optimization algorithms ICA-GA and R-ICA-GA for training the ANN and the second goal of the paper is to compare different evolutionary algorithms. It is shown that the ICA-GA has the best performance, in number of epochs, compared to the other algorithms. For this purpose, learning algorithms are applied on six known datasets (WINE, PIMA, WDBC, IRIS, SONAR and GLASS) which are used for classification problems.

Neural networks-based formulation for predicting ultimate strength of bolted shear connectors in composite cold-formed steel beams

Automatic detection of clustered microcalcifications in digital mammograms using mathematical morphology and neural networks

The application of the Generative Adversarial Networks (GANs) model has increased rapidly in the past few years. This paper will demonstrate the training results of different GANs models in different epochs. Since GANs is a structure, they can be applied to different neural network layers such as Convolutional Neural Network (CNN) and multilayer perceptron (MLP). Different neural network layers are constructed and trained separately to compare their performance and efficiency in this experiment. To achieve this goal, all models perform the same task, generating handwritten numbers. In the experiment, three GANs models are tested. Two of them are based on MLP. They use the same structure and the same number of units. The only difference is the activation function. One uses the sigmoid function, and the other uses the LeakyRelu function. The last model is based on CNN. Specifically, the GANs model using Convolutional layers is called DCGAN. All models are constructed using TensorFlow, each of them was trained for 10, 20, 40, 80, 120, and 160 epochs. The generated images and loss value are recorded in all cases of the previously mentioned epochs of training. The experimental results demonstrated that Compared with the GANs model based on MLP, DCGAN has higher training efficiency and better image quality. Different activation functions can also affect the efficiency of the GANs model.

Optimization of the weights and asymmetric activation function family of neural network for time series forecasting