Performance Evaluation of Back propagation Algorithm by change in learning rate and momentum value – A Machine learning Approach

Stability analysis of a three-term backpropagation algorithm

One type of artificial neural network is a backpropagation, This algorithm trained with the network architecture used during the training as well as providing the correct output to insert a similar but not the same with the architecture in use at training.The selection of appropriate parameters also affects the outcome, value of learning rate is one of the parameters which influence the process of training, Learning rate affects the speed of learning process on the network architecture.If the learning rate is set too large, then the algorithm will become unstable and otherwise the algorithm will converge in a very long period of time.So this study was made to determine the value of learning rate on the backpropagation algorithm. LVQ models of learning rate is one of the models used in the determination of the value of the learning rate of the algorithm LVQ.By modifying this LVQ model to be applied to the backpropagation algorithm. From the experimental results known to modify the learning rate LVQ models were applied to the backpropagation algorithm learning process becomes faster (epoch less).

Back Propagation (BP) is commonly used algorithm that optimize the performance of network for training multilayer feed-forward artificial neural networks. However, BP is inherently slow in learning and it sometimes gets trapped at local minima. These problems occur mailnly due to a constant and non-optimum learning rate (a fixed step size) in which the fixed value of learning rate is set to an initial starting value before training patterns for an input layer and an output layer. This fixed learning rate often leads the BP network towrds failure during steepest descent. Therefore to overcome the limitations of BP, this paper introduces an improvement to back propagation gradient descent with adapative learning rate (BPGD-AL) by changing the values of learning rate locally during the learning process. The simulation results on selected benchmark datasets show that the adaptive learning rate significantly improves the learning efficiency of the Back Propagation Algorithm

Within the domain of password security classification, the pursuit of practical and dependable methodologies has prompted the examination of both biological and technological paradigms. The present study investigates the efficacy of Mouth Brooding Fish (MBF) as an innovative method in contrast to conventional Machine Learning (ML) approaches for classifying password security. The research approach entails a rigorous examination of the comparative analysis of MBF and ML algorithms, evaluating their effectiveness in password classification using many criteria, including accuracy, robustness, flexibility, and durability against adversarial assaults. The findings suggest that ML approaches have shown significant effectiveness in classifying passwords. However, using methodologies inspired by the minimum Bayes risk framework demonstrates a higher degree of resistance against typical cyber dangers. The intrinsic biological mechanisms of MBF, encompassing adaptive behaviors and inherent protection, play a role in enhancing the resilience and adaptability of the password security categorization system. The results offer significant insights that can inform the evolution of password security systems, integrating biological principles with technical progress to enhance safeguarding measures in digital environments. To emphasize the advantages of the suggested approach, several ML approaches are investigated, such as Support Vector Machines (SVM), AdaBoost, Multilayer Perceptron (MLP), Gaussian Kernel (GK), and Random Forest (RF). The F-score, accuracy, sensitivity, and specificity metrics for MBF exhibit noteworthy performance compared to the other selected models, with values of 100%.

Slope stability prediction using integrated metaheuristic and machine learning approaches: A comparative study

Identification of damage in its early stage can have a great contribution in decreasing the maintenance costs and prolonging the life of valuable structures. Although conventional damage detection techniques have a mature background, their widespread application in industrial practice is still missing. In recent years the application of Machine Learning (ML) algorithms have been more and more exploited in structural health monitoring systems (SHM). Because of the superior capabilities of ML approaches in recognizing and classifying available patterns in a dataset, they have demonstrated a significant improvement in traditional damage identification algorithms. This review study focuses on the use of machine learning (ML) approaches in Ultrasonic Guided Wave (UGW)-based SHM, in which a structure is continually monitored using permanent sensors. Accordingly, multiple steps required for performing damage detection through UGWs are stated. Moreover, it is outlined that the employment of ML techniques for UGW-based damage detection can be subtended into two main phases: (1) extracting features from the data set, and reducing the dimension of the data space, (2) processing the patterns for revealing patterns, and classification of instances. With this regard, the most frequent techniques for the realization of those two phases are elaborated. This study shows the great potential of ML algorithms to assist and enhance UGW-based damage detection algorithms.

The objective of this study is to evaluate and compare the performance of different machine learning (ML) algorithms, viz., multi-layer perceptron (MLP), random forest (RF), extra trees regressor (ETR), CatBoost, and gradient boost (GB), considering the impact of variation in sample sizes for theprediction of soil nutrients. In this context, this study evaluates the impact of sample size on the prediction performance of five ML algorithms for mapping 14 soil properties, including key soil physico-chemical properties (soil organic carbon, pH, and electrical conductivity), and multiple macro (available nitrogen, phosphorus, potassium, calcium, magnesium, and sulfur) and micronutrients (available iron, manganese, copper, zinc, and boron) at a geographical extent of 8303 km2 located in eastern India. A total of 1024 surface soil samples were collected, of which 800 were used for model training, while the remaining 224 were reserved for independent validation (IV) of the resultant maps. The original training data set (800 samples) was reduced by random selection into six different sample sizes, i.e., 800, 400, 200, 100, 50, and 25. An exhaustive set of 574 environmental variables, derived from digital terrain derivatives and Sentinel-2 satellite imagery, was used as predictors. Popular statistical indicators, such as Lin's concordance correlation coefficient (CCC) and root mean squared error (RMSE) were employed to evaluate the predictive capability of the algorithms under different sample size scenarios. The results showed that prediction accuracy and reliability of prediction performance across multiple target variables improved with sample size. However, beyond a certain point, improvement in predictive performance became substantially negligible compared to the efforts in additional sampling. All the ML algorithms performed well (mean IV-CCC varied between 0.26 and 0.64 for different soil properties) to increase insample size, except MLP, which exhibited a poorer prediction performance(mean IV-CCC varied between 0.14 and 0.29 for different soil properties). Micronutrients in general responded well to the increase in sample sizes. The uncertainty analysis revealed that increasing sample size generally reduced prediction uncertainty, though the extent varied by soil property and ML algorithm. Concisely, the results presented herein showed an effective manner of selecting an appropriate sample size and a suitable ML algorithm to predict multiple soil nutrients accurately, which would be recommended to achieve optimal accuracy for a project.

Raman spectroscopy (RS) is a promising tool for cancer diagnosis. In particular, in the last years several studies have demonstrated how the diagnostic performances of RS can be significantly improved by employing machine learning (ML) algorithms for the interpretation of Raman-based data. Recently, it has been demonstrated that RS can perform an accurate classification of chondrosarcoma tissues. Chondrosarcoma is a cancer of bones, that can occur in the soft tissues near the bones. It is normally characterized by three different malignant degrees and a benign counterpart, knows as enchondroma. In line with these findings, in this paper, we exploited ML algorithms to distinguish, as well as possible, between the three grades of chondrosarcoma and to distinguish between chondrosarcoma and enchondroma. We obtained a high level of accuracy of classification by analyzing a dataset composed of a relatively small number of Raman spectra, collected in a previous study by one of the authors of this paper. Such spectra were acquired from micrometric tissue sections with a confocal Raman microscope. We tested the classification performances of a support vector machine (SVM) and a random forest classifier (RFC), as representatives of ML algorithms, and two versions of the multi-layer perceptron (MLPC) as representatives of deep learning (DL). These models, especially RFC and MLPC, showed excellent classification performances, with accuracy reaching 99.7%. This outcome makes the aforementioned models a promising route for future improvements of diagnostic devices focused on detecting cancerous bone tissues. Alongside the diagnostic purpose, the aforementioned approach allowed us to identify characteristic molecules, i.e., amino acids, nucleic acids, and bioapatites, relevant for obtaining the final diagnostic response, through the use of a tool named by us Raman Band Identification (RBI). The method to evaluate RBI is the most important contribution of this paper, because RBI could represent a relevant parameter for the identification of biochemical processes on the basis of the tumor progression and associated malignant degree. In turn, the spectral bands highlighted by RBI could provide precious indicators in an attempt to restrict the spectral acquisition to specific Raman bands. This last objective could help to reduce the amount of experimental data needed to obtain an accurate final grading outcome, with a consequent reduction in the computational cost.

Constructing the outer crust of the neutron stars requires the knowledge of the Binding Energy (BE) of the atomic nuclei. Although the BE of a lot of the nuclei is experimentally determined and can be obtained from the AME data table, for the others we need to depend on theoretical models. There exist a lot of physical theories to predict the BE, each with its own strengths and weaknesses. In this paper we apply Machine Learning (ML) algorithms on AME2016 data set to predict the Binding Energy of atomic nuclei. The novel feature of our work is that it is model independent. We do not assume or use any nuclear physics model but use only ML algorithms directly on the AME2016 data set. Our results are further refined by using another ML algorithm to train the errors of the first algorithm, and repeating this process iteratively. Our best algorithm gives σrms ∼ 0.58 MeV for Binding Energy on randomized testing sets. This is comparable to all physics models or ML improved physics models studied in literature till date. Using the predictions of our Machine Learning algorithm, we construct the outer crust equation of state (EoS) of a neutron star and show that our model is comparable to existing models. This work also demonstrates the use of various ML algorithms and a detailed analysis on how we arrived at our best algorithm. It will help the physics community in understanding how to choose an ML algorithm which would be suited for their data set. Our algorithms and best fit model is also made publicly available for the use of the community.

This research mainly based on multilayer perceptron (MLP) neural networks technique of data mining to forecast the wheat crop yield at the district level. There are many statistical and simulation models available, but the proposed algorithm with new activation function provides promising results in a shorter time with more accuracy. Sigmoid and hyperbolic tangent activation functions are widely used in the neural network. The activation functions play an important role in the neural network learning algorithm. The main objective of the proposed work is to develop an amended MLP neural network with new activation function and revised random weights and bias values for crop yield estimation by using the different weather parameter datasets. MLP model has been tested by existing activation functions and newly created activation functions with different cases including weights and bias values. In this research study, we evaluate the result of different activation functions and recommend some new simple activation functions, named DharaSig, DharaSigm and SHBSig, to improve the performance of neural networks and accurate results. Also, three new activation functions created with little variations in the DharaSig function named DharaSig1, DharaSig2 and DharaSig3. In this research study, variable numbers of hidden layers are tested with the variable number of neurons per hidden layer for the agriculture dataset. Variable values of momentum, seed and learning rate are also used in this study. Experiments show that newly created activation functions provide better results compared to ‘sigmoid’ default neural network activation function for agriculture datasets.

The back propagation (BP) algorithm is the most popular supervised learning method for multi-layered feed forward Neural Network. It has been successfully deployed in numerous practical problems and disciplines. Regardless of its popularity, BP is still known for some major drawbacks such as easily getting stuck in local minima and slow convergence; since, it uses Gradient Descent (GD) method to learn the network. Over the years, many improved modifications of the BP learning algorithm have been made by researchers but the local minima problem remains unresolved. Therefore, to resolve the inherent problems of BP algorithm, this paper proposed BPGD-A3T algorithm where the approach introduces three adaptive parameters which are gain, momentum and learning rate in BP. The performance of the proposed BPGD-A3T algorithm is then compared with BPGD two term parameters (BPGD-2T), BP with adaptive gain (BPGD-AG) and conventional BP algorithm (BPGD) by means of simulations on classification datasets. The simulation results show that the proposed BPGD-A3T shows better performance and performed highest accuracy for all dataset as compared to other.

Recent technological advances allowed an exponential expansion of biological sequence data, and the extraction of meaningful information through Machine Learning (ML) algorithms. This knowledge improved the understanding of the mechanisms related to several fatal diseases, e.g., Cancer and COVID-19, helping to develop innovative solutions, such as CRISPR-based gene editing, coronavirus vaccine, and precision medicine. These advances benefit our society and economy, directly impacting peoples lives in various areas, such as health care, drug discovery, forensic analysis, and food analysis. Nevertheless, ML approaches applied to biological data require representative, quantitative, and informative features. Necessarily, as many ML algorithms can handle only numerical data, sequences need to be translated into a feature vector. This process, known as feature extraction, is a fundamental step for the elaboration of high-quality ML-based models in bioinformatics, by allowing the feature engineering stage, with the design and selection of suitable features. Feature engineering, ML algorithm selection, and hyperparameter tuning are often manual and time-consuming processes, requiring extensive domain knowledge, and performed manually by a human expert. To deal with this problem, we developed a new package, BioAutoML, which automatically runs an end-to-end ML pipeline. BioAutoML extracts numerical and informative features from biological sequence databases, automating feature selection, recommendation of ML algorithm(s), and tuning of hyperparameters, using Automated ML (AutoML). BioAutoML has two components, divided into four modules, (1) automated feature engineering (feature extraction and selection modules) and (2) Metalearning (algorithm recommendation and hyperparameter tuning modules). Our experimental results, assessing the relevance of our proposal, indicate robust results for different problem domains, such as SARS-CoV-2, anticancer peptides, HIV sequences, and non-coding RNAs. According to our systematic review, our proposal is innovative compared to available studies in the literature, being the first study to propose automated feature engineering and metalearning for biological sequences. BioAutoML has a high potential to significantly reduce the expertise required to use ML pipelines, aiding researchers in combating diseases, particularly in low- and middle-income countries. This initiative can provide biologists, physicians, epidemiologists, and other stakeholders with an opportunity for widespread use of these techniques to enhance the health and well-being of their communities.

Predicting intraurban PM2.5 concentrations using enhanced machine learning approaches and incorporating human activity patterns

The signal strength of the maternal ECG (MECG) is usually many times that of the fetal ECG (FECG). Separating FECG from abdominal ECG (AECG) is therefore always a challenge. Some multiple-lead algorithms use the thoracic MECG to cancel the MECG in the AECG to get FECG, though this is inconvenient for the patient during long-term monitoring. Hence, this paper describes an adaptive method to separate fetal ECG from composite electrocardiography (ECG) that consists of both maternal and fetal ECGs by using adaptive linear neural network (ADALINE) for easy fetal monitoring. The input signal is the maternal signal and the target signal is the composite signal. The network emulate maternal signal as closely as possible to abdominal signal, thus only predict the maternal ECG in the abdominal ECG. The network error equals abdominal ECG minus maternal ECG, which is the fetal ECG. The characteristic that enables fetal extraction is due to the correlation between maternal ECG signals and the abdominal ECG signal of pregnant woman. A graphic user interface (GUI) program is written in Matlab to detect the changes in extracted fetal ECG by different values of momentum, learning rate and initial weights used in the network. However, the learning rate, momentum and initial weights are adjusted until the results are reasonably well. It is found that filtering performs best by high learning rate, low momentum and small initial weights. Key words: Neural network, fetal monitoring, fetal electrocardiography (ECG), QRS, maternal ECG, pregnancy.

BackgroundThe Centers for Medicare and Medicaid Services projects that health care costs will continue to grow over the next few years. Rising readmission costs contribute significantly to increasing health care costs. Multiple areas of health care, including readmissions, have benefited from the application of various machine learning algorithms in several ways.ObjectiveWe aimed to identify suitable models for predicting readmission charges billed by hospitals. Our literature review revealed that this application of machine learning is underexplored. We used various predictive methods, ranging from glass-box models (such as regularization techniques) to black-box models (such as deep learning–based models).MethodsWe defined readmissions as readmission with the same major diagnostic category (RSDC) and all-cause readmission category (RADC). For these readmission categories, 576,701 and 1,091,580 individuals, respectively, were identified from the Nationwide Readmission Database of the Healthcare Cost and Utilization Project by the Agency for Healthcare Research and Quality for 2013. Linear regression, lasso regression, elastic net, ridge regression, eXtreme gradient boosting (XGBoost), and a deep learning model based on multilayer perceptron (MLP) were the 6 machine learning algorithms we tested for RSDC and RADC through 10-fold cross-validation.ResultsOur preliminary analysis using a data-driven approach revealed that within RADC, the subsequent readmission charge billed per patient was higher than the previous charge for 541,090 individuals, and this number was 319,233 for RSDC. The top 3 major diagnostic categories (MDCs) for such instances were the same for RADC and RSDC. The average readmission charge billed was higher than the previous charge for 21 of the MDCs in the case of RSDC, whereas it was only for 13 of the MDCs in RADC. We recommend XGBoost and the deep learning model based on MLP for predicting readmission charges. The following performance metrics were obtained for XGBoost: (1) RADC (mean absolute percentage error [MAPE]=3.121%; root mean squared error [RMSE]=0.414; mean absolute error [MAE]=0.317; root relative squared error [RRSE]=0.410; relative absolute error [RAE]=0.399; normalized RMSE [NRMSE]=0.040; mean absolute deviation [MAD]=0.031) and (2) RSDC (MAPE=3.171%; RMSE=0.421; MAE=0.321; RRSE=0.407; RAE=0.393; NRMSE=0.041; MAD=0.031). The performance obtained for MLP-based deep neural networks are as follows: (1) RADC (MAPE=3.103%; RMSE=0.413; MAE=0.316; RRSE=0.410; RAE=0.397; NRMSE=0.040; MAD=0.031) and (2) RSDC (MAPE=3.202%; RMSE=0.427; MAE=0.326; RRSE=0.413; RAE=0.399; NRMSE=0.041; MAD=0.032). Repeated measures ANOVA revealed that the mean RMSE differed significantly across models with P<.001. Post hoc tests using the Bonferroni correction method indicated that the mean RMSE of the deep learning/XGBoost models was statistically significantly (P<.001) lower than that of all other models, namely linear regression/elastic net/lasso/ridge regression.ConclusionsModels built using XGBoost and MLP are suitable for predicting readmission charges billed by hospitals. The MDCs allow models to accurately predict hospital readmission charges.