PredRSA: a gradient boosted regression trees approach for predicting protein solvent accessibility.

A number of methods for predicting levels of solvent accessibility or accessible surface area (ASA) of amino acid residues in proteins have been developed. These methods either predict regularly spaced states of relative solvent accessibility or an analogue real value indicating relative solvent accessibility. While discrete states of exposure can be easily obtained by post prediction assignment of thresholds to the predicted or computed real values of ASA, the reverse, that is, obtaining a real value from quantized states of predicted ASA, is not straightforward as a two-state prediction in such cases would give a large real valued errors. However, prediction of ASA into larger number of ASA states and then finding a corresponding scheme for real value prediction may be helpful in integrating the two approaches of ASA prediction. We report a novel method of obtaining numerical real values of solvent accessibility, using accumulation cutoff set and support vector machine. This so-called SVM-Cabins method first predicts discrete states of ASA of amino acid residues from their evolutionary profile and then maps the predicted states onto a real valued linear space by simple algebraic methods. Resulting performance of such a rigorous approach using 13-state ASA prediction is at least comparable with the best methods of ASA prediction reported so far. The mean absolute error in this method reaches the best performance of 15.1% on the tested data set of 502 proteins with a coefficient of correlation equal to 0.66. Since, the method starts with the prediction of discrete states of ASA and leads to real value predictions, performance of prediction in binary states and real values are simultaneously optimized.

Protein structure prediction is often assisted by predicting one-dimensional structural properties including relative solvent accessibility (RSA) surface and backbone torsion angles (BTA) of residues, and these two properties are continuously varying variables because proteins can move freely in a three-dimensional space. Instead of subdividing them into a few arbitrarily defined states that many popular approaches used, this paper proposes an integrated system for realvalue prediction of protein structural motifs’ two local properties, based on the modified Hidden Markov Model that we previously presented. The model was used to capture the relevance of RSA and the dependency of BTA between adjacent residues along the local protein chain in motifs with definite probabilities. These two properties were predicted according to their own probability distribution. The method was applied to a protein fragment library. For nine different classes of motifs, real values of RSA were predicted with mean absolute error (MAE) of 0.122-0.175 and Pearson’s correlation coefficient (PCC) of 0.623-0.714 between predicted and actual RSA. Meanwhile, real values of BTA were obtained with MAE of 8.50-29.40 for Φ angles, 11.20-38.50 for ψ angles and PCC of 0.601-0.716 for Φ, 0.597-0.713 for ψ. The results were compared with well-known Real-SPINE Server, and indicate the proposed method may at least serve as the foundation to obtain better local properties from structural motifs for protein structure prediction. Keywords: Protein structure predication, structural motifs, hidden Markov model, solvent accessibility surface, backbone torsion angles, directional statistics distribution

On nucleotide solvent accessibility in RNA structure

Predicting protein structure is a fundamental problem in biology, especially in the genomic era where over one third of newly discovered genes have unknown structure and function. Because sequence and structure data (hence training sets) continue to grow exponentially, this area is ideally suited for machine learning approaches. Neural networks, in particular, have had remarkable success and have led, for instance, to the construction of the best secondary structure predictors. We will provide an overview of our own work and the state-of-the-art for several structure prediction problem including: (1) prediction of protein secondary structures; (2) prediction of relative solvent accessibility; (3) prediction of contacts; (4) prediction of three-dimensional protein structures; (5) prediction of interchain beta-sheet quaternary structures; using machine learning methods. The methods we have developed are based on the theory of graphical models but use deterministic recursive neural networks to speed up learning. We will discuss their applicability to other problems and the lessons learnt for the design of complex neural architectures.

BackgroundProtein structure mediates site-specific patterns of sequence divergence. In particular, residues in the core of a protein (solvent-inaccessible residues) tend to be more evolutionarily conserved than residues on the surface (solvent-accessible residues).ResultsHere, we present a model of sequence evolution that explicitly accounts for the relative solvent accessibility of each residue in a protein. Our model is a variant of the Goldman-Yang 1994 (GY94) model in which all model parameters can be functions of the relative solvent accessibility (RSA) of a residue. We apply this model to a data set comprised of nearly 600 yeast genes, and find that an evolutionary-rate ratio ω that varies linearly with RSA provides a better model fit than an RSA-independent ω or an ω that is estimated separately in individual RSA bins. We further show that the branch length t and the transition-transverion ratio κ also vary with RSA. The RSA-dependent GY94 model performs better than an RSA-dependent Muse-Gaut 1994 (MG94) model in which the synonymous and non-synonymous rates individually are linear functions of RSA. Finally, protein core size affects the slope of the linear relationship between ω and RSA, and gene expression level affects both the intercept and the slope.ConclusionsStructure-aware models of sequence evolution provide a significantly better fit than traditional models that neglect structure. The linear relationship between ω and RSA implies that genes are better characterized by their ω slope and intercept than by just their mean ω.

The accuracy of predicting protein local and global structural properties such as secondary structure and solvent accessible surface area has been stagnant for many years because of the challenge of accounting for non-local interactions between amino acid residues that are close in three-dimensional structural space but far from each other in their sequence positions. All existing machine-learning techniques relied on a sliding window of 10-20 amino acid residues to capture some 'short to intermediate' non-local interactions. Here, we employed Long Short-Term Memory (LSTM) Bidirectional Recurrent Neural Networks (BRNNs) which are capable of capturing long range interactions without using a window. We showed that the application of LSTM-BRNN to the prediction of protein structural properties makes the most significant improvement for residues with the most long-range contacts (|i-j| >19) over a previous window-based, deep-learning method SPIDER2. Capturing long-range interactions allows the accuracy of three-state secondary structure prediction to reach 84% and the correlation coefficient between predicted and actual solvent accessible surface areas to reach 0.80, plus a reduction of 5%, 10%, 5% and 10% in the mean absolute error for backbone ϕ , ψ , θ and τ angles, respectively, from SPIDER2. More significantly, 27% of 182724 40-residue models directly constructed from predicted C α atom-based θ and τ have similar structures to their corresponding native structures (6Å RMSD or less), which is 3% better than models built by ϕ and ψ angles. We expect the method to be useful for assisting protein structure and function prediction. The method is available as a SPIDER3 server and standalone package at http://sparks-lab.org . yaoqi.zhou@griffith.edu.au or yuedong.yang@griffith.edu.au. Supplementary data are available at Bioinformatics online.

This paper presents a hidden Markov model method (referred as HMM_AA_SA) for the identification of Helix-Turn-Helix (HTH) DNA-binding motifs. The method takes amino acid sequence and predicted solvent accessibility as input. Solvent accessibility of amino acids is predicted from amino acid sequence and discretized into three categories: buried (B), medium (M) and exposed (E). At each state, HMM_AA_SA emits not only one letter of amino acid but also one letter of solvent accessibility. The method is evaluated using 12 families of HTH motifs from the Pfam. Hidden Markov models are built and tested for each family individually based on three-fold cross validations. The results show that adding predicted solvent accessibility into the model increases the sensitivity by 5.7%, reaching 94.9%. We explore several reduced alphabets of amino acids in order to reduce the complexity of protein sequences and reduce the number of parameters in the model. The results show that using reduced alphabets can not only reduce the number of parameters in the system but also improve the performance. One interesting discovery is that HMM_AA_SA built from a HTH family can identify HTH motifs from other families, suggesting that the HMM_SA_AA method can capture features shared by different families of HTH motifs. This ability is improved when the hidden Markov models are built from the sequence fragments directly involved in the HTH motifs

We previously developed the first method to predict real valued accessible surface area in proteins, which was shown to contain more information than 100% correct predictions in a binary space of exposure states. Neural networks used for solvent accessibility predictions, however work as good predictors with invisible modeling of knowledge they learn. In view of this a direct statistical analysis of solvent accessibility of residues was desired, in terms of the contribution of residue neighbors in making them buried or exposed to solvent. Present work aims at compiling singlet to quintets of residue fragments and examining, how the solvent accessibility of residues is affected by the change in a neighboring residue in C-terminal, N-terminal or other positions of these residues. In addition, a graphical method of representing solvent accessibility was developed. Online graphical plots for all proteins in the entire Protein Data Bank (PDB) were made available on the web at www.netasa.org/ asa-view/.

Background/Objectives: Smartphone addiction (SA) and gaming addiction (GA) have become risk factors for individuals of all ages in recent years. Especially during adolescence, it has become very difficult for parents to control this situation. Physical activity and the effective use of free time are the most important factors in eliminating such addictions. This study aimed to test a new machine learning method by combining routine regression analysis with the gradient-boosting machine (GBM) and random forest (RF) methods to analyze the relationship between SA and GA with leisure time management (LTM) and the enjoyment of physical activity (EPA) among adolescents. Methods: This study presents the results obtained using our developed GBM + RF hybrid model, which incorporates LTM and EPA scores as inputs for predicting SA and GA, following the preprocessing of data collected from 1107 high school students aged 15-19 years. The results were compared with those obtained using routine regression results and the lasso, ElasticNet, RF, GBM, AdaBoost, bagging, support vector regression (SVR), K-nearest neighbors (KNN), multi-layer perceptron (MLP), and light gradient-boosting machine (LightGBM) models. In the GBM + RF model, probability scores obtained from GBM were used as input to RF to produce final predictions. The performance of the models was evaluated using the R2, mean absolute error (MAE), and mean squared error (MSE) metrics. Results: Classical regression analyses revealed a significant negative relationship between SA scores and both LTM and EPA scores. Specifically, as LTM and EPA scores increased, SA scores decreased significantly. In contrast, GA scores showed a significant negative relationship only with LTM scores, whereas EPA was not a significant determinant of GA. In contrast to the relatively low explanatory power of classical regression models, ML algorithms have demonstrated significantly higher prediction accuracy. The best performance for SA prediction was achieved using the Hybrid GBM + RF model (MAE = 0.095, MSE = 0.010, R2 = 0.9299), whereas the SVR model showed the weakest performance (MAE = 0.310, MSE = 0.096, R2 = 0.8615). Similarly, the Hybrid GBM + RF model also showed the highest performance for GA prediction (MAE = 0.090, MSE = 0.014, R2 = 0.9699). Conclusions: These findings demonstrate that classical regression analyses have limited explanatory power in capturing complex relationships between variables, whereas ML algorithms, particularly our GBM + RF hybrid model, offer more robust and accurate modeling capabilities for multifactorial cognitive and performance-related predictions.

Sequence-based prediction of one dimensional structural properties of proteins has been a long-standing subproblem of protein structure prediction. Recently, prediction accuracy has been significantly improved due to the rapid expansion of protein sequence and structure libraries and advances in deep learning techniques, such as residual convolutional networks (ResNets) and Long-Short-Term Memory Cells in Bidirectional Recurrent Neural Networks (LSTM-BRNNs). Here we leverage an ensemble of LSTM-BRNN and ResNet models, together with predicted residue-residue contact maps, to continue the push towards the attainable limit of prediction for 3- and 8-state secondary structure, backbone angles (θ, τ, ϕ and ψ), half-sphere exposure, contact numbers and solvent accessible surface area (ASA). The new method, named SPOT-1D, achieves similar, high performance on a large validation set and test set (≈1000 proteins in each set), suggesting robust performance for unseen data. For the large test set, it achieves 87% and 77% in 3- and 8-state secondary structure prediction and 0.82 and 0.86 in correlation coefficients between predicted and measured ASA and contact numbers, respectively. Comparison to current state-of-the-art techniques reveals substantial improvement in secondary structure and backbone angle prediction. In particular, 44% of 40-residue fragment structures constructed from predicted backbone Cα-based θ and τ angles are less than 6 Å root-mean-squared-distance from their native conformations, nearly 20% better than the next best. The method is expected to be useful for advancing protein structure and function prediction. SPOT-1D and its data is available at: http://sparks-lab.org/. Supplementary data are available at Bioinformatics online.

In the rapidly evolving landscape of retail analytics, the accurate prediction of sales figures holds paramount importance for informed decision-making and operational optimization. Leveraging diverse machine learning methodologies, this study aims to enhance the precision of Walmart sales forecasting, utilizing a comprehensive dataset sourced from Kaggle. Exploratory data analysis reveals intricate patterns and temporal dependencies within the data, prompting the adoption of advanced predictive modeling techniques. Through the implementation of linear regression, ensemble methods such as Random Forest, Gradient Boosting Machines (GBM), eXtreme Gradient Boosting (XGBoost), and Light Gradient Boosting Machine (LightGBM), this research endeavors to identify the most effective approach for predicting Walmart sales. Comparative analysis of model performance showcases the superiority of advanced machine learning algorithms over traditional linear models. The results indicate that XGBoost emerges as the optimal predictor for sales forecasting, boasting the lowest Mean Absolute Error (MAE) of 1226.471, Root Mean Squared Error (RMSE) of 1700.981, and an exceptionally high R-squared value of 0.9999900, indicating near-perfect predictive accuracy. This model's performance significantly surpasses that of simpler models such as linear regression, which yielded an MAE of 35632.510 and an RMSE of 80153.858. Insights from bias and fairness measurements underscore the effectiveness of advanced models in mitigating bias and delivering equitable predictions across temporal segments. Our analysis revealed varying levels of bias across different models. Linear Regression, Multiple Regression, and GLM exhibited moderate bias, suggesting some systematic errors in predictions. Decision Tree showed slightly higher bias, while Random Forest demonstrated a unique scenario of negative bias, implying systematic underestimation of predictions. However, models like GBM, XGBoost, and LGB displayed biases closer to zero, indicating more accurate predictions with minimal systematic errors. Notably, the XGBoost model demonstrated the lowest bias, with an MAE of -7.548432 (Table 4), reflecting its superior ability to minimize prediction errors across different conditions. Additionally, fairness analysis revealed that XGBoost maintained robust performance in both holiday and non-holiday periods, with an MAE of 84273.385 for holidays and 1757.721 for non-holidays. Insights from the fairness measurements revealed that Linear Regression, Multiple Regression, and GLM showed consistent predictive performance across both subgroups. Meanwhile, Decision Tree performed similarly for holiday predictions but exhibited better accuracy for non-holiday sales, whereas, Random Forest, XGBoost, GBM, and LGB models displayed lower MAE values for the non-holiday subgroup, indicating potential fairness issues in predicting holiday sales. The study also highlights the importance of model selection and the impact of advanced machine learning techniques on achieving high predictive accuracy and fairness. Ensemble methods like Random Forest and GBM also showed strong performance, with Random Forest achieving an MAE of 12238.782 and an RMSE of 19814.965, and GBM achieving an MAE of 10839.822 and an RMSE of 1700.981. This research emphasizes the significance of leveraging sophisticated analytics tools to navigate the complexities of retail operations and drive strategic decision-making. By utilizing advanced machine learning models, retailers can achieve more accurate sales forecasts, ultimately leading to better inventory management and enhanced operational efficiency. The study reaffirms the transformative potential of data-driven approaches in driving business growth and innovation in the retail sector.

Ensemble approach based on bagging, boosting and stacking for short-term prediction in agribusiness time series

Accurately estimating the amount of evaporation loss is necessary for scheduling and calculating irrigation water requirements. In this study, four machine learning (ML) modeling approaches, extreme learning machine (ELM), gradient boosting machine (GBM), quantile random forest (QRF), and Gaussian process regression (GPR), have been developed to estimate the monthly evaporation loss over two stations located in Iraq. Monthly climatical parameters have been used as an input variable for simulating the evaporation rate. Several statistical measures (e.g., mean absolute error (MAE), correlation coefficient (R), mean absolute percentage error (MAPE), and modified index of agreement (Md)), as well as graphical inspection, were used to compare the performances of the applied models. The results showed that the GBM model has much better performance in predicting monthly evaporation over two stations compared to other applied models. For the first case study which was in Diyala, the results showed a prediction enhancement in terms of MAE and RMSE by 7.17%, 21.01%; 16.51%, 15.74%; and 23.14%, 26.64%; using GBM compared to ELM, GPR, and QRF, respectively. However, for the second case study (in Erbil), the prediction enhancement was improved in terms of reduction of MAE and RMSE by 10.88%, 9.24%; 15.24%, 5%; and 16.06%, 15.76%; respectively, compared to ELM, GPR, and QRF models. The results of the proposed GMBM model can therefore assist local stakeholders in the management of water resources.

Numerous tests are used to determine the performance of concrete, but compressive strength (CS) is usually regarded as the most important. The recycled aggregate concrete (RAC) exhibits lower CS compared to natural aggregate concrete. Several variables, such as the water-cement ratio, the strength of the parent concrete, recycled aggregate replacement ratio, density, and water absorption of recycled aggregate, all impact the RAC’s CS. Many studies have been carried out to ascertain the influence of each of these elements separately. However, it is difficult to investigate their combined effect on the CS of RAC experimentally. Experimental investigations entail casting, curing, and testing samples, which require considerable work, expense, and time. It is vital to adopt novel methods to the stated aim in order to conduct research quickly and efficiently. The CS of RAC was predicted in this research utilizing machine learning techniques like decision tree, gradient boosting, and bagging regressor. The data set included eight input variables, and their effect on the CS of RAC was evaluated. Coefficient correlation (R2), the variance between predicted and experimental outcomes, statistical checks, and k-fold evaluations, were carried out to validate and compare the models. With an R2 of 0.92, the bagging regressor technique surpassed the decision tree and gradient boosting in predicting the strength of RAC. The statistical assessments also validated the superior accuracy of the bagging regressor model, yielding lower error values like mean absolute error (MAE) and root mean square error (RMSE). MAE and RMSE values for the bagging model were 4.258 and 5.693, respectively, which were lower than the other techniques employed, i.e., gradient boosting (MAE = 4.956 and RMSE = 7.046) and decision tree (MAE = 6.389 and RMSE = 8.952). Hence, the bagging regressor is the best suitable technique to predict the CS of RAC.

The technology known as cemented paste backfill (CPB) has gained considerable popularity worldwide. Yield stress (YS) is a significant factor considered in the assessment of CPB’s flowability or transportability. The minimal shear stress necessary to start the flow is known as Yield stress (YS), and it serves as an excellent measure of the strength of the particle-particle interaction. The traditional evaluation and measurement of YS performed by experimental tests are time-consuming and costly, which induces delays in construction projects. Moreover, the YS of CPB depends on numerous factors such as cement/tailing ratio, solid content and oxide content of tailing. Therefore, in order to simplify YS estimation and evaluation, the Artificial Intelligence (AI) approaches including eight Machine Learning techniques such as the Extreme Gradient Boosting algorithm, Gradient Boosting algorithm, Random Forest algorithm, Decision Trees, K-Nearest Neighbor, Support Vector Machine, Multivariate Adaptive Regression Splines and Gaussian Process are used to build the soft-computing model in predicting the YS of CPB. The performance of these models is evaluated by three metrics coefficient of determination (R2), Root Mean Square Error (RMSE) and Mean Absolute Error (MAE). The 3 best models were found to predict the Yield Stress of CPB (Gradient Boosting (GB), Extreme Gradient Boosting (XGB) and Random Forest (RF), respectively) with the 3 metrics of the three models, respectively, GB {R2 = 0.9811, RMSE = 0.1327 MPa, MAE = 0.0896 MPa}, XGB {R2 = 0.9034, RMSE = 0.3004 MPa, MAE = 0.1696 MPa} and RF {R2 = 0.8534, RMSE = 0.3700 MPa, MAE = 0.1786 MPa}, for the testing dataset. Based on the best performance model including GB, XG and RF, the other AI techniques such as SHapley Additive exPlanations (SHAP), Permutation Importance, and Individual Conditional Expectation (ICE) are also used for evaluating the factor effect on the YS of CPB. The results of this investigation can help the engineers to accelerate the mixed design of CPB.