SNP interaction detection with Random Forests in high-dimensional genetic data

BackgroundVariable importance measures for random forests have been receiving increased attention as a means of variable selection in many classification tasks in bioinformatics and related scientific fields, for instance to select a subset of genetic markers relevant for the prediction of a certain disease. We show that random forest variable importance measures are a sensible means for variable selection in many applications, but are not reliable in situations where potential predictor variables vary in their scale of measurement or their number of categories. This is particularly important in genomics and computational biology, where predictors often include variables of different types, for example when predictors include both sequence data and continuous variables such as folding energy, or when amino acid sequence data show different numbers of categories.ResultsSimulation studies are presented illustrating that, when random forest variable importance measures are used with data of varying types, the results are misleading because suboptimal predictor variables may be artificially preferred in variable selection. The two mechanisms underlying this deficiency are biased variable selection in the individual classification trees used to build the random forest on one hand, and effects induced by bootstrap sampling with replacement on the other hand.ConclusionWe propose to employ an alternative implementation of random forests, that provides unbiased variable selection in the individual classification trees. When this method is applied using subsampling without replacement, the resulting variable importance measures can be used reliably for variable selection even in situations where the potential predictor variables vary in their scale of measurement or their number of categories. The usage of both random forest algorithms and their variable importance measures in the R system for statistical computing is illustrated and documented thoroughly in an application re-analyzing data from a study on RNA editing. Therefore the suggested method can be applied straightforwardly by scientists in bioinformatics research.

There is growing momentum to develop statistical learning (SL) methods as an alternative to conventional genome-wide association studies (GWAS). Methods such as random forests (RF) and gradient boosting machine (GBM) result in variable importance measures that indicate how well each single-nucleotide polymorphism (SNP) predicts the phenotype. For RF, it has been shown that variable importance measures are systematically affected by minor allele frequency (MAF) and linkage disequilibrium (LD). To establish RF and GBM as viable alternatives for analyzing genome-wide data, it is necessary to address this potential bias and show that SL methods do not significantly under-perform conventional GWAS methods. Both LD and MAF have a significant impact on the variable importance measures commonly used in RF and GBM. Dividing SNPs into overlapping subsets with approximate linkage equilibrium and applying SL methods to each subset successfully reduces the impact of LD. A welcome side effect of this approach is a dramatic reduction in parallel computing time, increasing the feasibility of applying SL methods to large datasets. The created subsets also facilitate a potential correction for the effect of MAF using pseudocovariates. Simulations using simulated SNPs embedded in empirical data-assessing varying effect sizes, minor allele frequencies and LD patterns-suggest that the sensitivity to detect effects is often improved by subsetting and does not significantly under-perform the Armitage trend test, even under ideal conditions for the trend test. Code for the LD subsetting algorithm and pseudocovariate correction is available at http://www.nd.edu/~glubke/code.html.

Clinical prediction rules could help identify children at risk of slowed growth after an episode of diarrheal illness, and these rules may be generalizable to all children, regardless of diarrhea status.

The primary goal of genome-wide association studies (GWAS) is to discover genes or variants associated with complex diseases. Most GWA studies use single SNP (single nucleotide polymorphism) approaches that mainly focused on assessing the association between each individual SNP and disease; therefore they cannot take into account the combinations of SNPs. However, complex diseases are thought to involve complex etiologies including complicated interactions between many SNPs. Thus, different approaches are necessary to identify SNPs that influence disease risk jointly or in complex interactions. To discover SNP-SNP interactions, in this paper we propose first to use an improvement of Random Forest algorithm tailored for structured GWAS data, all rules are then extracted from the trees to analyse SNPs interactions. Our method allows one to select subgroups of informative SNPs which are most relevant to disease for building accurate decision trees and then we enable educe SNPs interactions from these trees. By this way, it reduces the dimensionality and can perform well with high-dimensional SNPs data sets. We conducted experiments on two genome-wide SNP data sets to demonstrate the effectiveness of the method for the SNP-SNP interactions.

Because of the poor performance of the Random Forest algorithm in processing the classification of high-dimensional unbalanced data, a Hybrid Samping&Feature Selection Random Forest optimization strategy (Hybrid Samping&Feature Selection Random Forest (HF_RF) is proposed in this paper. First, from the data level, the high-dimensional unbalanced data set is preprocessed by SMOTE algorithm combined with random undersampling to achieve balanced unbalanced data. At the same time, the clustering algorithm is combined with SMOTE algorithm to improve the processing ability of the algorithm for negative samples; On the algorithm level, through the Relief F algorithm, different weight values are given to the preprocessed high-dimensional data, irrelevant and redundant features are eliminated, and high-dimensional data is reduced for dimensionality; Finally, the weighted voting principle is used to further elevate the predictive performance of HF_RF. The experimental results show that compared with the traditional algorithm, the proposed algorithm has higher indicators when dealing with high-dimensional unbalanced data, which proves that the HF_RF proposed in this paper is The correctness of the algorithm and its effectiveness in improving the classification performance of high-dimensional unbalanced data.

Identifying genetic variants associated with complex disease in high-dimensional data is a challenging problem, and complicated etiologies such as gene-gene interactions are often ignored in analyses. The data-mining method Random Forests (RF) can handle high-dimensions; however, in high-dimensional data, RF is not an effective filter for identifying risk factors associated with the disease trait via complex genetic models such as gene-gene interactions without strong marginal components. Here we propose an extension called Weighted Random Forests (wRF), which incorporates tree-level weights to emphasize more accurate trees in prediction and calculation of variable importance. We demonstrate through simulation and application to data from a genetic study of addiction that wRF can outperform RF in high-dimensional data, although the improvements are modest and limited to situations with effect sizes that are larger than what is realistic in genetics of complex disease. Thus, the current implementation of wRF is unlikely to improve detection of relevant predictors in high-dimensional genetic data, but may be applicable in other situations where larger effect sizes are anticipated.

Improving land cover classification in an urbanized coastal area by random forests: The role of variable selection

In recent years, several association analysis methods for case-control studies have been developed. However, as we turn towards the identification of single nucleotide polymorphisms (SNPs) for prognosis, there is a need to develop methods for the identification of SNPs in high dimensional data with survival outcomes. Traditional methods for the identification of SNPs have some drawbacks. First, the majority of the approaches for case-control studies are based on single SNPs. Second, SNPs that are identified without incorporating biological knowledge are more difficult to interpret. Random forests has been found to perform well in gene expression analysis with survival outcomes. In this paper we present the first pathway-based method to correlate SNP with survival outcomes using a machine learning algorithm. We illustrate the application of pathway-based analysis of SNPs predictive of survival with a data set of 192 multiple myeloma patients genotyped for 500,000 SNPs. We also present simulation studies that show that the random forests technique with log-rank score split criterion outperforms several other machine learning algorithms. Thus, pathway-based survival analysis using machine learning tools represents a promising approach for the identification of biologically meaningful SNPs associated with disease.

When researchers use statistical models to gain insights into various phenomena, they make a tacit assumption that most (if not all) of the important predictor variables with respect to the outcome are included in the analysis. However, in practice this may not always be possible, whether because some important variables could not be measured, or because a researcher was not aware of all such important predictors. Prior research has shown that when important variables are omitted from both linear and nonlinear regression models, the model coefficients can be biased, with greater levels of bias being associated with larger correlations between the missing and retained variables. However, very little work has examined how such omissions impact the performance of variable importance measures used with popular machine learning algorithms. Therefore, the purpose of this simulation study was to address this gap in the literature and thereby provide insights into the impact of such omissions on variable importance measures for classification and regression trees, random forests, and boosting algorithms. Results showed that when an important variable is omitted from an analysis, other predictors that are correlated with and/or involved in an interaction with it will have inflated variables importance measures themselves. An empirical example and implications of these results are discussed. Journal of Statistical Research 2021, Vol. 55, No. 2, pp. 335-358

Shapley value regression with machine learning models has recently emerged as an axiomatic approach to the development of diagnostic models. However, when large numbers of predictor variables have to be considered, these methods become infeasible, owing to the inhibitive computational cost. In this paper, an approximate Shapley value approach with random forests is compared with a full Shapley model, as well as other methods used in variable importance analysis. Three case studies are considered, namely one based on simulated data, a model predicting throughput in a calcium carbide furnace as a function of operating variables, and a case study related to energy consumption in a steel plant. The approximately Shapley approach achieved results very similar to those achieved with the full Shapley approach but at a fraction of the computational cost. Moreover, although the variable importance measures considered in this study consistently identified the most influential predictors in the case studies, they yielded different results when fewer influential predictors were considered, and none of the variable importance measures performed better than the other measures across all three case studies.

This chapter illustrates use of a popular supervised machine-learning methods viz., random forest. It provides an application of Boruta algorithm for feature selection and then uses random forest algorithm for predictive modeling of housing data involving 79 explanatory features. The chapter aims to develop a predictive model for the sale price of the house based on appropriate features. Predictive modeling in the presence of a large number of exploratory variables requires the use of methods that support feature selection. The random forest algorithm is a popular machine-learning method that automatically calculates variable importance measure as a by-product and has been successfully used by various researchers. The Boruta package available in R software is used for feature selection. It uses a wrapper algorithm and can work with any classification methodology that yields variable importance measure as an output and by default uses random forest.

Machine learning methods and in particular random forests are promising approaches for prediction based on high dimensional omics data sets. They provide variable importance measures to rank predictors according to their predictive power. If building a prediction model is the main goal of a study, often a minimal set of variables with good prediction performance is selected. However, if the objective is the identification of involved variables to find active networks and pathways, approaches that aim to select all relevant variables should be preferred. We evaluated several variable selection procedures based on simulated data as well as publicly available experimental methylation and gene expression data. Our comparison included the Boruta algorithm, the Vita method, recurrent relative variable importance, a permutation approach and its parametric variant (Altmann) as well as recursive feature elimination (RFE). In our simulation studies, Boruta was the most powerful approach, followed closely by the Vita method. Both approaches demonstrated similar stability in variable selection, while Vita was the most robust approach under a pure null model without any predictor variables related to the outcome. In the analysis of the different experimental data sets, Vita demonstrated slightly better stability in variable selection and was less computationally intensive than Boruta.In conclusion, we recommend the Boruta and Vita approaches for the analysis of high-dimensional data sets. Vita is considerably faster than Boruta and thus more suitable for large data sets, but only Boruta can also be applied in low-dimensional settings.

In a regression setting, it is often of interest to quantify the importance of various features in predicting the response. Commonly, the variable importance measure used is determined by the regression technique employed. For this reason, practitioners often only resort to one of a few regression techniques for which a variable importance measure is naturally defined. Unfortunately, these regression techniques are often suboptimal for predicting the response. Additionally, because the variable importance measures native to different regression techniques generally have a different interpretation, comparisons across techniques can be difficult. In this work, we study a variable importance measure that can be used with any regression technique, and whose interpretation is agnostic to the technique used. This measure is a property of the true data-generating mechanism. Specifically, we discuss a generalization of the analysis of variance variable importance measure and discuss how it facilitates the use of machine learning techniques to flexibly estimate the variable importance of a single feature or group of features. The importance of each feature or group of features in the data can then be described individually, using this measure. We describe how to construct an efficient estimator of this measure as well as a valid confidence interval. Through simulations, we show that our proposal has good practical operating characteristics, and we illustrate its use with data from a study of risk factors for cardiovascular disease in SouthAfrica.

Identification of important factors in an inpatient fall risk prediction model to improve the quality of care using EHR and electronic administrative data: A machine-learning approach

One of the most commonly encountered problems in biomedical studies is analyzing censored survival data. Survival analysis differs from standard regression problems by one central feature: the event of interest may not be fully observed. Therefore, statistical methods used to analyze this data must be adapted to handle the missing information. In this chapter, we provide a brief introduction of right-censored survival data and introduce survival random forest models for analyzing them. Random forests are among the most popular machine learning algorithms. During the past decade, they have seen tremendous success in biomedical studies for prediction and decision-making. In addition to the statistical formulation, we also provide details of tuning parameters commonly considered in practice. An analysis example of breast cancer relapse free survival data is used as a demonstration. We further introduce the variable importance measure that serves as a variable selection tool in high-dimensional analysis. These examples are carried out using a newly developed R package RLT, which is available on GitHub.