On the structural dimension of sliced inverse regression

With the advent of high throughput genome-wide assays it has become possible to simultaneously measure multiple types of genomic data. Several projects like TCGA, ICGC, NCI-60 has generated comprehensive, multi-dimensional maps of the key genomic changes like MiRNA, MRNA, proteomics etc. from cancer samples[2,4]. These genomic data can be used for classifying tumour types[5]. Integrative analysis of these data from multiple sources can potentially provide additional biological insights, but methods to do any such analysis are lacking. One of the widely used solutions to handle high dimension data is by removing redundant information in the integrated sample. Most of the expressed genes are overlapped and can be projected onto lower dimension, and then be used to classify different tumor types, without the loss of any/much information. Sufficient dimension reduction (SDR) [1], a supervised dimension reduction approach, can be ideal to achieve such a goal. In this paper, we propose a novel integrative SDR method that can reduce dimensions of multiple data types simultaneously while sharing common latent structures to improve prediction and interpretation. In particular, we extend the sliced inverse regression (SIR) technique, a major SDR method, to integrate multiple omits data for simultaneous dimension reduction. SIR is a supervised dimension reduction method that assumes that the outcome variable Y depends on the predictor variable X through d unknown linear combinations of the predictor[3]. The predictor variable is replaced by its projection into a lower dimension subspace of the predictor space without the loss of information. The aim is to find the intersection of all the subspaces δ called the central susbspace (CS) of the predictor space satisfying the property Y ╨ X| Pδ X. To integrate multiple types of data, we propose and implement a new integrative sufficient dimension reduction method extending SIR[3], called integrative SIR. The main idea is that we take into account all the multi-omics data information simultaneously while finding a basis matrix for each data type with some sharing latent structures. Finally, we get d dimension data which is much smaller than the original data dimension. The reduced dimension d was achieved by cross validation. To demonstrate the integrated analysis of multi-omics data, we applied and compared conventional SIR and integrative SIR to analyze MRNA, MiRNA and proteomics expression profile of a subset of cell lines from the NCI-60 panel. The data used is taken from [6]. The outcomes we have to classify are CNS, Leukemia and Melanoma tumor types. We pre-screened 400 variables from each data type with the criteria of high variance. To find classification error, we performed random forest classification after we applied to each method with leave-one-out cross-validation. As a result, we found out that integrative SIR leads to less classification error as compared to conventional SIR. To summarize, we proposed a new integrative SIR method, a supervised dimension reduction technique for integrative analysis of multi-omics data types. Unlike conventional SDR methods, the new approach can reduce the dimensions of multiple omics data simultaneously while sharing common latent structures across data types without losing any information in prediction. By efficiently capturing the common information, our numerical study shows that integrative SIR classifies tumor types more accurately as compared to conventional SDR methods.

Sliced average third‐moment estimation (SATME) is a typical method for sufficient dimension reduction (SDR) based on high‐order conditional moment. It is useful, particularly in the scenarios of regression mixtures. However, as SATME uses the third‐order conditional moment of the predictors given the response, it may not as robust as some other SDR methods that use lower order moments, say, sliced inverse regression (SIR) and slice average variance estimation (SAVE). Based on the space displacement function, a local influence analysis framework of SATME is constructed including a statistic of influence assessment for the observations. Furthermore, a data‐trimming strategy is suggested based on the above influence assessment. The proposed methodologies solve a typical issue that also exists in some other SDR methods. A real‐data analysis and simulations are presented.

Sufficient dimension reduction (SDR) continues to be an active field of research. When estimating the central subspace (CS), inverse regression based SDR methods involve solving a generalized eigenvalue problem, which can be problematic under the large-p-small-n situation. In recent years, new techniques have emerged in numerical linear algebra, called randomized algorithms or random sketching, for high-dimensional and large scale problems. To overcome the large-p-small-n SDR problem, we combine the idea of statistical inference with random sketching to propose a new SDR method, called integrated random-partition SDR (iRP-SDR). Our method consists of the following three steps: (i)Randomly partition the covariates into subsets to construct an envelope subspace with low dimension. (ii)Obtain a sketch of the CS by applying a conventional SDR method within the constructed envelope subspace. (iii)Repeat the above two steps many times and integrate these multiple sketches to form the final estimate of the CS. After describing the details of these steps, the asymptotic properties of iRP-SDR are established. Unlike existing methods, iRP-SDR does not involve the determination of the structural dimension until the last stage, which makes it more adaptive to a high-dimensional setting. The advantageous performance of iRP-SDR is demonstrated via simulation studies and a practical example analyzing EEG data.

Robust sufficient dimension reduction via ball covariance

Sufficient dimension reduction (SDR) is a popular framework for supervised dimension reduction, aiming at reducing the dimensionality of input data while information on output data is maximally maintained. On the other hand, in many recent supervised classification learning tasks, it is conceivable that the balance of samples in each class varies between the training and testing phases. Such a phenomenon, referred to as class-prior change, causes existing SDR methods to perform undesirably particularly when the training data is highly imbalanced. In this paper, we extend the state-of-the-art SDR method called leastsquares gradients for dimension reduction (LSGDR) to be able to cope with such class-prior change under the semi-supervised learning setup where unlabeled test data are available in addition to labeled training data. Through experiments, we demonstrate the usefulness of our proposed method.

On permutation tests for predictor contribution in sufficient dimension reduction

On testing common indices for two multi-index models: A link-free approach

On model-free conditional coordinate tests for regressions

To reduce the predictors dimension without loss of information on the regression, we develop in this paper a sufficient dimension reduction method which we term cumulative Hessian directions. Unlike many other existing sufficient dimension reduction methods, the estimation of our proposal avoids completely selecting the tuning parameters such as the number of slices in slicing estimation or the bandwidth in kernel smoothing. We also investigate the asymptotic properties of our proposal when the predictors dimension diverges. Illustrations through simulations and an application are presented to evidence the efficacy of our proposal and to compare it with existing methods.

Summary The partial least squares (PLS) approach first constructs new explanatory variables, known as factors (or components), which are linear combinations of available predictor variables. A small subset of these factors is then chosen and retained for prediction. We study the performance of PLS in estimating single-index models, especially when the predictor variables exhibit high collinearity. We show that PLS estimates are consistent up to a constant of proportionality. We present three simulation studies that compare the performance of PLS in estimating single-index models with that of sliced inverse regression (SIR). In the first two studies, we find that PLS performs better than SIR when collinearity exists. In the third study, we learn that PLS performs well even when there are multiple dependent variables, the link function is non-linear and the shape of the functional form is not known.

Distributed estimation in heterogeneous reduced rank regression: With application to order determination in sufficient dimension reduction

Sufficient dimension reduction (SDR) techniques have become essential tools in high‐dimensional data analysis. However, many SDR methods rely on iterative procedures, which impose substantial computational costs when applied to large‐scale datasets. To address these challenges, we develop optimal subsampling strategies tailored to two representative SDR approaches: The refined outer product of gradients based on the conditional density functions (rdOPG) and the minimum average (conditional) variance estimation based on the conditional density functions (dMAVE). Our methods are designed to minimize the trace of the asymptotic variance–covariance matrix under an inverse probability weighting framework, thereby achieving an effective trade‐off between computational efficiency and statistical accuracy. We derive explicit forms for the optimal subsampling probabilities and establish the consistency and asymptotic normality of the resulting estimators. Extensive simulation studies and a real‐data application demonstrate that our methods substantially reduce computational burden while achieving improved estimation accuracy compared to uniform subsampling approaches, highlighting their practical value for large‐scale SDR problems.

In high-dimensional data analysis, sliced inverse regression (SIR) has proven to be an effective dimension reduction tool and has enjoyed wide applications. The usual SIR, however, cannot work with problems where the number of predictors, p, exceeds the sample size, n, and can suffer when there is high collinearity among the predictors. In addition, the reduced dimensional space consists of linear combinations of all the original predictors and no variable selection is achieved. In this article, we propose a regularized SIR approach based on the least-squares formulation of SIR. The L2 regularization is introduced, and an alternating least-squares algorithm is developed, to enable SIR to work with n < p and highly correlated predictors. The L1 regularization is further introduced to achieve simultaneous reduction estimation and predictor selection. Both simulations and the analysis of a microarray expression data set demonstrate the usefulness of the proposed method.

A new sufficient dimension reduction method via rank divergence

Because sliced inverse regression (SIR) using the conditional mean of the inverse regression fails to recover the central subspace when the inverse regression mean degenerates, sliced average variance estimation (SAVE) using the conditional variance was proposed in the sufficient dimension reduction literature. However, the efficacy of SAVE depends heavily upon the number of slices. In the present article, we introduce a class of weighted variance estimation (WVE), which, similar to SAVE and simple contour regression (SCR), uses the conditional variance of the inverse regression to recover the central subspace. The strong consistency and the asymptotic normality of the kernel estimation of WVE are established under mild regularity conditions. Finite sample studies are carried out for comparison with existing methods and an application to a real data is presented for illustration.