Local influence analysis for the sliced average third‐moment estimation

With the advent of high throughput genome-wide assays it has become possible to simultaneously measure multiple types of genomic data. Several projects like TCGA, ICGC, NCI-60 has generated comprehensive, multi-dimensional maps of the key genomic changes like MiRNA, MRNA, proteomics etc. from cancer samples[2,4]. These genomic data can be used for classifying tumour types[5]. Integrative analysis of these data from multiple sources can potentially provide additional biological insights, but methods to do any such analysis are lacking. One of the widely used solutions to handle high dimension data is by removing redundant information in the integrated sample. Most of the expressed genes are overlapped and can be projected onto lower dimension, and then be used to classify different tumor types, without the loss of any/much information. Sufficient dimension reduction (SDR) [1], a supervised dimension reduction approach, can be ideal to achieve such a goal. In this paper, we propose a novel integrative SDR method that can reduce dimensions of multiple data types simultaneously while sharing common latent structures to improve prediction and interpretation. In particular, we extend the sliced inverse regression (SIR) technique, a major SDR method, to integrate multiple omits data for simultaneous dimension reduction. SIR is a supervised dimension reduction method that assumes that the outcome variable Y depends on the predictor variable X through d unknown linear combinations of the predictor[3]. The predictor variable is replaced by its projection into a lower dimension subspace of the predictor space without the loss of information. The aim is to find the intersection of all the subspaces δ called the central susbspace (CS) of the predictor space satisfying the property Y ╨ X| Pδ X. To integrate multiple types of data, we propose and implement a new integrative sufficient dimension reduction method extending SIR[3], called integrative SIR. The main idea is that we take into account all the multi-omics data information simultaneously while finding a basis matrix for each data type with some sharing latent structures. Finally, we get d dimension data which is much smaller than the original data dimension. The reduced dimension d was achieved by cross validation. To demonstrate the integrated analysis of multi-omics data, we applied and compared conventional SIR and integrative SIR to analyze MRNA, MiRNA and proteomics expression profile of a subset of cell lines from the NCI-60 panel. The data used is taken from [6]. The outcomes we have to classify are CNS, Leukemia and Melanoma tumor types. We pre-screened 400 variables from each data type with the criteria of high variance. To find classification error, we performed random forest classification after we applied to each method with leave-one-out cross-validation. As a result, we found out that integrative SIR leads to less classification error as compared to conventional SIR. To summarize, we proposed a new integrative SIR method, a supervised dimension reduction technique for integrative analysis of multi-omics data types. Unlike conventional SDR methods, the new approach can reduce the dimensions of multiple omics data simultaneously while sharing common latent structures across data types without losing any information in prediction. By efficiently capturing the common information, our numerical study shows that integrative SIR classifies tumor types more accurately as compared to conventional SDR methods.

In this work, we address the longstanding puzzle that Sliced Inverse Regression (SIR) often performs poorly for sufficient dimension reduction when the structural dimension d (the dimension of the central space) exceeds 4. We first show that in the multiple index model Y=f(PX)+ϵ where X is a p-standard normal vector, ϵ is an independent noise, and P is a projection operator from Rp to Rd, if the link function f follows the law of a Gaussian process. Then with high probability, the dth eigenvalue λd of Cov[E(X|Y)] satisfies λd≤Ce−θd for some positive constants C and θ. We then focus on the low signal regime where λd can be arbitrarily small and not larger than d−8.1, and prove that the minimax risk of estimating the central space is lower bounded by dp nλd. Combining these two results, we provide a convincing explanation for the poor performance of SIR when d is large, a phenomenon that has perplexed researchers for nearly three decades. The technical tools developed here may be of independent interest for studying other sufficient dimension reduction methods.

On model-free conditional coordinate tests for regressions

Sufficient dimension reduction (SDR) continues to be an active field of research. When estimating the central subspace (CS), inverse regression based SDR methods involve solving a generalized eigenvalue problem, which can be problematic under the large-p-small-n situation. In recent years, new techniques have emerged in numerical linear algebra, called randomized algorithms or random sketching, for high-dimensional and large scale problems. To overcome the large-p-small-n SDR problem, we combine the idea of statistical inference with random sketching to propose a new SDR method, called integrated random-partition SDR (iRP-SDR). Our method consists of the following three steps: (i)Randomly partition the covariates into subsets to construct an envelope subspace with low dimension. (ii)Obtain a sketch of the CS by applying a conventional SDR method within the constructed envelope subspace. (iii)Repeat the above two steps many times and integrate these multiple sketches to form the final estimate of the CS. After describing the details of these steps, the asymptotic properties of iRP-SDR are established. Unlike existing methods, iRP-SDR does not involve the determination of the structural dimension until the last stage, which makes it more adaptive to a high-dimensional setting. The advantageous performance of iRP-SDR is demonstrated via simulation studies and a practical example analyzing EEG data.

Robust sufficient dimension reduction via ball covariance

On testing common indices for two multi-index models: A link-free approach

Sufficient dimension reduction (SDR) is a popular framework for supervised dimension reduction, aiming at reducing the dimensionality of input data while information on output data is maximally maintained. On the other hand, in many recent supervised classification learning tasks, it is conceivable that the balance of samples in each class varies between the training and testing phases. Such a phenomenon, referred to as class-prior change, causes existing SDR methods to perform undesirably particularly when the training data is highly imbalanced. In this paper, we extend the state-of-the-art SDR method called leastsquares gradients for dimension reduction (LSGDR) to be able to cope with such class-prior change under the semi-supervised learning setup where unlabeled test data are available in addition to labeled training data. Through experiments, we demonstrate the usefulness of our proposed method.

To reduce the predictors dimension without loss of information on the regression, we develop in this paper a sufficient dimension reduction method which we term cumulative Hessian directions. Unlike many other existing sufficient dimension reduction methods, the estimation of our proposal avoids completely selecting the tuning parameters such as the number of slices in slicing estimation or the bandwidth in kernel smoothing. We also investigate the asymptotic properties of our proposal when the predictors dimension diverges. Illustrations through simulations and an application are presented to evidence the efficacy of our proposal and to compare it with existing methods.

On permutation tests for predictor contribution in sufficient dimension reduction

Sufficient dimension reduction (SDR) techniques have become essential tools in high‐dimensional data analysis. However, many SDR methods rely on iterative procedures, which impose substantial computational costs when applied to large‐scale datasets. To address these challenges, we develop optimal subsampling strategies tailored to two representative SDR approaches: The refined outer product of gradients based on the conditional density functions (rdOPG) and the minimum average (conditional) variance estimation based on the conditional density functions (dMAVE). Our methods are designed to minimize the trace of the asymptotic variance–covariance matrix under an inverse probability weighting framework, thereby achieving an effective trade‐off between computational efficiency and statistical accuracy. We derive explicit forms for the optimal subsampling probabilities and establish the consistency and asymptotic normality of the resulting estimators. Extensive simulation studies and a real‐data application demonstrate that our methods substantially reduce computational burden while achieving improved estimation accuracy compared to uniform subsampling approaches, highlighting their practical value for large‐scale SDR problems.

Due to the demand for tackling the problem of streaming data with high-dimensional covarites, we propose an online sparse sliced inverse regression (OSSIR) method for online sufficient dimension reduction (SDR). The existing online SDR methods focus on the case when [Formula: see text] (dimension of covariates) is small. In this paper, we adapt the sparse sliced inverse regression to cope with high-dimensional streaming data where the dimension [Formula: see text] is large. There are two important steps in our method, one is to extend the online principal component analysis to iteratively obtain the eigenvalues and eigenvectors of the kernel matrix, the other is to use the truncated gradient to perform online [Formula: see text] regularization. Theoretical properties of the proposed online learner are established. By comparing with several existing methods in simulations and real data applications, we demonstrate the effectiveness and efficiency of our method.

A new sufficient dimension reduction method via rank divergence

Optimal transformation: A new approach for covering the central subspace

Because sliced inverse regression (SIR) using the conditional mean of the inverse regression fails to recover the central subspace when the inverse regression mean degenerates, sliced average variance estimation (SAVE) using the conditional variance was proposed in the sufficient dimension reduction literature. However, the efficacy of SAVE depends heavily upon the number of slices. In the present article, we introduce a class of weighted variance estimation (WVE), which, similar to SAVE and simple contour regression (SCR), uses the conditional variance of the inverse regression to recover the central subspace. The strong consistency and the asymptotic normality of the kernel estimation of WVE are established under mild regularity conditions. Finite sample studies are carried out for comparison with existing methods and an application to a real data is presented for illustration.

Distributed estimation in heterogeneous reduced rank regression: With application to order determination in sufficient dimension reduction