Efficient semiparametric estimation in two‐sample comparison via semisupervised learning

The authors consider semiparametric efficient estimation of parameters in the conditional mean model for a simple incomplete data structure in which the outcome of interest is observed only for a random subset of subjects but covariates and surrogate (auxiliary) outcomes are observed for all. They use optimal estimating function theory to derive the semiparametric efficient score in closed form. They show that when covariates and auxiliary outcomes are discrete, a Horvitz‐Thompson type estimator with empirically estimated weights is semiparametric efficient. The authors give simulation studies validating the finite‐sample behaviour of the semiparametric efficient estimator and its asymptotic variance; they demonstrate the efficiency of the estimator in realistic settings.

We study a class of general M-estimators in the semi-supervised setting, wherein the data are typically a combination of a relatively small labeled dataset and large amounts of unlabeled data. A new estimator, which efficiently uses the useful information contained in the unlabeled data, is proposed via a projection technique. We prove consistency and asymptotic normality, and provide an inference procedure based on K -fold cross-validation. The optimal weights are derived to balance the contributions of the labeled and unlabeled data. It is shown that the proposed method, by taking advantage of the unlabeled data, produces asymptotically more efficient estimation of the target parameters than the supervised counterpart. Supportive numerical evidence is shown in simulation studies. Applications are illustrated in analysis of the homeless data in Los Angeles. Supplementary materials for this article are available online.

Deep learning demonstrates impressive performance in many medical image analysis tasks. However, its reliability builds on the labeled medical datasets and the assumption of the same distributions between the training data (source domain) and the test data (target domain). Therefore, some unsupervised medical domain adaptation networks transfer knowledge from the source domain with rich labeled data to the target domain with only unlabeled data by learning domain-invariant features. We observe that conventional adversarial-training-based methods focus on the global distributions alignment and may overlook the class-level information, which will lead to negative transfer. In this paper, we attempt to learn the robust features alignment for the cross-domain medical image analysis. Specifically, in addition to a discriminator for alleviating the domain shift, we further introduce an auxiliary classifier to achieve robust features alignment with the class-level information. We first detect the unreliable target samples, which are far from the source distribution via diverse training between two classifiers. Next, a cross-classifier consistency regularization is proposed to align these unreliable samples and the negative transfer can be avoided. In addition, for fully exploiting the knowledge of unlabeled target data, we further propose a within-classifier consistency regularization to improve the robustness of the classifiers in the target domain, which enhances the unreliable target samples detection as well. We demonstrate that our proposed dual-consistency regularizations achieve state-of-the-art performance on multiple medical adaptation tasks in terms of both accuracy and Macro-F1-measure. Extensive ablation studies and visualization results are also presented to verify the effectiveness of each proposed module. For the skin adaptation results, our method outperforms the baseline and the second-best method by around 10 and 4 percentage points. Similarly, for the COVID-19 adaptation task, our model achieves consistently the best performance in terms of both accuracy (96.93%) and Macro-F1 (86.52%).

Bias-Reduced Log-Periodogram and Whittle Estimation of the Long-Memory Parameter Without Variance Inflation

Current methods used to analyze time to event data either rely on highly parametric assumptions which result in biased estimates of parameters which are purely chosen out of convenience, or are highly unstable because they ignore the global constraints of the true model. By using Targeted Maximum Likelihood Estimation (TMLE) one may consistently estimate parameters which directly answer the statistical question of interest. Targeted Maximum Likelihood Estimators are substitution estimators, which rely on estimating the underlying distribution. However, unlike other substitution estimators, the underlying distribution is estimated specifically to reduce bias in the estimate of the parameter of interest. We will present here an extension of TMLE for observational time to event data, the Collaborative Targeted Maximum Likelihood Estimator (C-TMLE) for the treatment specific survival curve. Through the use of a simulation study we will show that this method improves on commonly used methods in both robustness and efficiency. In fact, we will show that in certain situations the C-TMLE produces estimates whose mean square error is lower than the semi-parametric efficiency bound. We will also demonstrate that a semi-parametric efficient substitution estimator (TMLE) outperforms a semi-parametric efficient non-substitution estimator (the Augmented Inverse Probability Weighted estimator) in sparse data situations. Lastly, we will show that the bootstrap is able to produce valid 95 percent confidence intervals in sparse data situations, while influence curve based inference breaks down.

A common and important problem in clustered sampling designs is that the effect of within-cluster exposures (i.e., exposures that vary within clusters) on outcome may be confounded by both measured and unmeasured cluster-level factors (i.e., measurements that do not vary within clusters). When some of these are ill/not accounted for, estimation of this effect through population-averaged models or random-effects models may introduce bias. We accommodate this by developing a general theory for the analysis of clustered data, which enables consistent and asymptotically normal estimation of the effects of within-cluster exposures in the presence of cluster-level confounders. Semiparametric efficient estimators are obtained by solving so-called conditional generalized estimating equations. We compare this approach with a popular proposal by Neuhaus and Kalbfleisch (1998, Biometrics 54, 638-645) who separate the exposure effect into a within- and a between-cluster component within a random intercept model. We find that the latter approach yields consistent and efficient estimators when the model is linear, but is less flexible in terms of model specification. Under nonlinear models, this approach may yield inconsistent and inefficient estimators, though with little bias in most practical settings.

To address the objective in a clinical trial to estimate the mean or mean difference of an expensive endpoint Y, one approach employs a two-phase sampling design, wherein inexpensive auxiliary variables W predictive of Y are measured in everyone, Y is measured in a random sample, and the semiparametric efficient estimator is applied. This approach is made efficient by specifying the phase two selection probabilities as optimal functions of the auxiliary variables and measurement costs. While this approach is familiar to survey samplers, it apparently has seldom been used in clinical trials, and several novel results practicable for clinical trials are developed. We perform simulations to identify settings where the optimal approach significantly improves efficiency compared to approaches in current practice. We provide proofs and R code. The optimality results are developed to design an HIV vaccine trial, with objective to compare the mean 'importance-weighted' breadth (Y) of the T-cell response between randomized vaccine groups. The trial collects an auxiliary response (W) highly predictive of Y and measures Y in the optimal subset. We show that the optimal design-estimation approach can confer anywhere between absent and large efficiency gain (up to 24 % in the examples) compared to the approach with the same efficient estimator but simple random sampling, where greater variability in the cost-standardized conditional variance of Y given W yields greater efficiency gains. Accurate estimation of E[Y | W] is important for realizing the efficiency gain, which is aided by an ample phase two sample and by using a robust fitting method.

Efficient local IV estimation of an empirical auction model

Efficient estimation for partially linear varying coefficient models when coefficient functions have different smoothing variables

Social scientists often classify text documents to use the resulting labels as an outcome or a predictor in empirical research. Automated text classification has become a standard tool since it requires less human coding. However, scholars still need many human-labeled documents for training. To reduce labeling costs, we propose a new algorithm for text classification that combines a probabilistic model with active learning. The probabilistic model uses both labeled and unlabeled data, and active learning concentrates labeling efforts on difficult documents to classify. Our validation study shows that with few labeled data, the classification performance of our algorithm is comparable to state-of-the-art methods at a fraction of the computational cost. We replicate the results of two published articles with only a small fraction of the original labeled data used in those studies and provide open-source software to implement our method.

We propose UnMixMatch, a semi-supervised learning framework which can learn effective representations from unconstrained unlabelled data in order to scale up performance. Most existing semi-supervised methods rely on the assumption that labelled and unlabelled samples are drawn from the same distribution, which limits the potential for improvement through the use of free-living unlabeled data. Consequently, the generalizability and scalability of semi-supervised learning are often hindered by this assumption. Our method aims to overcome these constraints and effectively utilize unconstrained unlabelled data in semi-supervised learning. UnMixMatch consists of three main components: a supervised learner with hard augmentations that provides strong regularization, a contrastive consistency regularizer to learn underlying representations from the unlabelled data, and a self-supervised loss to enhance the representations that are learnt from the unlabelled data. We perform extensive experiments on 4 commonly used datasets and demonstrate superior performance over existing semi-supervised methods with a performance boost of 4.79%. Extensive ablation and sensitivity studies show the effectiveness and impact of each of the proposed components of our method. The code for our work is publicly available.

Longitudinal studies are widely used in various fields, such as public health, clinic trials and financial data analysis. A major challenge for longitudinal studies is the repeated measurements from each subject, which cause time dependent correlations within subjects. Generalized Estimating Equations (GEE) can deal with correlated outcomes for longitudinal data through marginal effect. Our proposed model will be based on GEE, with a semi-parametric approach, to provide a flexible structure for regression models: coefficients for parametric covariates will be estimated and nuisance covariates will be fitted in kernel smoothers for the non-parametric part. The profile kernel estimator and the seemingly unrelated kernel estimator (SUR) will be used to obtain consistent and efficient semi-parametric estimators. We provide simulation results for estimating semi-parametric models with one or multiple non-parametric terms. Financial market data is a major component of data analysis; thus, we focus on the financial market in the application part. Credit card loan data will be used with the payment information for each customer across six months to investigate whether gender, income, age, or other factors will influence payment status significantly. Furthermore, we propose model comparisons to evaluate whether different models should be fitted for different subgroups of consumers, such as male and female.

Semisupervised learning (SSL) has been extensively studied in related literature. Despite its success, many existing learning algorithms for semisupervised problems require specific distributional assumptions, such as "cluster assumption" and "low-density assumption," and thus, it is often hard to verify them in practice. We are interested in quantifying the effect of SSL based on kernel methods under a misspecified setting. The misspecified setting means that the target function is not contained in a hypothesis space under which some specific learning algorithm works. Practically, this assumption is mild and standard for various kernel-based approaches. Under this misspecified setting, this article makes an attempt to provide a theoretical justification on when and how the unlabeled data can be exploited to improve inference of a learning task. Our theoretical justification is indicated from the viewpoint of the asymptotic variance of our proposed two-step estimation. It is shown that the proposed pointwise nonparametric estimator has a smaller asymptotic variance than the supervised estimator using the labeled data alone. Several simulated experiments are implemented to support our theoretical results.

Probabilistic models have been previously shown to be efficient and effective for modeling and recognition of human motion. In particular we focus on methods which represent the human motion model as a triangulated graph. Previous approaches learned models based just on positions and velocities of the body parts while ignoring their appearance. Moreover, a heuristic approach was commonly used to obtain translation invariance. In this paper we suggest an improved approach for learning such models and using them for human motion recognition. The suggested approach combines multiple cues, i.e., positions, velocities and appearance into both the learning and detection phases. Furthermore, we introduce global variables in the model, which can represent global properties such as translation, scale or view-point. The model is learned in an unsupervised manner from unlabelled data. We show that the suggested hybrid probabilistic model (which combines global variables, like translation, with local variables, like relative positions and appearances of body parts), leads to: (i) faster convergence of learning phase, (it) robustness to occlusions, and, (Hi) higher recognition rate.

Domain adaptive object detection (DAOD) assumes that both labeled source data and unlabeled target data are available for training, but this assumption does not always hold in real-world scenarios. Thus, source-free DAOD is proposed to adapt the source-trained detectors to target domains with only unlabeled target data. Existing source-free DAOD methods typically utilize pseudo labeling, where the performance heavily relies on the selection of confidence threshold. However, most prior works adopt a single fixed threshold for all classes to generate pseudo labels, which ignore the imbalanced class distribution, resulting in biased pseudo labels. In this work, we propose a refined pseudo labeling framework for source-free DAOD. First, to generate unbiased pseudo labels, we present a category-aware adaptive threshold estimation module, which adaptively provides the appropriate threshold for each category. Second, to alleviate incorrect box regression, a localization-aware pseudo label assignment strategy is introduced to divide labels into certain and uncertain ones and optimize them separately. Finally, extensive experiments on four adaptation tasks demonstrate the effectiveness of our method.