Optimal incremental learning under covariate shift

Theory, algorithms, and applications of machine learning techniques to overcome “covariate shift” non-stationarity. As the power of computing has grown over the past few decades, the field of machine learning has advanced rapidly in both theory and practice. Machine learning methods are usually based on the assumption that the data generation mechanism does not change over time. Yet real-world applications of machine learning, including image recognition, natural language processing, speech recognition, robot control, and bioinformatics, often violate this common assumption. Dealing with non-stationarity is one of modern machine learning's greatest challenges. This book focuses on a specific non-stationary environment known as covariate shift, in which the distributions of inputs (queries) change but the conditional distribution of outputs (answers) is unchanged, and presents machine learning theory, algorithms, and applications to overcome this variety of non-stationarity. After reviewing the state-of-the-art research in the field, the authors discuss topics that include learning under covariate shift, model selection, importance estimation, and active learning. They describe such real world applications of covariate shift adaption as brain-computer interface, speaker identification, and age prediction from facial images. With this book, they aim to encourage future research in machine learning, statistics, and engineering that strives to create truly autonomous learning machines able to learn under non-stationarity.

Learning distance metrics is a fundamental problem in machine learning. Previous distance-metric learning research assumes that the training and test data are drawn from the same distribution, which may be violated in practical applications. When the distributions differ, a situation referred to as covariate shift, the metric learned from training data may not work well on the test data. In this case the metric is said to be inconsistent. In this paper, we address this problem by proposing a novel metric learning framework known as consistent distance metric learning (CDML), which solves the problem under covariate shift situations. We theoretically analyze the conditions when the metrics learned under covariate shift are consistent. Based on the analysis, a convex optimization problem is proposed to deal with the CDML problem. An importance sampling method is proposed for metric learning and two importance weighting strategies are proposed and compared in this work. Experiments are carried out on synthetic and real world datasets to show the effectiveness of the proposed method.

During on-site verification activities conducted by the Technical Secretariat of Organization for the Prohibition of Chemical Weapons, identification by gas chromatography retention indices (RI) data, in addition to mass spectrometry data, increase the reliability of factual findings. However, reference RIs do not cover all the possible chemical structures. That is why it is important to have models to predict RIs. Applicable only for narrow data sets of chemicals with a fixed scaffold (G- and V-series gases as example), the non-learning incremental method demonstrated predictive median absolute and percentage errors of 2–4 units and 0.1–0.2%; these are comparable with the experimental bias in RI measurements in the same laboratory with the same GC conditions. It outperforms the accuracy of two reported machine learning methods–median absolute and percentage errors of 11–52 units and 0.5–2.8%. However, for the whole Chemical Weapons Convention (CWC) data set of chemicals, when a fixed scaffold is absent, the incremental method is not applicable; essential machine learning methods achieved accuracy: median absolute and percentage errors of 29–33 units and 0.5–2.2%, depending on the machine learning method. In addition, we have developed a homology tree approach as a convenient method for the visualization of the CWC chemical space. We conclude that non-learning incremental methods may be more accurate than the state-of-the-art machine learning techniques in particular cases, such as predicting the RIs of homologues and isomers of chemicals related to CWC.

Computer vision applications predict on digital images acquired by a camera from physical scenes through light. However, conventional robustness benchmarks rely on perturbations in digitized images, diverging from distribution shifts occurring in the image acquisition process. To bridge this gap, we introduce a new distribution shift dataset, ImageNet-ES, comprising variations in environmental and camera sensor factors by directly capturing 202k images with a real camera in a controllable testbed. With the new dataset, we evaluate out-of-distribution (OOD) detection and model robustness. We find that existing OOD detection methods do not cope with the covariate shifts in ImageNet-ES, implying that the definition and detection of OOD should be revisited to embrace real-world distribution shifts. We also observe that the model becomes more robust in both ImageNet-C and -ES by learning environment and sensor variations in addition to existing digital augmentations. Lastly, our results suggest that effective shift mitigation via camera sensor control can significantly improve performance without increasing model size. With these findings, our benchmark may aid future research on robustness, OOD, and camera sensor control for computer vision. Our code and dataset are available at https://github.com/Edw2n/ImageNet-ES.

Online learning has become crucial to many problems in machine learning. As more data is collected sequentially, quickly adapting to changes in the data distribution can offer several competitive advantages such as avoiding loss of prior knowledge and more efficient learning. However, adaptation to changes in the data distribution (also known as covariate shift) needs to be performed without compromising past knowledge already built in into the model to cope with voluminous and dynamic data. In this paper, we propose an online stacked Denoising Autoencoder whose structure is adapted through reinforcement learning. Our algorithm forces the network to exploit and explore favourable architectures employing an estimated utility function that maximises the accuracy of an unseen validation sequence. Different actions, such as Pool, Increment and Merge are available to modify the structure of the network. As we observe through a series of experiments, our approach is more responsive, robust, and principled than its counterparts for non-stationary as well as stationary data distributions. Experimental results indicate that our algorithm performs better at preserving gained prior knowledge and responding to changes in the data distribution.

Ensemble learning has become a cornerstone of modern machine learning, yet most existing approaches such as bagging, boosting, and random forests lack theoretical guarantees in high-dimensional, nonlinear data regimes. In this study, we propose an Adaptive Nonlinear Ensemble Learning (ANEL) framework that integrates data-dependent model selection with distribution-free statistical guarantees. The framework adaptively weights nonlinear base learners according to their local predictive relevance, thereby improving both accuracy and robustness in heterogeneous, high-dimensional datasets. Unlike conventional methods that rely on asymptotic assumptions, our approach provides distribution-free generalization error bounds using concentration inequalities and PAC-Bayesian analysis. Through theoretical derivations and empirical validation on synthetic and real-world datasets (including biomedical imaging and climate prediction), ANEL demonstrates: (i) provable reduction in generalization error compared to standard ensemble methods, (ii) scalability to datasets with millions of features through parallel optimization, and (iii) robustness against covariate shift and heavy-tailed distributions. This contribution advances ensemble methodology by bridging algorithmic adaptivity with rigorous statistical guarantees, offering a novel paradigm for reliable machine learning in high-dimensional environments.

Many machine learning methods assume that the training and test data follow the same distribution. However, in the real world, this assumption is often violated. In particular, the marginal distribution of the data changes, called covariate shift, is one of the most important research topics in machine learning. We show that the well-known family of covariate shift adaptation methods is unified in the framework of information geometry. Furthermore, we show that parameter search for a geometrically generalized covariate shift adaptation method can be achieved efficiently. Numerical experiments show that our generalization can achieve better performance than the existing methods it encompasses.

Background and Objective Medical machine learning (ML) models tend to perform better on data from the same cohort than on new data, often due to overfitting, or co-variate shifts. For these reasons, external validation (EV) is a necessary practice in the evaluation of medical ML. However, there is still a gap in the literature on how to interpret EV results and hence assess the robustness of ML models. MethodsWe fill this gap by proposing a meta-validation method, to assess the soundness of EV procedures. In doing so, we complement the usual way to assess EV by considering both dataset cardinality, and the similarity of the EV dataset with respect to the training set. We then investigate how the notions of cardinality and similarity can be used to inform on the reliability of a validation procedure, by integrating them into two summative data visualizations. ResultsWe illustrate our methodology by applying it to the validation of a state-of-the-art COVID-19 diagnostic model on 8 EV sets, collected across 3 different continents. The model performance was moderately impacted by data similarity (Pearson ρ = 0.38, p< 0.001). In the EV, the validated model reported good AUC (average: 0.84), acceptable calibration (average: 0.17) and utility (average: 0.50). The validation datasets were adequate in terms of dataset cardinality and similarity, thus suggesting the soundness of the results. We also provide a qualitative guideline to evaluate the reliability of validation procedures, and we discuss the importance of proper external validation in light of the obtained results. ConclusionsIn this paper, we propose a novel, lean methodology to: 1) study how the similarity between training and validation sets impacts the generalizability of a ML model; 2) assess the soundness of EV evaluations along three complementary performance dimensions: discrimination, utility and calibration; 3) draw conclusions on the robustness of the model under validation. We applied this methodology to a state-of-the-art model for the diagnosis of COVID-19 from routine blood tests, and showed how to interpret the results in light of the presented framework.

Images are uploaded to the Internet over time which makes concept drifting and distribution change in semantic classes unavoidable. Current hashing methods being trained using a given static database may not be suitable for nonstationary semantic image retrieval problems. Moreover, directly retraining a whole hash table to update knowledge coming from new arriving image data may not be efficient. Therefore, this paper proposes a new incremental hash-bit learning method. At the arrival of new data, hash bits are selected from both existing and newly trained hash bits by an iterative maximization of a 3-component objective function. This objective function is also used to weight selected hash bits to re-rank retrieved images for better semantic image retrieval results. The three components evaluate a hash bit in three different angles: 1) information preservation; 2) partition balancing; and 3) bit angular difference. The proposed method combines knowledge retained from previously trained hash bits and new semantic knowledge learned from the new data by training new hash bits. In comparison to table-based incremental hashing, the proposed method automatically adjusts the number of bits from old data and new data according to the concept drifting in the given data via the maximization of the objective function. Experimental results show that the proposed method outperforms existing stationary hashing methods, table-based incremental hashing, and online hashing methods in 15 different simulated nonstationary data environments.

With the exponential growth in the speed and volume of remote sensing data, deep learning models are expected to adapt and continually learn over time. Unfortunately, the domain shift between multi-source remote sensing data from various sensors and regions poses a significant challenge. Segmentation models face difficulty in adapting to incremental domains due to catastrophic forgetting, which can be addressed via incremental learning methods. However, current incremental learning methods mainly focus on class-incremental learning, wherein classes belong to the same remote sensing domain, and neglect investigations into incremental domains in remote sensing. To solve this problem, we propose a domain-incremental learning method for semantic segmentation in multi-source remote sensing data. Specifically, our model aims to incrementally learn a new domain while preserving its performance on previous domains without accessing previous domain data. To achieve this, our model has a unique parameter learning structure that reparametrizes domain-agnostic and domain-specific parameters. We use different optimization strategies to adapt to domain shift in incremental domain learning. Additionally, we adopt multi-level knowledge distillation loss to mitigate the impact of label space shift among domains. The experiments demonstrate that our method achieves excellent performance in domain-incremental settings, outperforming existing methods with only a few parameters.

In the field of machine learning, incremental learning methods allow models to learn new tasks without forgetting the old ones, but the catastrophic forgetting problem often encountered in this process greatly limits their applications. In this paper, an improved algorithm EPASS (Enhanced Prototype Augmentation and Self-Supervision) is proposed for incremental learning in image classification tasks, which effectively alleviates the limitations of the traditional PASS algorithm and the EWC algorithm in terms of catastrophic forgetting, sensitivity of parameter selection, and computational cost. The EPASS algorithm improves the flexibility and adaptability of the model by automatically selecting which prototypes to keep for different tasks through a dynamic prototype storage strategy. Meanwhile, the EWC-based prototype update strategy allows prototypes to be updated as new data arrives, further improving the model's adaptability to new tasks while reducing computational cost. Experimental results on the CIFAR-100 dataset show that the EPASS algorithm outperforms the PASS algorithm to some extent in terms of accuracy at all stages. These results validate the effectiveness of the EPASS algorithm in mitigating the catastrophic forgetting problem and provide new directions for future incremental learning research.

Data Assimilation with Missing Data in Nonstationary Environments for Probabilistic Machine Learning Models

Premature rupture of membranes (PROM) poses significant risks to both maternal and neonatal health. This study aims to construct a risk factor prediction model related to PROM by using machine learning technology and explore the association with nutritional inflammatory index. A retrospective analysis was conducted on patients with PROM. Based on the variables screened out by ridge regression and Boruta algorithm, univariate and multivariate logistic regression analyses were further adopted. According to the sample data, it is divided into the training set and the internal validation set in a ratio of 7:3. The research group adopted four machine learning algorithms: Extreme Gradient Boost (XGBoost), Support Vector Machine (SVM), Logistic Regression (LR), and Random Forest (RF). The selected variables were incorporated into model construction, with the area under the receiver operating characteristic (ROC) curve (AUC) serving as a criterion for model selection. Model performance was assessed using AUC values, sensitivity, specificity, recall, F1 score, and accuracy. The variables were selected based on the contribution degree of the variables in Shapley additive Interpretation (SHAP) to construct the nomogram. A retrospective analysis was conducted involving 800 parturients at Binhai County People's Hospital from January 2023 to October 2024, comprising 400 with PROM and 400 with normal delivery. The RF model demonstrated superior performance with an AUC of 0.757, sensitivity of 67.4%, and specificity of 65.1%. Key predictive factors identified included body mass index (BMI), prognostic nutritional index (PNI), platelet, albumin, and aggregate index of systemic inflammation (AISI). The ROC of the model also showed good efficacy, with an AUC of 0.777. This study highlights the potential of machine learning in enhancing the understanding and prediction of PROM, and emphasizes the significance of inflammatory and nutritional indicators, paving the way for future research in maternal-fetal medicine.

The increasing availability of numerous heterogeneous but related sources of data arises in many fields, such as bioinformatics, robotics, computer vision, information retrieval, and many others. Multiple sources of data can be viewed as different, related views of the same learning problem, where dependencies between the views could potentially take on complex structures. This gives rise to interesting and challenging machine learning problems where data sources are combined for learning. This framework encompasses several data fusion tasks and related topics, such as transfer learning, multitask learning, multiview learning, and learning under covariate shift. Often the general concept is to augment training data for each problem of interest with data from other partially related learning problems. Several approaches for inferring and exploiting complex relationships between sources have been presented. The advantages of the multiple source learning paradigm can be seen in situations where individual data sources are noisy, incomplete, and learning from more than one source can filter out problem-independent noise. Additionally, another aspect of multiple source learning is that data sources need only contain partially relevant information to the desired learning problem, making it possible to exploit a large number of data sources. This special issue on “Learning from Multiple Sources” was inspired by a successful (eponymous) NIPS workshop which took place in Whistler (Canada) on December 13, 2008. The workshop received twenty-three submissions, from which seven contributed talks and

Summary Traditional statistical and machine learning methods typically assume that the training and test data follow the same distribution. However, this assumption is frequently violated in real-world applications, where the training data in the source domain may underrepresent specific subpopulations in the test data of the target domain. This article addresses target-independent learning under covariate shift, focusing on multicalibration for survival probability and restricted mean survival time. A black-box post-processing boosting algorithm specifically designed for censored survival data is introduced. By leveraging pseudo-observations, our method produces a multicalibrated predictor that is competitive with inverse propensity score weighting in predicting the survival outcome in an unlabelled target domain, ensuring, not only overall accuracy, but also fairness across diverse subpopulations. Our theoretical analysis of pseudo-observations builds upon the functional delta method and the $ p $-variational norm. The algorithm’s sample complexity, convergence properties and multicalibration guarantees for post-processed predictors are provided. The results establish a fundamental connection between multicalibration and universal adaptability, demonstrating that our calibrated function is comparable to, or outperforms, the inverse propensity score weighting estimator. Extensive numerical simulations and a real-world case study on cardiovascular disease risk prediction using two large prospective cohort studies validate the effectiveness of our approach.