PUMA: Performance Unchanged Model Augmentation for Training Data Removal

Similarity search plays an important role in many applications involving high-dimensional data. Due to the known dimensionality curse, the performance of most existing indexing structures degrades quickly as the feature dimensionality increases. Hashing methods, such as locality sensitive hashing (LSH) and its variants, have been widely used to achieve fast approximate similarity search by trading search quality for efficiency. However, most existing hashing methods make use of randomized algorithms to generate hash codes without considering the specific structural information in the data. In this paper, we propose a novel hashing method, namely, robust hashing with local models (RHLM), which learns a set of robust hash functions to map the high-dimensional data points into binary hash codes by effectively utilizing local structural information. In RHLM, for each individual data point in the training dataset, a local hashing model is learned and used to predict the hash codes of its neighboring data points. The local models from all the data points are globally aligned so that an optimal hash code can be assigned to each data point. After obtaining the hash codes of all the training data points, we design a robust method by employing l2,1 -norm minimization on the loss function to learn effective hash functions, which are then used to map each database point into its hash code. Given a query data point, the search process first maps it into the query hash code by the hash functions and then explores the buckets, which have similar hash codes to the query hash code. Extensive experimental results conducted on real-life datasets show that the proposed RHLM outperforms the state-of-the-art methods in terms of search quality and efficiency.

We study classifiers based on prototypes, which classify data points by the nearest neighbor rule, namely a data point is labeled the class which has its nearest prototype. The prototype-based classifiers are trained by the max-margin principle, which is used by support vector machines. For each training data point, we consider the difference of the distances between the nearest prototype of the class to which the point belongs and the nearest one of the other classes. And then, we define a margin by the minimum value of these differences for all the training data points. We verify the efficiency of the obtained classifier by numerical experiments.

The K Nearest Neighbor (kNN) method has widely been used in the applications of data mining and machine learning due to its simple implementation and distinguished performance. However, setting all test data with the same k value in the previous kNN methods has been proven to make these methods impractical in real applications. This article proposes to learn a correlation matrix to reconstruct test data points by training data to assign different k values to different test data points, referred to as the Correlation Matrix kNN (CM-kNN for short) classification. Specifically, the least-squares loss function is employed to minimize the reconstruction error to reconstruct each test data point by all training data points. Then, a graph Laplacian regularizer is advocated to preserve the local structure of the data in the reconstruction process. Moreover, an ℓ 1 -norm regularizer and an ℓ 2, 1 -norm regularizer are applied to learn different k values for different test data and to result in low sparsity to remove the redundant/noisy feature from the reconstruction process, respectively. Besides for classification tasks, the kNN methods (including our proposed CM-kNN method) are further utilized to regression and missing data imputation. We conducted sets of experiments for illustrating the efficiency, and experimental results showed that the proposed method was more accurate and efficient than existing kNN methods in data-mining applications, such as classification, regression, and missing data imputation.

Finding a new drug which is both safe and efficient is an expensive and time-consuming endeavour. In particular, establishing the ‘dose-effect relationship’ – how beneficial a drug is at different dosages – can be challenging. Predicting this curve requires gathering experimental data by exposing and recording how cells respond to various levels of the drug. However, extreme values are often observed at low and high dosages, potentially introducing errors that are hard to correct in the prediction process. Yet, these extreme observations are sometimes genuine so researchers cannot just ignore them. To improve dose-effect estimation, Zhou, Liu, Fang et al. developed a new general-purpose approach. It uses advanced statistical modelling to account for extremes in lab data. This strategy outperformed other methods when dealing with these observations while also providing higher efficiency in data analysis with more uniform data in experiments. To facilitate implementation, Zhou, Liu, Fang et al. set up a user-friendly tool baptized ‘REAP’; this free online resource allows scientists without advanced statistical experience to harness the new approach and to perform dose-effect analysis more easily and accurately. This could boost research across many different disciplines that examine the effects of chemicals on cells.

The functions of multiple instances (FUMI) approach for learning target and nontarget signatures is introduced. FUMI is a generalization of the multiple-instance learning (MIL) approach for supervised learning. FUMI differs significantly from standard MIL and supervised learning approaches because only data points which are functions of class concepts/signatures are available. In particular, this paper addresses the problem in which data points are convex combinations of target and nontarget signatures. Two algorithms, convex FUMI ( $c$ FUMI) and extended $c$ FUMI ( $e$ FUMI) , are presented and applied to the problem of hyperspectral unmixing and target detection. $c$ FUMI learns target and nontarget signatures (i.e., target and nontarget endmembers) , the number of nontarget signatures, and the proportion of each signature for every data point. The $e$ FUMI algorithm extends the $c$ FUMI to allow for additional “bag level” uncertainty in training labels. For these methods, training data need only binary labels indicating whether a data point (or some spatial area in the case of $e$ FUMI) contains or does not contain some proportion of the target; the specific target proportions for the training data are not needed. After learning the target signature using the binary-labeled training data, target detection can be performed on test data. Results for subpixel target detection on simulated and real airborne hyperspectral data are shown.

In recent years, with the rapid development of large language models, optimizing BERT's performance on small-scale datasets using large language models has gradually become a research hotspot. To this end, this paper proposes a data augmentation method based on large language models to enhance BERT's performance in text classification tasks. Specifically, we first use large language models to back-translate the training data. By translating the text into other languages and then back into the original language, we generate new samples that are semantically consistent but have diverse expressions, thereby increasing the diversity of the training data. Subsequently, the augmented training set is used to train the BERT model, which significantly improves classification accuracy on the Reuters News Classification dataset. Experimental results show that this method effectively mitigates the limitations of small-scale datasets and significantly enhances the model's generalization ability, providing a novel and efficient solution for text classification tasks.

In machine translation evaluation, the traditional wisdom measures model's generalization ability in an average sense, for example by using corpus BLEU. However, the statistics of corpus BLEU cannot provide comprehensive understanding and fine-grained analysis on model's generalization ability. As a remedy, this paper attempts to understand NMT at fine-grained level, by detecting contextual barriers within an unseen input sentence that cause the degradation in model's translation quality. It proposes a principled definition of source contextual barriers as well as its modified version which is tractable in computation and operates at word-level. Based on the modified one, three simple methods are proposed for barrier detection by search-aware risk estimation through counterfactual generation. Extensive analyses are conducted on those detected contextual barrier words on both Zh $\Leftrightarrow$ En NIST benchmarks. Potential usages motivated from barrier words are also discussed.

Many theoretical or experimental studies has shown that ensemble learning is an effective technique to achieve better classification accuracy and stability than individual classifiers. In this paper, we propose a novel weighted classifier ensemble method through classifiers pruning with two stages. In the first stage, we use canonical correlation analysis (CCA) to model maximum correlation relationships between training data points and base classifiers. Based on such globally multi-linear projections, a sparse regression method is proposed to prune base classifiers so that each test data point will dynamically select a subset of classifiers to form a unique classifier ensemble, to decrease effects of noisy input data and incorrect classifiers in such a global view. In the second stage, the pruned classifiers are weighted locally by a fusion method, which utilizes the generalization ability of pruned classifiers among nearest neighbors of testing data points. By this way, each test data point can build a unique locally weighted classifier ensemble. Analysis of experimental results on several UCI data sets shows that the classification results of our method are better than other ensemble methods such as Random Forests, Majority Voting, AdaBoost and DREP.

Surface display techniques, including contour mapping, usually use an interpolant when data values are on a rectangular grid; but when values are irregularly located some form of assumed or estimated distance-weighting function is often used to estimate the influence of data points at other locations. If moving average (or some other) techniques are then applied, slopes may be zero at data points; extrema may fall at data points exclusively; or surfaces may not pass through data points.An alternative method is proposed whereby a map is automatically divided into suitable triangular domains with a data point at each vertex. Surface estimation and contour plotting are then performed independently for each triangle using a measured or best-fit plane associated with each data point. The approach requires the utilization of three non-original aspects: 1) the derivation of a local homogeneous area coordinate system for any arbitrary triangle; 2) the construction of a data-structure linking each triangular domain with its three neighbouring triangles and three associated data points, and 3) the use of a conforming triangular finite-element interpolating function.Use of the first two of these concepts permits the economic generation and optimization of a triangular mesh from a set of data points. Optimization criteria used to define the best triangular partition are described in some detail, along with computer timing for this step. Use of the first and third concepts permits the interpolation of a smooth surface over the whole map even though each triangular element is estimated and plotted independently. The requirements for a suitable interpolating function are discussed, and an interpolant is suggested that preserves elevation and slope at each data point as well as elevation and slope continuity between domains. Extrema need not be located at data points but are constrained by their associated planes. Contour line segments are produced by division of each triangle into N2 sub-triangles, elevation estimation at each resulting node, and linear interpolation and plotting withing each sub-triangle. Resolution is then a function of N and local data point density. Extension of the method to higher dimensions is briefly discussed.

This paper focuses on the domain generalization task, which aims to learn a model that generalizes to unseen domains by utilizing multiple training domains. More specifically, we follow the idea of adversarial data augmentation, which aims to synthesize and augment training data with “hard” domains for improving the model's domain generalization ability. Previous works augment training data only with samples similar to the training data, resulting in limited generalization ability. We propose a novel adversarial data augmentation method, termed GADA (Generative Adversarial Domain Augmentation), which employs an image-to-image translation model to obtain a distribution of novel domains that are semantically different from the training domains, and, at the same time, hard to classify. Evaluation and further analysis suggest that adversarial data augmentation with semantically different samples leads to better domain generalization performance.

All data presented pertain to 11 advertised brands of same low-priced, high-consumption food product. The data span an eight-year period with nine data points for each brand. Two measurements were taken during first year with about a four-month interval between readings; remaining seven measurements were obtained at approximate yearly intervals. The 11 brands came from three separate markets, and each reported value was based on no fewer than 200 principal female shoppers in a family who used product category. Sampling was of two types: (1) replicated probability samples of blocks with clusters split proportionally between central city and surrounding suburban towns or (2) random samples from metropolitan telephone directory with specification of individual telephone numbers. The share-of-families data were obtained from question: of do you buy? If respondents had difficulty with usually buy, interviewers were instructed to add: the you buy most often. Share of families, for study's food category, is judged to be a reasonable estimate of market share based on an additional analysis. The surveys also included a question on units purchased per week. These data, in recent surveys, have been multiplied by users to develop individual volume shares. The resulting percentages have been close to share-offamily percentages. The reliability of brand bought as an indicator of share of families was also examined. Data from two additional questions were analyzed: What of do you have in house right now? and particular have you purchased most often in past three weeks? Although there are interesting differences, high similarity in percentages exists across three questions. The top-of-mind awareness data were obtained by asking, When you think of (the type of product) what first comes to mind? This was first question asked a respondent, after she was qualified as principal female shopper and a user of pro uct category. The top-of-mind awareness and share-of-families data for 11 brands are shown in figure. All data points are on a percentage basis. The order is based on market performance accumulated over nine data points. For each change in share-of-families between successive data points was determined algebraically by subtracting share of families for each data point from preceding data point. These values were accumulated and 11 brands were rank-ordered by algebraic total as shown in Column 1 of Tables 1 and 2. Thus, Brand A has largest total data point increase in share of families, and Brand K has largest total decrease in data point share changes.

By comparison with the commonly seen visible light images that can be effectively characterized within a Euclidean space, infrared images have non-Euclidean characteristics since their pixels contain rich thermal radiation information, such as heat distribution, surface temperature and thermal radiation. Considering the advantages of Graph Convolutional Networks (GCNs) in processing non-Euclidean data, this study proposes to introduce the GCNs to estimate the quality of infrared images by developing the Node-to-Graph Regression (NGR) model. To specify, the proposed NGR model is composed of two main steps, namely network establishment and network training. In the first step, following the classical researches of image quality estimation that include local distortion measurement followed by pooling for inferring the image quality score, this study captures the local distortion of the input infrared images by stacking up a set of Vision Graph (VSG) blocks to generate one node map, and then conducts the weighted pooling method on the node map to yield the graph output as the estimated quality score. In the second step, for enhancing the model's performance and generalization ability in the network training process, this study implements the node regression with the big data pre-training method to raise the local distortion extraction ability in a broad range of image scenarios and distortion intensities, and then performs the graph regression by using the knowledge distillation method to reduce the over-fitting risk. Using the largest-size infrared image quality evaluation database (I2QED), this study compared the proposed NGR model with three dozen mainstream and state-of-the-art competitors, and results showed that our proposed NGR model achieved the optimal performance.

Medical image segmentation is vital for clinical diagnosis, yet deep learning models face domain shift challenges when test data distributions differ from training data. Single-source domain generalization (DG) has emerged as a solution to these limitations of deep learning models by training models on a single source domain to generalize to unseen target domains. Current single-source DG methods employ domain randomization (e.g., input/feature-space perturbations) to simulate unseen target domains, but face two key limitations: (1) Texture bias in CNN-based architectures due to their local processing nature, leading to overfitting and poor generalization; (2) Restricted augmented style diversity caused by solely source-dependent feature perturbations and input- or feature-only augmentation, resulting in insufficient coverage of target domain distributions and degraded model's generalizationability. Motivated by the Fourier transform's capability to model global dependency and decouple domain invariant features, we propose Random Domain Generalization (RandDG), a frequency-aware domain randomization method for single-source DG in medical image segmentation. This method enhances generalization ability through coordinated input and feature spaces perturbations while maintaining parameter efficiency via lightweight architectural design in the frequencydomain. The proposed RandDG incorporates a novel Global U-Shape Network (GUNet) segmentation architecture for efficient long-range dependency modeling and a Uniform Low Frequency spectrum Transform (ULoFT) filter for feature-space perturbation to enhance domain generalization ability, facilitating the learning of global domain invariant structural information through consistency constraints within the dual-space randomization framework. More specifically, first, the GUNet architecture combines a lightweight encoder with frequency-domain global filtering and forms a U-shape structure through a skip-connected decoder, utilizing Fourier transforms to capture long-range dependencies while minimizing parameters. Second, the ULoFT filter perturbs feature-space styles by Bernoulli mixing statistics of the source low-frequency amplitude spectrum with uniformly sampled values, thereby reducing sole reliance on source statistics and broadening the coverage of potential target domains. Third, a dual-space randomization framework simultaneously applies input-space style augmentation via global intensity nonlinear augmentation (GIN) filters and feature-space amplitude spectrum perturbation via ULoFT filter, regularized by a consistency loss to enforce domain invariant representationlearning. Experiments were conducted on an abdominal CT-MRI dataset for abdominal multi-organ segmentation from CT to T2 spectral presaturation with inversion recovery (T2-SPIR) MRI and a cross-center prostate dataset for prostate segmentation from one center to others. The proposed RandDG method demonstrated superior generalization ability, significantly improving robustness against domain shifts compared to competitive methods. When tested on unseen domains, the proposed method achieved an average DSC of 87.96% and an average HD of 4.82 mm on the abdominal CT-MRI dataset, as well as an average DSC of 75.95% and an average HD of 8.36 mm on the cross-center prostate dataset. Compared to the UNet baseline, the proposed method achieved an improvement of 10.65% in average DSC and a reduction of 5.23 mm in average HD on the abdominal CT-MRI dataset, as well as 19.36% higher average DSC and 3.39 mm lower average HD on the cross-center prostate dataset. Ablation studies confirmed the effectiveness of each component of the proposed RandDGmethod. The proposed RandDG method effectively addresses the critical challenges of texture bias and limited style diversity in single-source DG for medical image segmentation through frequency-aware dual-space randomization framework. This method shows promise for practical deployment in clinical settings, where target domain data may not always beavailable.

Stochastic Gradient Descent (SGD) is widely used to train a machine learning model over large datasets, yet its slow convergence rate can be a bottleneck. As a remarkable family of variance reduction techniques, memory algorithms such as SAG and SAGA have been proposed to accelerate the convergence rate of SGD. However, these algorithms need to store per training data point corrections in memory. The unlimited space usage feature is impractical for modern large-scale applications, especially over data points that arrive over time (referred to as streaming data in this paper). To overcome this weakness, this paper investigates the methods that bound the space usage in the state-of-the-art family of variance-reduced stochastic gradient descent over streaming data, and presents CHEAPS2AGA. At each step of updating the model, the key idea of CHEAPS2AGA is always reserving N random data points as samples, while re-using information about past stochastic gradients across all the observed data points with limited space usage. In addition, training an accurate model over streaming data requires the algorithm to be time-efficient. To accelerate the model training phase, CHEAPS2AGA embraces a lock-free data structure to insert new data points and remove unused data points in parallel, and updates the model parameters without using any locking. We conduct comprehensive experiments to compare CHEAPS2AGA to prior related algorithms suited for streaming data. The experimental results demonstrate the practical competitiveness of CHEAPS2AGA in terms of scalability and accuracy.

Passive sensing applications are limited by their inability to determine who is using a device, a critical concern in child mobile device use research, where devices are often shared between siblings or between a child and their parent. Our previous work leveraged behavioral biometrics to identify a target child user; however, it is unknown what type of training data is necessary for optimal model performance. This study evaluated model performance across different characteristics of training data. Thirty-six children (11.3 ± 0.9 years, 56% female) self-selected a video or a game on iPads for 10 min while laying and for another 5 min while sitting. The SensorLog application captured iPad accelerometer and gyroscope data while the child interacted with the device. Machine learning algorithms including Neural Network (NN), Random Forest (RF), k-Nearest Neighbors (k-NN), and SwipeFormer were applied to determine the most important aspects of training data to optimize model performance. The aspects of training data evaluated included (1) varying the length (i.e., seconds of training data), (2) varying the user position (i.e., sitting, laying), and (3) varying the time proximity between training and testing data. F1 score was used to evaluate model performance. The SwipeFormer F1 scores were lowest when the training data was further from the test data (0 when training data was 11 min away from test data) and highest when training data was close to test data (0.91 when training data was the minute preceding test data). The SwipeFormer F1 scores were highest when predicting the user laying from laying (0.97) and sitting from sitting (0.94), and lowest when predicting the user sitting from laying (0) and laying from sitting (0). The length of training data had little impact on performance, with a SwipeFormer F1 score of 0.91 when training on one minute of data and a SwipeFormer F1 score of 0.94 when training on twelve minutes of data. Because researchers would likely be predicting users at different timepoints than their training data, research should focus on improving model performance for identifying users independent of time proximity for training and test data.