Improving Active Learning Performance through the Use of Data Augmentation

Limited experimental data remains a key challenge in applying machine learning to drug discovery, particularly for cancer-related targets. In this study, we present a data-efficient active meta-deep learning framework to predict mitogen-activated protein kinase 1 (MAPK1) inhibitors, which are promising candidates for cancer-related therapies. Our approach integrates active learning (AL) with a meta-model that combines four deep architectures: a convolutional neural network, an attention, a graph convolutional network, and a graph neural network-attention, trained on molecular descriptors and graph-based representations. These models generate four probability-based features that feed into an attention-based meta-learner, improving predictive performance by 5.12% in the area under the precision-recall curve (AUPRC) and 5.48% in the Matthews correlation coefficient (MCC) using only 10% of the training data. Among the AL sampling strategies evaluated, entropy sampling showed competitive performance in selecting informative molecules for model improvement. Overall, our framework achieves an AUPRC of 0.835 ± 0.017 and MCC of 0.817 ± 0.017, on par with a traditional training method despite using only 26.7% of the training data. Compared to a conventional random forest model trained on brute-force, a 100% full training set, our approach shows a 10.6% improvement in AUPRC and modest gains in MCC, confirming the effectiveness of the proposed framework. Under severe class imbalance, balanced accuracy steadily increased across AL iterations, reaching values greater than 0.85 at the final iteration for all uncertainty-driven strategies. Molecular docking confirmed successful prioritization of the top four predicted compounds. Evaluation on an external MAPK1 data set demonstrated generalizability, with our approach achieving an AUPRC of 0.818 and an MCC of 0.403, comparable to the independent test set. These results highlight the potential of combining intelligent data selection with deep learning architectures through the meta-model to accelerate predictive performance in data-scarce drug discovery. Scientific contribution: This study contributes a novel, data-efficient active meta-deep learning framework for predicting MAPK1 inhibitors, addressing the challenge of limited experimental data in a cancer-specific target. By integrating AL with a meta-model composed of four deep architectures, the approach significantly enhances the predictive performance using only a fraction of the training data. The framework achieves superior metrics compared to traditional training methods, highlighting its potential to accelerate drug discovery in data-scarce settings.

Deep neural network has been extensively applied to hyperspectral image (HSI) classification recently. However, its success is greatly attributed to numerous labeled samples, whose acquisition costs a large amount of time and money. In order to improve the classification performance while reducing the labeling cost, this article presents an active deep learning approach for HSI classification, which integrates both active learning and deep learning into a unified framework. First, we train a convolutional neural network (CNN) with a limited number of labeled pixels. Next, we actively select the most informative pixels from the candidate pool for labeling. Then, the CNN is fine-tuned with the new training set constructed by incorporating the newly labeled pixels. This step together with the previous step is iteratively conducted. Finally, Markov random field (MRF) is utilized to enforce class label smoothness to further boost the classification performance. Compared with the other state-of-the-art traditional and deep learning-based HSI classification methods, our proposed approach achieves better performance on three benchmark HSI data sets with significantly fewer labeled samples.

Active learning has recently attracted increasing attention in the task of person re-identification, due to its unique scalability that not only maximally reduces the annotation cost but also retains the satisfying performance. Although some preliminary active learning methods have been explored in scalable person re-identification task, they have the following two problems: 1) the inefficiency in the selection process of image pairs due to the huge search space, and 2) the ineffectiveness caused by ignoring the impact of unlabeled data in model training. Considering that, we propose a Multi-grained Active Semi-Supervised learning framework, named MASS, to address the scalable person re-identification problem existing in the practical scenarios. Specifically, we firstly design a cluster-scatter procedure to alleviate the inefficiency problem, which consists of two components: cluster step and scatter step. The cluster step shrinks the search space into individual small clusters by a coarse-grained clustering method, and the subsequent scatter step further mines the hard distinguished image pairs from unlabelled set to purify the learned clusters by a novel centrality-based adaptive purification strategy. Afterward, we introduce a customized purification loss for the purified clustering, which utilizes the complementary information in both labeled and unlabeled data to optimize the model for solving the ineffectiveness problem. The cluster-scatter procedure and the model optimization are performed in an iterative fashion to achieve the promising performance while greatly reducing the annotation cost. Extensive experimental results have demonstrated that MASS can even achieve a competitive performance with fully supervised methods in the case of extremely less annotation requirements.

Automatic analysis methods of electrocardiograms (ECGs) usually required large-scale annotated training data, but the annotation process is extremely time-consuming. While semi-supervised learning can leverage unlabeled data, its performance depends heavily on the quality of the initial labeled subset. Active learning has been used to identify the most informative samples for annotation, but conventional approaches face three critical limitations: (1) dependency on manual intervention for iterative query design, (2) prohibitive computational costs during sample selection, and (3) limited compatibility with semi-supervised learning frameworks. To address these limitations, we proposed an Unsupervised Active Feature-selective Semi-Supervised Learning (UAFSSL) framework for ECG analysis, including an unsupervised feature selection-based active learning module and a semi-supervised learning module. UAFSSL captures latent data distributions via unsupervised feature extraction, selects diverse and representative samples using pseudo-label clustering, and integrates seamlessly with semi-supervised learning to eliminate human intervention. We validated our algorithm on an ECG waveform segmentation task and an atrial fibrillation detection task. In the waveform segmentation task, our method improved the F1-score for P-wave delineation by 2.4% compared to random sampling, using only 5% of labeled samples. For the atrial fibrillation detection task, we evaluated our method on both the AFDB and a 24-hour dataset collected from 500 atrial fibrillation patients. Using only 200 labeled samples for model training, our method achieved AUC improvements of 2.5% and 2.2% over random sampling in five-fold cross validation. This is the first study to integrate unsupervised active learning with semi-supervised learning for automatic ECG analysis, offering a robust, automated solution to reduce annotation costs while enhancing clinical applicability.

To extract valuable information from satellite data for applications such as agriculture, geological research, and environmental monitoring, the classification of hyperspectral images is an essential task. Labeling each pixel in this process is time-consuming and requires financial resources. To this end, working with a small number of samples is very important. In order to provide high classification performances with a limited number of samples, this paper aims to enhance the performance with an active learning framework. The framework incorporates dimensionality reduction, an edge-preserving filter, and active learning steps. From this perspective, we investigated different edge-preserving filter methods to analyze the effects on performance. By combining edge-preserving filters with dimensionality reduction, the study presents a unique method that improves classification performance while maintaining image quality and reducing noise. The following five edge-preserving smoothing filters are evaluated: weighted least squares (WLS), Joint-Histogram weighted median filter (Joint WMF), fast global image smoother (FGS), bilateral filter (BF), and static/dynamic (SD). Our experiments demonstrate that compared to the reference research (CNN+AL+MRF), the proposed framework increased overall and average accuracies about 2-5% for Indian Pines, Pavia University, and Salinas datasets.

Recently, Time Series Remote Sensing Images (TSRSIs) have been proven to be a significant resource for land use/land cover (LULC) mapping. Deep learning methods perform well in managing and processing temporal dependencies and have shown remarkable advancements within this domain. Although deep learning methods have exhibited outstanding performance in classifying TSRSIs, they rely on enough labeled time series samples for effective training. Labeling data with a wide geographical range and a long time span is highly time-consuming and labor-intensive. Active learning (AL) is a promising method of selecting the most informative data for labeling to save human labeling efforts. It has been widely applied in the remote sensing community, except for the classification of TSRSIs. The main challenge of AL in TSRSI classification is dealing with the internal temporal dependencies within TSRSIs and evaluating the informativeness of unlabeled time series data. In this paper, we propose a data-driven active deep learning framework for TSRSI classification to address the problem of limited labeled time series samples. First, a temporal classifier for TSRSI classification tasks is designed. Next, we propose an effective active learning method to select informative time series samples for labeling, which considers representativeness and uncertainty. For representativeness, we use the K-shape method to cluster time series data. For uncertainty, we construct an auxiliary deep network to evaluate the uncertainty of unlabeled data. The features with rich temporal information in the classifier's middle-hidden layers will be fed into the auxiliary deep network. Then, we define a new loss function with the aim of improving the deep model's performance. Finally, the proposed method in this paper was verified on two TSRSI datasets. The results demonstrate a significant advantage of our method over other approaches to TSRSI. On the MUDS dataset, when the initial number of samples was 100 after our method selected and labeled 2000 samples, an accuracy improvement of 4.92% was achieved. On the DynamicEarthNet dataset, when the initial number of samples was 1000 after our method selected and labeled 2000 samples, an accuracy improvement of 7.81% was attained. On the PASTIS dataset, when the initial number of samples was 1000 after our method selected and labeled 2000 samples, an accuracy improvement of 4.89% was achieved. Our code is available in Data Availability Statement.

PURPOSE: To assess previous sample representation limitations and provide classification performance metrics to evaluate the accuracy of the Athlete Psychological Strain Questionnaire (APSQ) in triaging athletes to Step 2 of the Sport Mental Health Assessment Tool (SMHAT-1). METHODS: Prior to the Paris 2024 Olympic and Paralympic Games, Team USA athletes (n = 911, age: 27.3 ± 6.1 years, 52.1% female, 26.1% Paralympic) completed the SMHAT-1 as part of a mandatory pre-Games health screening. Athletes completed both the APSQ (Step 1) and all Step 2 questionnaires, regardless of their APSQ score. Step 2 questionnaires included the Patient Health Questionnaire-9 (PHQ-9), Generalized Anxiety Disorder-7 (GAD-7), Athlete Sleep Screening Questionnaire (ASSQ), CAGE-AID for drug use, Alcohol Use Disorders Identification Test-Concise (AUDIT-C), Brief Eating Disorder in Athletes Questionnaire (BEDA-Q), and Post-Traumatic Stress Disorder (PTSD). All questionnaires were scored and marked as “flag” or “no flag” according to established thresholds. Classification performance metrics were calculated and compared to previous literature. An exploratory factor analysis was conducted to investigate covariance between Step 2 questionnaires and explain observed false negative rates (FNRs). RESULTS: The prevalence of a flagged APSQ was 24.7%. FNRs were calculated as PHQ9 = 0% GAD7 = 7.7%, ASSQ = 52.9%, AUDIT-C = 61.2%, CAGE-AID = 37.5%, BEDA-Q = 63.16%, PTSD = 11.1%, congruent with previous findings. Exploratory factor analysis showed that GAD-7 and PHQ-9 were grouped under the first factor, while CAGE-AID was grouped under the second factor. The other questionnaires did not fit distinctly into either factor, suggesting the SMAHT is assessing a broad spectrum of mental health factors, which would be difficult for a single generic screening tool to triage adequately. CONCLUSIONS: While the SMHAT-1 remains an essential tool for athlete mental health screening, the high FNRs indicate limitations in using the APSQ as an initial triage instrument. The APSQ may be better suited as a Step 2 assessment. A bespoke Step 1 screening tool that captures the diverse spectrum of mental health concerns in athletes should be considered to enhance the tool’s effectiveness and potentially increase its adoption across all levels of sport. Supported by: This work was funded in part by a grant from the International Olympic Committee.

Active deep learning classification of hyperspectral images is considered in this paper. Deep learning has achieved success in many applications, but good-quality labeled samples are needed to construct a deep learning network. It is expensive getting good labeled samples in hyperspectral images for remote sensing applications. An active learning algorithm based on a weighted incremental dictionary learning is proposed for such applications. The proposed algorithm selects training samples that maximize two selection criteria, namely representative and uncertainty. This algorithm trains a deep network efficiently by actively selecting training samples at each iteration. The proposed algorithm is applied for the classification of hyperspectral images, and compared with other classification algorithms employing active learning. It is shown that the proposed algorithm is efficient and effective in classifying hyperspectral images.

Collection of training samples for remote sensing image classification is always time-consuming and expensive. In this context, active learning (AL) that aims at using limited training samples to achieve promising classification performances is developed. Recently, integration of spatial information into AL exhibits new potential for image classification. In this letter, an AL approach with two-stage spatial computation (AL-2SC) is proposed to improve the selection of training samples. The spatial features derived from remote sensing image and the probability outputs from the neighboring pixels are introduced in AL process. Moreover, we compare several AL approaches which take spatial information into account. In experiments, random sampling (RS) and four AL methods, including AL using breaking ties heuristic (BT), AL with spatial feature (AL-SF), AL with neighbouring responses (AL-NR), and AL-2SC, are considered. Three remote sensing datasets, including one hyperspectral and two multispectral images, are used to compare the performance of different methods. It is illustrated that, the utilization of spatial information is very important for the improvement of AL performance, and the proposed AL-2SC shows the most satisfactory result.

Active learning is an important technology to solve the lack of data in crack detection model training. However, the sampling strategies of most existing active learning methods for crack detection are based on the uncertainty or representation of the samples, which cannot effectively balance the exploitation and exploration of active learning. To solve this problem, this study proposes an active learning method for crack detection based on subset searching and weighted sampling. First, a new active learning framework is established to successively search subsets with large uncertainty from the candidate dataset, and select training samples with large diversity from the subsets to update the crack detection model. Second, to realize the active learning process, a subset searching method based on sample relative error is proposed to adaptively select subsets with large uncertainty, and a weighted sampling method based on flow-based deep generative network is introduced to select training samples with large diversity form the subsets. Third, a termination criterion for active learning directly based on the prediction accuracy of the trained model is proposed to adaptively determine the maximum number of iterations. Finally, the proposed method is tested using two open-source crack datasets. The experimental comparison results on the Bridge Crack Library dataset show that the proposed method has higher calculation efficiency and prediction accuracy in crack detection than the uncertainty-based and representation-based active learning methods. The test results on the DeepCrack dataset show that the crack detection model trained by the proposed method has good transferability on different datasets with multi-scale concrete cracks and scenes.

Active extreme learning machines for quad-polarimetric SAR imagery classification

Active learning is a useful technique for tasks for which unlabeled data is abundant but manual labeling is expensive. One example of such a task is semantic role labeling (SRL), which relies heavily on labels from trained linguistic experts. One challenge in applying active learning algorithms for SRL is that the complete knowledge of the SRL model is often unavailable, against the common assumption that active learning methods are aware of the details of the underlying models. In this paper, we present an active learning framework for black-box SRL models (i.e., models whose details are unknown). In lieu of a query strategy based on model details, we propose a neural query strategy model that embeds both language and semantic information to automatically learn the query strategy from predictions of an SRL model alone. Our experimental results demonstrate the effectiveness of both this new active learning framework and the neural query strategy model.

Annotating data is a time-consuming and costly task, but it is inherently required for supervised machine learning. Active Learning (AL) is an established method that minimizes human labeling effort by iteratively selecting the most informative unlabeled samples for expert annotation, thereby improving the overall classification performance. Even though AL has been known for decades [1], AL is still rarely used in real-world applications. As indicated in the two community web surveys among the NLP community about AL [2], [3], two main reasons continue to hold practitioners back from using AL: first, the complexity of setting AL up, and second, a lack of trust in its effectiveness. We hypothesize that both reasons share the same culprit: the large hyperparameter space of AL. This mostly unexplored hyperparameter space often leads to misleading and irreproducible glsAL experiment results. In this study, we first compiled a large hyperparameter grid of over 4.6 million hyperparameter combinations, second, recorded the performance of all combinations in the so-far biggest conducted AL study, and third, analyzed the impact of each hyperparameter in the experiment results. Rather than merely reporting correlations, we explicitly focus on distilling these results into practitioner-oriented rulesof-thumb for designing AL experiments under realistic resource constraints. In the end, we give recommendations about the influence of each hyperparameter, demonstrate the surprising influence of the concrete AL strategy implementation, and outline an experimental study design for reproducible AL experiments with minimal computational effort, thus contributing to more reproducible and trustworthy AL research in the future.

Active learning is one form of supervised machine learning. In supervised learning, a set of labeled samples is passed to a learning algorithm for training a classifier. However, labeling large amounts of training samples can be costly and error-prone. Active learning deals with the development of algorithms that interactively select a subset of the available unlabeled samples for labeling, and aims at minimizing the labeling effort while maintaining classification performance. The key challenge for the development of so-called active learning strategies is the balance between exploitation and exploration: On the one hand, the estimated decision boundary needs to be refined in feature space regions where it has already been established, while, on the other hand, the feature space needs to be scanned carefully for unexpected class distributions. In this thesis, two approaches to active learning are presented that consider these two aspects in a novel way. In order to lay the foundations for the first one, it is proposed to express the uncertainty in class prediction of a classifier at a test point in terms of a second-order distribution. The mean of this distribution corresponds to the common estimate of the posterior class probabilities and thus is related to the distance of the test point to the decision boundary, whereas the spread of the distribution indicates the degree of exploration in the corresponding region of feature space. This allows for the evaluation of the utility of labeling a yet unlabeled point with respect to classifier improvement in a principled way and leads to a completely novel approach to active learning. The proposed strategy is then implemented and evaluated based on kernel density classification. The generic active learning strategy can be combined with any other classifier, but it performs best if the derived second-order distributions are sufficiently good approximations to the sampling distribution. Although second-order distributions for random forests are derived in this thesis, they do not approximate sufficiently well the sampling distribution and mainly allow only for the relative comparison of prediction uncertainty between test points. In order to combine the state of the art classification performance of random forests with the principal ideas of the first active learning approach, a related second approach for random forests is derived. It is, in addition, tailored to the demands in industrial optical inspection: bag-wise labeling with weak labels and strongly imbalanced classes. Moreover, an outlier detection scheme based on random forests is derived that is used by the proposed active learning algorithm. Finally, a new computational scheme for Gaussian process classification is presented. It is compared to two standard methods in geostatistics, both with respect to theoretical consistency and practical performance. The method evolved as a by-product when considering using Gaussian process models for active learning.

Dual-metal-site catalysts (DMSCs) are increasingly important catalysts in the field of electrochemical carbon dioxide reduction reaction (CO2RR) in recent years. However, rapid screening of suitable metal combinations of DMSCs remains a huge challenge. Herein, we constructed an active learning (AL) framework to study CO2RR to HCOOH. This AL framework turned out a success in the accurate prediction of 282 DMSCs for CO2RR through interactive learning between users and machine learning (ML) models. Among the 42 DMSCs calculated in three iteration loops of AL, 29 DMSCs were obtained, where the screening success rate was as high as 70%. Furthermore, we found five experimentally unexplored DMSCs that exhibited better CO2RR activity and selectivity than pure Bi. Low prediction errors on other DMSCs show that the AL model possessed outstanding universality. The results prove the excellent potential of the AL method and provide guidance on the design of high-performance electrocatalysts for CO2RR.