Research on positive and negative sample sampling strategies in contrastive learning

The aim of the study was to inspect the acoustic properties and sleep characteristics of a preapneic snoring sound. The feasibility of forecasting upcoming respiratory events by snoring sound was also investigated. Participants with habitual snoring or a heavy breathing sound during sleep were recruited consecutively. Polysomnography was conducted, and snoring-related breathing sound was recorded simultaneously. Acoustic features and sleep features were extracted from 30-second samples, and a machine learning algorithm was used to establish 2 prediction models. A total of 74 eligible participants were included. Model 1, tested by 5-fold cross-validation, achieved an accuracy of 0.92 and an area under the curve of 0.94 for respiratory event prediction. Model 2, with acoustic features and sleep information tested by Leave-One-Out cross-validation, had an accuracy of 0.78 and an area under the curve of 0.80. Sleep position was found to be the most important among all sleep features contributing to the performance of the 2 models. Preapneic sound presented unique acoustic characteristics, and snoring-related breathing sound could be deployed as a real-time apneic event predictor. The models, combined with sleep information, serve as a promising tool for an early warning system to forecast apneic events. Wang B, Yi X, Gao J, etal. Real-time prediction of upcoming respiratory events via machine learning using snoring sound signal. J Clin Sleep Med. 2021;17(9):1777-1784.

BackgroundCytokines act by binding to specific receptors in the plasma membrane of target cells. Knowledge of cytokine–receptor interaction (CRI) is very important for understanding the pathogenesis of various human diseases—notably autoimmune, inflammatory and infectious diseases—and identifying potential therapeutic targets. Recently, machine learning algorithms have been used to predict CRIs. “Gold Standard” negative datasets are still lacking and strong biases in negative datasets can significantly affect the training of learning algorithms and their evaluation. To mitigate the unrepresentativeness and bias inherent in the negative sample selection (non-interacting proteins), we propose a clustering-based approach for representative negative sample selection.ResultsWe used deep autoencoders to investigate the effect of different sampling approaches for non-interacting pairs on the training and the performance of machine learning classifiers. By using the anomaly detection capabilities of deep autoencoders we deduced the effects of different categories of negative samples on the training of learning algorithms. Random sampling for selecting non-interacting pairs results in either over- or under-representation of hard or easy to classify instances. When K-means based sampling of negative datasets is applied to mitigate the inadequacies of random sampling, random forest (RF) together with the combined feature set of atomic composition, physicochemical-2grams and two different representations of evolutionary information performs best. Average model performances based on leave-one-out cross validation (loocv) over ten different negative sample sets that each model was trained with, show that RF models significantly outperform the previous best CRI predictor in terms of accuracy (+ 5.1%), specificity (+ 13%), mcc (+ 0.1) and g-means value (+ 5.1). Evaluations using tenfold cv and training/testing splits confirm the competitive performance.ConclusionsA comparative analysis was performed to assess the effect of three different sampling methods (random, K-means and uniform sampling) on the training of learning algorithms using different evaluation methods. Models trained on K-means sampled datasets generally show a significantly improved performance compared to those trained on random selections—with RF seemingly benefiting most in our particular setting. Our findings on the sampling are highly relevant and apply to many applications of supervised learning approaches in bioinformatics.

Live HIV rapid test kits (RTKs) are usually packaged with only a single test buffer vial. A major challenge with HIV testing is the problem of lack of the RTK buffer vial to complete the number of tests in an RTK pack due to misuse, misplacement or contamination of buffer with clients' blood during HIV testing. This has led to the use of sterile normal saline and tap water as substitute liquids in place of the buffer vial. This study compared the use of the commercially provided buffer vial and these substitute liquids in HIV rapid testing while using the commercially provided buffer as gold standard. One hundred (100) HIV positive samples from patients attending antiretroviral therapy (ART) clinic and 100 HIV negative blood samples from blood donors placed in EDTA were tested with Determine RTK using the commercially provided buffer and substitute liquids (Sterile normal saline and tap water). The Determine RTK was also tested using the commercially provided buffer, normal saline and tap water without addition of blood samples. Results of the tests with substitute liquids were compared with those obtained with that of RTK's commercially provided buffer. With HIV positive and negative blood samples, the expected test line and control line or only control line (for negative samples) were observed with the commercially provided buffer and sterile normal saline. When tested without blood samples using the RTK's buffer and normal saline, the control line was observed but no test line for all 100 tests done. Invalid test results were recorded in 48% of the negative samples and 40% of test with no blood added when with tap water was used. This study also shows varied colour intensity for the buffer vial and substitute liquids used. For samples tested with sterile normal saline, 52% of the tested samples showed stronger colour intensity in test line that control line, 37% had equal colour intensity while 11% had higher colour intensity in control than test line. Seventy seven (77%) of Samples tested with tap water had the colour intensity greater in test line than control line while 23% had equal colour intensity in both test and control line. With sterile normal saline used with HIV positive and negative blood samples, a 100% clear background like that of the gold standard was obtained. However, tap water when used as a substitute buffer to run all 100 HIV positive samples had an unclear background. An unclear background was also observed in 100% and 28% of the test ran with tap water only (without blood samples) and with HIV Negative samples respectively.

The Covid-19 pandemic has not ended yet. Therefore, laboratory tests are very important for determining the diagnosis of Covid-19. At this case, PCR is the gold standard in laboratory tests. Therefore, the accuracy of the PCR assay should be a major concern. This study aims to evaluate the validation results of several Covid-19 PCR reagent kits. A total of 10 positive and negative clinical samples were used as test samples. DNA extraction was performed using a commercial reagent kit. PCR examination was carried out using four different blinded commercial reagent kits. The results of the examination were compared in each sample. The results showed that Reagents kit A, C, and D gave the same results for each sample, namely five positive and five negative samples. Reagent kit B showed different results, namely eight positive samples and two negative samples. A total of 3 (33.3%) of the 10 sample examinations indicated inaccurate results for Reagent kit B. The Reagent kit B test results using two different boxes showed different results with seven positive results by box 1, while eight positive results by another box. In conclusion, 1 out of 4 tested Covid-19 PCR kit reagents gave inaccurate results. Therefore, in covid-19 testing laboratories, it is recommended to choose a PCR reagent kit that has been approved by a trusted organization and perform reagent validation tests.

Few-shot relation extraction aims to identify and classify specific semantic relations between entities from text with a small number of annotated examples. Recent studies have shown that innovative model designs and learning strategies can significantly enhance the model’s generalization ability and performance, even with limited training samples. However, the samples used for training face the dual challenges of scarce annotated data and semantic ambiguity. Given the limitations of the existing SaCon framework in negative sample generation and discrimination efficiency, we propose a dynamic adversarial negative sample enhancement strategy. This strategy introduces adversarial perturbations before encoding the pre-trained language model by constructing a multi-granularity semantic space. Specifically, we first design an entity permutation mechanism to randomly exchange the subject/object entities of sentences in a small batch to generate negative sample clusters with similar semantics but misplaced relations, then, we integrate a multi-view contrastive learning framework to embed adversarial samples into the feature space topology optimization process.To strengthen boundary-sensitive features, an adaptive margin ranking loss function is proposed to dynamically adjust the representation distance constraints of positive and negative samples, forcing the model to capture the deep semantic invariance of relational predicates under limited samples. This method aims to optimize the traditional negative sample random sampling paradigm, actively explore the semantic space through an adversarial generation mechanism, and construct “difficult samples” with minimal semantic deviation through gradient back propagation, thereby improving the model’s ability to parse implicit relational patterns. The experiment result shows that this framework effectively alleviates the risk of overfitting in small sample scenarios by decoupling relational semantics and surface syntactic features, and its dynamic loss design provides mathematical guarantees for orthogonal separation of feature space. Our code is available online at https://github.com/hhy-test/ESCR.

Graph contrastive learning has emerged as a powerful unsupervised graph representation learning tool. The key to the success of graph contrastive learning is to acquire high-quality positive and negative samples as contrasting pairs to learn the underlying structural semantics of the input graph. Recent works usually sample negative samples from the same training batch with the positive samples or from an external irrelevant graph. However, a significant limitation lies in such strategies: the unavoidable problem of sampling false negative samples. In this paper, we propose a novel method to utilize Counterfactual mechanism to generate artificial hard negative samples for Graph Contrastive learning, namely CGC. We utilize a counterfactual mechanism to produce hard negative samples, ensuring that the generated samples are similar but have labels that differ from the positive sample. The proposed method achieves satisfying results on several datasets. It outperforms some traditional unsupervised graph learning methods and some SOTA graph contrastive learning methods. We also conducted some supplementary experiments to illustrate the proposed method, including the performances of CGC with different hard negative samples and evaluations for hard negative samples generated with different similarity measurements. The implementation code is available online to ease reproducibility1.

Collaborative filtering (CF) is widely used to learn informative latent representations of users and items from observed interactions. Existing CF-based methods commonly adopt negative sampling to discriminate different items. That is, observed user-item pairs are treated as positive instances; unobserved pairs are considered as negative instances and are sampled under a defined distribution for training. Training with negative sampling on large datasets is computationally expensive. Further, negative items should be carefully sampled under the defined distribution, in order to avoid selecting an observed positive item in the training dataset. Unavoidably, some negative items sampled from the training dataset could be positive in the test set. Recently, self-supervised learning (SSL) , has emerged as a powerful tool to learn a model without negative samples. In this paper, we propose a self-supervised collaborative filtering framework (SelfCF) , that is specially designed for recommender scenario with implicit feedback. The proposed SelfCF framework simplifies Siamese networks and can be easily applied to existing deep-learning based CF models, which we refer to as backbone networks. The main idea of SelfCF is to augment the latent embeddings generated by backbone networks instead of the raw input of user/item ids. We propose and study three embedding perturbation techniques that can be applied to different types of backbone networks including both traditional CF models and graph-based models. The framework enables learning informative representations of users and items without negative samples, and is agnostic to the encapsulated backbones. We conduct experimental comparisons on four datasets, one self-supervised framework, and eight baselines to show that our framework may achieve even better recommendation accuracy than the encapsulated supervised counterpart with a 2×–4× faster training speed. The results also demonstrate that SelfCF can boost up the accuracy of a self-supervised framework BUIR by 17.79% on average and shows competitive performance with baselines.

Object tracking is an essential and challenging sub-domain in the field of computer vision owing to its wide range of applications and complexities of real-life situations. It has been studied extensively over the last decade, leading to the proposal of several tracking frameworks and approaches. Recently, the introduction of reinforcement learning and the Actor-Critic framework has effectively improved the tracking speed of deep learning trackers. However, most existing deep reinforcement learning trackers experience a slight performance degradation mainly owing to the drift issues. Drifts pose a threat to the tracking performance, which may lead to losing the tracked target. Herein, we propose a drift-proof tracker with deep reinforcement learning that aims to improve the tracking performance by counteracting drifts while maintaining its real-time advantage. We utilize a reward function with the Distance-IoU (DIoU) metric to guide the reinforcement learning to alleviate the drifts caused by the trained model. Furthermore, double negative samples (hard negative and drift samples) are constructed in tracking for network initialization, which is followed by calculating the loss by a small error-friendly loss function. Therefore, our tracker can better discriminate between the positive and negative samples and correct the predicted bounding boxes when the drift occurs. Meanwhile, a generative adversarial network is adopted for positive sample augmentation. Extensive experimental results on multiple popular benchmarks show that our algorithm effectively reduces the occurrences of drift and boosts the tracking performance, compared to those of other state-of-the-art trackers.

Behavior sequences are generated by a series of spatio-temporal interactions and have a high-dimensional nonlinear manifold structure. Therefore, it is difficult to learn 3D behavior representations without relying on supervised signals. To this end, self-supervised learning methods can be used to explore the rich information contained in the data itself. Context-context contrastive self-supervised methods construct the manifold embedded in Euclidean space by learning the distance relationship between data, and find the geometric distribution of data. However, traditional Euclidean space is difficult to express context joint features. In order to obtain an effective global representation from the relationship between data under unlabeled conditions, this paper adopts contrastive learning to compare global feature, and proposes a self-supervised learning method based on hyperbolic embedding to mine the nonlinear relationship of behavior trajectories. This method adopts the framework of discarding negative samples, which overcomes the shortcomings of the paradigm based on positive and negative samples that pull similar data away in the feature space. Meanwhile, the output of the network is embedded in a hyperbolic space, and a multi-layer perceptron is added to convert the entire module into a homotopic mapping by using the geometric properties of operations in the hyperbolic space, so as to obtain homotopy invariant knowledge. The proposed method combines the geometric properties of hyperbolic manifolds and the equivariance of homotopy groups to promote better supervised signals for the network, which improves the performance of unsupervised learning.

Self-supervised learning methods (SSL) have achieved significant success via maximizing the mutual information between two augmented views, where cropping is a popular augmentation technique. Cropped regions are widely used to construct positive pairs, while the remained regions after cropping have rarely been explored in existing methods, although they together constitute the same image instance and both contribute to the description of the category. In this paper, we make the first attempt to demonstrate the importance of both regions in cropping from a complete perspective and the effectiveness of using both regions via designing a simple yet effective pretext task called Region Contrastive Learning (RegionCL). Technically, to construct the two kinds of regions, we randomly crop a region (called the paste view) from each input image with the same size and swap them between different images to compose new images together with the remained regions (called the canvas view). Then, instead of taking the new images as a whole for positive or negative samples, contrastive pairs are efficiently constructed from the regional perspective based on the following simple criteria, i.e., each view is (1) positive with views augmented from the same original image and (2) negative with views augmented from other images. With minor modifications to popular SSL methods, RegionCL exploits those abundant pairs and helps the model distinguish the regions features from both canvas and paste views, therefore learning better visual representations. Experiments on ImageNet, MS COCO, and Cityscapes demonstrate that RegionCL improves MoCov2, DenseCL, and SimSiam by large margins and achieves state-of-the-art performance on classification, detection, and segmentation tasks. The code is publicly available at https://github.com/Annbless/RegionCL .

Antigen-capture enzyme-linked immunosorbent assay (ACE) testing of skin samples obtained by ear-notching is commonly used to detect animals persistently infected (PI) with bovine viral diarrhea virus (BVDV). At times practitioners may not have access to phosphate buffered saline (PBS) solution for specimen storage as suggested by the ACE kit manufacturer. The first objective of this study was to observe the effects of storing ear-notch samples in varying amounts of PBS on ACE results. The second objective was to observe the effects of delayed time before the ear-notch sample was placed in PBS on the ACE results. Using virus isolation and immunohistochemistry, nine PI-positive and eight PI-negative calves were identified to obtain ear notch biopsies for the study. The amount of PBS used to store both positive and negative ear-notch samples was varied (1, 2, 4, 6, 8 and 10 mL) in the first trial. In the second trial, the addition of PBS to both positive and negative samples was delayed post-collection; phosphate buffered saline was added to ear-notch samples at 0, 1, 2, 4, 8, 12 and 24 hours post-collection. Following sampling and application of treatments, all samples were tested in duplicate for BVDV antigen following the guidelines of the ACE kit manufacturer. Although the S/P ratios were different among treatment groups in both studies (P<0.05), neither varying the amount of PBS added to the sample vial nor delaying the addition of PBS to the sample affected the outcome when testing positive or negative samples.

Timely and effective diagnosis plays a pivotal role in schistosomiasis control efforts. This study aims to assess the utility of combined Schistosoma haematobium soluble egg antigen (Sh SEA) and S. mansoni worm antigen (Sm SWA) in serological diagnosis of urogenital schistosomiasis. Admixtures containing 10 µg/mL of both Sm SEA and Sh SEA, as well as Sm SWA and Sh SWA, were employed to detect S. haematobium infection via an indirect enzyme-linked immunosorbent assay (ELISA) using sera and urine from microscopically confirmed positive samples from an endemic population, along with confirmed negative samples from both endemic (NE) and non-endemic (NNE) populations. The diagnostic performance of Schistosoma eggs and worm antigen mixtures varied depending on sample type and negative endemicity. The Sm SEA and Sh SEA mixtures performed poorly with sera and urine from the pair of positive vs negative endemic samples, as well as positive vs non-endemic samples, but excellently with positive vs negative endemic urine samples pair (sensitivity 91.67%; specificity 66.67%). Conversely, SWA mixtures showed superior performance, particularly with the positive vs negative non-endemic sera samples pair (sensitivity 93.75%; specificity 72.92%). Other SWA-based mixtures, except SWA admixture using urine in positive vs NE samples, exhibited acceptable performance. Antibody titers varied significantly, with higher titers generally observed in negative endemic samples for SWA mixtures and in negative non-endemic urine samples for SEA mixtures (p < 0.05). Combined antigens improve Schistosoma diagnostics: SEA admixtures suit endemic urine samples, while SWA admixtures aid non-endemic sera detection.

Deep one-class probability learning for end-to-end image classification.

PurposeCOVID-19 infection in immunodeficient individuals can result in chronically poor health, persistent or relapsing SARS-CoV-2 PCR positivity, and long-term infectious potential. While clinical trials have demonstrated promising outcomes using anti-SARS-CoV-2 medicines in immunocompetent hosts, their ability to achieve sustained viral clearance in immunodeficient patients remains unknown. We therefore aimed to study long-term virological outcomes in patients treated at our centre.MethodsWe followed up immunocompromised inpatients treated with casirivimab-imdevimab (Ronapreve) between September and December 2021, and immunocompromised patients who received sotrovimab, molnupiravir, nirmatrelvir/ritonavir (Paxlovid), or no treatment from December 2021 to March 2022. Nasopharyngeal swab and sputum samples were obtained either in hospital or in the community until sustained viral clearance, defined as 3 consecutive negative PCR samples, was achieved. Positive samples were sequenced and analysed for mutations of interest.ResultsWe observed sustained viral clearance in 71 of 103 patients, none of whom died. Of the 32/103 patients where sustained clearance was not confirmed, 6 died (between 2 and 34 days from treatment). Notably, we observed 25 cases of sputum positivity despite negative nasopharyngeal swab samples, as well as recurrence of SARS-CoV-2 positivity following a negative sample in 12 cases. Patients were then divided into those who cleared within 28 days and those with PCR positivity beyond 28 days. We noted lower B cell counts in the group with persistent PCR positivity (mean (SD) 0.06 (0.10) ×109/L vs 0.22 (0.28) ×109/L, p = 0.015) as well as lower IgA (median (IQR) 0.00 (0.00–0.15) g/L vs 0.40 (0.00–0.95) g/L, p = 0.001) and IgM (median (IQR) 0.05 (0.00–0.28) g/L vs 0.35 (0.10–1.10) g/L, p = 0.005). No differences were seen in CD4+ or CD8+ T cell counts. Antiviral treatment did not impact risk of persistent PCR positivity.ConclusionPersistent SARS-CoV-2 PCR positivity is common among immunodeficient individuals, especially those with antibody deficiencies, regardless of anti-viral treatment. Peripheral B cell count and serum IgA and IgM levels are predictors of viral persistence.

Contrastive learning (CL) is a learning strategy that has received widespread attention in the field of machine learning, which trains models by learning to distinguish between pairs of positive and negative samples. Unfortunately, the model struggles to extract key features and effectively differentiate between false negative samples and difficult negative samples when the data contain a large amount of noise. Based on this, a Twin Contrastive Learning based on Negative Sample Attention (TCL-NSA) method is proposed in this paper. In TCL-NSA, clustering of features is executed to resolve the importance of difficult negative samples, and different attention is assigned to negative samples, allowing the model to learn more discriminative features. Meanwhile, the shrinking residual sub-model is constructed to effectively pre-process the noisy signals so that the encoder can extract accurate representations of the data and make the clustering results more accurate. In addition, a projection head is designed to better map the extracted backbone network representation to the feature space of the contrast loss. Experimental validation is carried out on two rolling bearing datasets with average accuracy of 99.45%, and comparison experiments are conducted with other popular methods and show superior performance of TCL-NSA.