Few-Shot Relation Extraction via Semantically Related Negative Samples

The aim of the study was to inspect the acoustic properties and sleep characteristics of a preapneic snoring sound. The feasibility of forecasting upcoming respiratory events by snoring sound was also investigated. Participants with habitual snoring or a heavy breathing sound during sleep were recruited consecutively. Polysomnography was conducted, and snoring-related breathing sound was recorded simultaneously. Acoustic features and sleep features were extracted from 30-second samples, and a machine learning algorithm was used to establish 2 prediction models. A total of 74 eligible participants were included. Model 1, tested by 5-fold cross-validation, achieved an accuracy of 0.92 and an area under the curve of 0.94 for respiratory event prediction. Model 2, with acoustic features and sleep information tested by Leave-One-Out cross-validation, had an accuracy of 0.78 and an area under the curve of 0.80. Sleep position was found to be the most important among all sleep features contributing to the performance of the 2 models. Preapneic sound presented unique acoustic characteristics, and snoring-related breathing sound could be deployed as a real-time apneic event predictor. The models, combined with sleep information, serve as a promising tool for an early warning system to forecast apneic events. Wang B, Yi X, Gao J, etal. Real-time prediction of upcoming respiratory events via machine learning using snoring sound signal. J Clin Sleep Med. 2021;17(9):1777-1784.

Sentence representation learning benefits from data augmentation strategies to improve model performance and generalization, yet existing approaches often encounter issues such as semantic inconsistencies and feature suppression.To address these limitations, we propose a method for generating Syntactically Aligned Negative (SAN) samples through a semantic importance-aware Masked Language Model (MLM) approach.Our method quantifies semantic contributions of individual words to produce negative samples that have substantial textual overlap with the original sentences while conveying different meanings.We further introduce Hierarchical-InfoNCE (HiNCE), a novel contrastive learning objective employing differential temperature weighting to optimize the utilization of both in-batch and syntactically aligned negative samples.Extensive evaluations across seven semantic textual similarity benchmarks demonstrate consistent improvements over state-of-the-art models 1 .

Federated knowledge graph reasoning (FedKGR) aims to perform reasoning over different clients while protecting data privacy, drawing increasing attention to its high practical value. Previous works primarily focus on data heterogeneity, ignoring challenges from limited data scale and primitive negative sample strategies, i.e., random entity replacement, which yield low-quality negatives and zero loss issues. Meanwhile, generative adversarial networks (GANs) are widely used in different fields to generate high-quality negative samples, but no work has been developed for FedKGR. To this end, we propose a plug-and-play <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">E</u>ntity-aware <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A</u>dversarial <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</u>egative sampling strategy for FedKGR, termed FedEAN. Specifically, we are the first to adopt GANs to generate high-quality negative samples in different clients. It takes the target triplet in each batch as input and outputs high-quality negative samples, which guaranteed by the joint training of the generator and discriminator. Moreover, we design an entity-aware adaptive negative sampling mechanism based on the similarity of entity representations before and after server aggregation, which can persevere the entity global consistency across clients during training. Extensive experiments demonstrate that FedEAN excels with various FedKGR backbones, demonstrating its ability to construct high-quality negative samples and address the zero-loss issue.

Contrastive learning (CL) is a learning strategy that has received widespread attention in the field of machine learning, which trains models by learning to distinguish between pairs of positive and negative samples. Unfortunately, the model struggles to extract key features and effectively differentiate between false negative samples and difficult negative samples when the data contain a large amount of noise. Based on this, a Twin Contrastive Learning based on Negative Sample Attention (TCL-NSA) method is proposed in this paper. In TCL-NSA, clustering of features is executed to resolve the importance of difficult negative samples, and different attention is assigned to negative samples, allowing the model to learn more discriminative features. Meanwhile, the shrinking residual sub-model is constructed to effectively pre-process the noisy signals so that the encoder can extract accurate representations of the data and make the clustering results more accurate. In addition, a projection head is designed to better map the extracted backbone network representation to the feature space of the contrast loss. Experimental validation is carried out on two rolling bearing datasets with average accuracy of 99.45%, and comparison experiments are conducted with other popular methods and show superior performance of TCL-NSA.

Multi-modal fake news detection is a technique designed to identify and classify fake news by integrating information from multiple modalities. However, existing multi-modal fake news detection models have significant limitations in capturing structural information when processing social context. The core issue stems from the reliance on simple linear aggregation or static attention mechanisms in existing graph detection methods, which are inadequate for effectively capturing complex long-distance propagation relationships and multi-layered social network structures. Furthermore, existing multi-modal detection approaches are limited by the feature representations within the respective semantic spaces of each modality. The semantic gaps between modalities lead to misalignment during information fusion, making it difficult to fully achieve modality complementarity. To address these issues, we propose GINMCL, a graph isomorphism network-driven modality enhancement and cross-modal consistency learning method for multi-modal fake news detection. This method builds on the extraction of text and image features by incorporating graph isomorphism networks (GIN) based on the Weisfeiler-Lehman (WL) injective aggregation mechanism to effectively capture both local dependencies and global relationships within social graphs. Modality consistency learning aligns text, image, and social graph information into a shared latent semantic space, enhancing modality correlations. Additionally, to overcome the limitations of traditional methods in modality fusion strategies, we leverage a hard negative contrastive learning mechanism, which softens the penalty on negative samples and optimizes contrastive loss, further improving the accuracy and robustness of the model. We conducted systematic evaluations of GINMCL on the Pheme and Weibo datasets, and experimental results demonstrate that GINMCL outperforms existing methods across all metrics, achieving state-of-the-art performance.

Forward-only learning algorithms have recently gained attention as alternatives to gradient backpropagation, replacing the backward step of this latter solver with an additional contrastive forward pass. Among these approaches, the so-called Forward-Forward Algorithm (FFA) has been shown to achieve competitive levels of performance in terms of generalization and complexity. Networks trained using FFA learn to contrastively maximize a layer-wise defined goodness score when presented with real data (denoted as positive samples) and to minimize it when processing synthetic data (corr. negative samples). However, this algorithm still faces weaknesses that negatively affect the model accuracy and training stability, primarily due to a gradient imbalance between positive and negative samples. To overcome this issue, in this work we propose a novel implementation of the FFA algorithm, denoted as Polar-FFA, which extends the original formulation by introducing a neural division (polarization) between positive and negative instances. Neurons in each of these groups aim to maximize their goodness when presented with their respective data type, thereby creating a symmetric gradient behavior. To empirically gauge the improved learning capabilities of our proposed Polar-FFA, we perform several systematic experiments using different activation and goodness functions over image classification datasets. Our results demonstrate that Polar-FFA outperforms FFA in terms of accuracy and convergence speed. Furthermore, its lower reliance on hyperparameters reduces the need for hyperparameter tuning to guarantee optimal generalization capabilities, thereby allowing for a broader range of neural network configurations.

This paper presents a method for semantic class disambiguation for all words. Unlike the ordinary word sense disambiguation, a set of semantic classes or coarse grained senses is defined as a common sense inventory, then universal classifiers to select an appropriate semantic class of a target word in a given context, which can be applicable to all words, are trained by supervised learning. In the previous method, imbalance of number of positive and negative samples in training data causes disambiguation errors. On the other hand, our new method can train the classifiers from well balanced training data. Results of the experiment showed that our method outperformed the previous method by 1.43% accuracy.

For the detection of wood surface defects, a convolutional neural network has a low detection efficiency and insufficient generalization ability, so it does not meet the requirements of online detection. Aiming to solve the above problems, the YOLOv3 baseline model, which has the advantage of multi-objective dynamic detection, was improved and applied to the online detection of wood surface defects. To solve the problem of the poor generalization ability of the network, GridMask was used to enhance the data and improve the robustness of the network. In order to solve the problem of the considerable amount of network parameter calculations and insufficient real-time performance, the residual block of the backbone network was changed to a Ghost block structure to achieve a lightweight model. Finally, the confidence loss function of the network was improved to reduce the influence of simple samples and negative samples on model convergence. The experimental results showed that, compared with the original network, the improved algorithm increased the mean average precision by 5.73% and the detection speed was increased to 28 frames per second (an increase of 11), which met the requirements for real-time industrial detection.

Considering the real-time and high-precision requirements of image processing in X-ray baggage security screening; and problems such as the inflexibility and complex computation of anchor-based object detection, this paper introduces an anchor-free mode convolutional neural network object detection method for detecting weapons (knives and handguns) in X-ray baggage security images. The advantage of the anchor-free method over the anchor-based method is that the size of the anchor box does not have to be set, and the generalization ability is strong; the absence of the anchor box reduces the number of computations, and solves the problem of unbalanced positive and negative samples in the anchor-based method. To fully evaluate the effectiveness of the anchor-free method for X-ray baggage screening image detection, a large number of images containing knives and handguns were collected and annotated in the early stages of this work to produce a dataset that could be used for training. Six mainstream anchor-free methods (CornerNet, CenterNet, CornerNet-Lite, ExtremeNet, Objects as Points and You Only Look Once(YOLOx)) are introduced. For experimental integrity, this paper adds an anchor-based comparison experiment, using Faster-RCNN, YOLOv3 and YOLOv5 to perform the same work. The experimental results show that the YOLOx, Objects as Points and ExtremeNet anchor-free methods used in this paper have excellent performance in weapon detection in X-ray baggage security images. Among them, the mean average precision (mAP) of YOLOx combined with the CSPDarknet53 network reached 0.905, and the mAP of ExtremeNet combined with the Hourglass-104 network reached 0.900; the performance of the Objects as Points method was also good. All these methods performed better than the anchor-based methods compared in this paper. Therefore, we believe that the anchor-free method has a practical effect in weapon detection for X-ray luggage images.

Behavior sequences are generated by a series of spatio-temporal interactions and have a high-dimensional nonlinear manifold structure. Therefore, it is difficult to learn 3D behavior representations without relying on supervised signals. To this end, self-supervised learning methods can be used to explore the rich information contained in the data itself. Context-context contrastive self-supervised methods construct the manifold embedded in Euclidean space by learning the distance relationship between data, and find the geometric distribution of data. However, traditional Euclidean space is difficult to express context joint features. In order to obtain an effective global representation from the relationship between data under unlabeled conditions, this paper adopts contrastive learning to compare global feature, and proposes a self-supervised learning method based on hyperbolic embedding to mine the nonlinear relationship of behavior trajectories. This method adopts the framework of discarding negative samples, which overcomes the shortcomings of the paradigm based on positive and negative samples that pull similar data away in the feature space. Meanwhile, the output of the network is embedded in a hyperbolic space, and a multi-layer perceptron is added to convert the entire module into a homotopic mapping by using the geometric properties of operations in the hyperbolic space, so as to obtain homotopy invariant knowledge. The proposed method combines the geometric properties of hyperbolic manifolds and the equivariance of homotopy groups to promote better supervised signals for the network, which improves the performance of unsupervised learning.

Over the years, various document‐clustering techniques were developed to group the textual data. The performance of document clustering systems heavily relies on the optimal use of text representations. Vector space model is an extensively used technique by existing clustering algorithms to present the text in a structured form. However, such representations suffer from a lack of semantic associations, high dimensionality, and sparsity. In order to enrich the document representation by retaining the semantic and morphological associations, this paper introduced a word cluster‐based modified term frequency‐inverse document frequency (WC_MTI) model, in which semantically associated word embeddings from the word2vec are supplemented with morphological information using kernel principal component analysis. In addition, to address high dimensionality and sparsity issues and improve the clustering, we use a self‐training technique that learns discriminative features using the WC_MTI model and autoencoder (AE) and then updates the encoder network weights using assignments from a clustering algorithm as supervision. The proposed model organizes documents into topically compatible clusters by maintaining the semantic and morphological similarity between terms using the skip‐gram with negative sampling (SGNS) and low‐dimensional vector representations. We evaluate the proposed approach against the existing text representation methods. Experimental findings suggest that the proposed approach enhanced the average accuracy by 89.62%.

Calculating semantic similarity is paramount in medical information processing, and it aims to assess the similarity of medical professional terminologies within medical databases. Natural language models based on Bidirectional Encoder Representations from Transformers(BERT) offer a novel approach to semantic representation for semantic similarity calculations. However, due to the specificity of medical terminologies, these models often struggle with accurately representing semantically similar medical terms, leading to inaccuracies in term representation and consequently affecting the accuracy of similarity calculations. To address this challenge, this study employs Chat Generative Pre-trained Transformer (ChatGPT) and contrastive loss during the training phase to adapt BERT, enhancing its semantic representation capabilities and improving the accuracy of similarity calculations. Specifically, we leverage ChatGPT-3.5 to generate semantically similar texts for medical professional terminologies, incorporating them as pseudo-labels into the model training process. Subsequently, contrastive loss is utilized to minimize the distance between relevant samples and maximize the distance between irrelevant samples, thereby enhancing the performance of medical similarity models, especially with limited training samples. Experimental validation is conducted on the open Electronic Health Record (OpenEHR) dataset, randomly divided into four groups to verify the effectiveness of the proposed methodology.

The PAP900 dataset centers on the semantic relationship between affective words in Portuguese. It contains 900 word pairs, each annotated by at least 30 human raters for both semantic similarity and semantic relatedness. In addition to the semantic ratings, the dataset includes the word categorization used to build the word pairs and detailed sociodemographic information about annotators, enabling the analysis of the influence of personal factors on the perception of semantic relationships. Furthermore, this article describes in detail the dataset construction process, from word selection to agreement metrics.Data was collected from Portuguese university psychology students, who completed two rounds of questionnaires. In the first round annotators were asked to rate word pairs on either semantic similarity or relatedness. The second round switched the relation type for most annotators, with a small percentage being asked to repeat the same relation. The instructions given emphasized the differences between semantic relatedness and semantic similarity, and provided examples of expected ratings of both.There are few semantic relations datasets in Portuguese, and none focusing on affective words. PAP900 is distributed in distinct formats to be easy to use for both researchers just looking for the final averaged values and for researchers looking to take advantage of the individual ratings, the word categorization and the annotator data. This dataset is a valuable resource for researchers in computational linguistics, natural language processing, psychology, and cognitive science.

Semantic relation computing theory and its application

Few-shot knowledge graph reasoning (FS-KGR) try to infer missing facts in a knowledge graphs using limited data (such as only 3/5 samples).Existing strategies have shown good performance by mining more supervised information for few-shot learning through meta-learning and self-supervised learning. However, the problem of insufficient samples has not been fundamentally solved. In this paper, we propose a novel algorithm based on adversarial learning for Enhancing Negative samples in few-shot scenarios of FS-KGR, termed FS-KEN. Specifically, we are the first to use GAN to conduct data augmentation on FS-KGR scenario. FS-KEN uses policy gradient GANs for negative sample augmentation, solving the gradient back-propagation issue in traditional GANs. The generator aims to produce high-quality negative entities. while the objective of the discriminator is to distinguish between generated entities and real entities. Comprehensive experiments conducted on two few-shot knowledge graph completion datasets reveal that FS-KEN surpasses other baseline models, achieving state-of-the-art results.