Semisupervised Learning on Graphs With an Alternating Diffusion Process.

Towards Dynamic Self-Training for Scalable Semi-Supervised Learning on Graphs

Speaker attribution is required in many real-world applications, such as\nmeeting transcription, where speaker identity is assigned to each utterance\naccording to speaker voice profiles. In this paper, we propose to solve the\nspeaker attribution problem by using graph-based semi-supervised learning\nmethods. A graph of speech segments is built for each session, on which\nsegments from voice profiles are represented by labeled nodes while segments\nfrom test utterances are unlabeled nodes. The weight of edges between nodes is\nevaluated by the similarities between the pretrained speaker embeddings of\nspeech segments. Speaker attribution then becomes a semi-supervised learning\nproblem on graphs, on which two graph-based methods are applied: label\npropagation (LP) and graph neural networks (GNNs). The proposed approaches are\nable to utilize the structural information of the graph to improve speaker\nattribution performance. Experimental results on real meeting data show that\nthe graph based approaches reduce speaker attribution error by up to 68%\ncompared to a baseline speaker identification approach that processes each\nutterance independently.\n

With the rapid growth of the Internet, more and more people interact with their friends in online social networks like Facebook1. Currently, the privacy issue of online social networks becomes a hot and dynamic research topic. Though some privacy protecting strategies are implemented, they are not stringent enough. Recently, Semi-Supervised Learning (SSL), which has the advantage of utilizing the unlabeled data to achieve better performance, attracts much attention from the web research community. By utilizing a large number of unlabeled data from websites, SSL can effectively infer hidden or sensitive information on the Internet. Furthermore, graph-based SSL is much more suitable for modeling real-world objects with graph characteristics, like online social networks. Thus, we propose a novel Community-based Graph (CG) SSL model that can be applied to exploit security issues in online social networks, then provide two consistent algorithms satisfying distinct needs. In order to evaluate the effectiveness of this model, we conduct a series of experiments on a synthetic data and two real-world data from StudiVZ2 and Facebook. Experimental results demonstrate that our approach can more accurately and confidently predict sensitive information of online users, comparing to previous models.

Graph based semi-supervised classification with probabilistic nearest neighbors

In this paper, we study the problem of how to reliably compute neighborhoods on affinity graphs. The k-nearest neighbors (kNN) is one of the most fundamental and simple methods widely used in many tasks, such as classification and graph construction. Previous research focused on how to efficiently compute kNN on vectorial data. However, most real-world data have no vectorial representations, and only have affinity graphs which may contain unreliable affinities. Since the kNN of an object o is a set of k objects with the highest affinities to o, it is easily disturbed by errors in pairwise affinities between o and other objects, and also it cannot well preserve the structure underlying the data. To reliably analyze the neighborhood on affinity graphs, we define the k-dense neighborhood (kDN), which considers all pairwise affinities within the neighborhood, i.e., not only the affinities between o and its neighbors but also between the neighbors. For an object o, its kDN is a set kDN(o) of k objects which maximizes the sum of all pairwise affinities of objects in the set {o}?kDN(o). We analyze the properties of kDN, and propose an efficient algorithm to compute it. Both theoretic analysis and experimental results on shape retrieval, semi-supervised learning, point set matching and data clustering show that kDN significantly outperforms kNN on affinity graphs, especially when many pairwise affinities are unreliable.

In real worlds applications, some former research papers have shown that manifold learning tries to discover the non-linear low-dimensional data manifold from a high-dimensional space. Many natural images and the face images are believed to be sampled from a manifold. In this paper, we try to investigate whether discovering such manifold can aid the semi-supervised learning algorithms. We propose a novel graph-based learning algorithm locality preserving graph-based semi-supervised method (LLGSM), which firstly use both labeled and unlabeled examples as unlabeled to discover the manifolds of the data samples and then use the projected labeled examples together with projected unlabeled ones to do classification. Experiments performed on some public image data sets have demonstrated the effectiveness of our algorithm.

Many image segmentation algorithms first generate an affinity graph and then partition it. We present a machine learning approach to computing an affinity graph using a convolutional network (CN) trained using ground truth provided by human experts. The CN affinity graph can be paired with any standard partitioning algorithm and improves segmentation accuracy significantly compared to standard hand-designed affinity functions. We apply our algorithm to the challenging 3D segmentation problem of reconstructing neuronal processes from volumetric electron microscopy (EM) and show that we are able to learn a good affinity graph directly from the raw EM images. Further, we show that our affinity graph improves the segmentation accuracy of both simple and sophisticated graph partitioning algorithms. In contrast to previous work, we do not rely on prior knowledge in the form of hand-designed image features or image preprocessing. Thus, we expect our algorithm to generalize effectively to arbitrary image types.

The fastest deterministic algorithms for connected components take logarithmic time and perform superlinear work on a Parallel Random Access Machine (PRAM). These algorithms maintain a spanning forest by merging and compressing trees, which requires pointer-chasing operations that increase memory access latency and are limited to shared-memory systems. Many of these PRAM algorithms are also very complicated to implement. Another popular method is “leader-contraction” where the challenge is to select a constant fraction of leaders that are adjacent to a constant fraction of non-leaders with high probability, but this can require adding more edges than were in the original graph. Instead we investigate label propagation because it is deterministic, easy to implement, and does not rely on pointer-chasing. Label propagation exchanges representative labels within a component using simple graph traversal, but it is inherently difficult to complete in a sublinear number of steps. We are able to overcome the problems with label propagation for graph connectivity. We introduce a surprisingly simple framework for deterministic, undirected graph connectivity using label propagation that is easily adaptable to many computational models. It achieves logarithmic convergence independently of the number of processors and without increasing the edge count. We employ a novel method of propagating directed edges in alternating direction while performing minimum reduction on vertex labels. We present new algorithms in PRAM, Stream, and MapReduce. Given a simple, undirected graph [Formula: see text] with [Formula: see text] vertices, [Formula: see text] edges, our approach takes O(m) work each step, but we can only prove logarithmic convergence on a path graph. It was conjectured by Liu and Tarjan (2019) to take [Formula: see text] steps or possibly [Formula: see text] steps. Our experiments on a range of difficult graphs also suggest logarithmic convergence. We leave the proof of convergence as an open problem.

We propose a robust inductive semi-supervised label prediction model over the embedded representation, termed adaptive embedded label propagation with weight learning (AELP-WL), for classification. AELP-WL offers several properties. First, our method seamlessly integrates the robust adaptive embedded label propagation with adaptive weight learning into a unified framework. By minimizing the reconstruction errors over embedded features and embedded soft labels jointly, our AELP-WL can explicitly ensure the learned weights to be joint optimal for representation and classification, which differs from most existing LP models that perform weight learning separately by an independent step before label prediction. Second, existing models usually precalculate the weights over the original samples that may contain unfavorable features and noise decreasing performance. To this end, our model adds a constraint that decomposes original data into a sparse component encoding embedded noise-removed sparse representations of samples and a sparse error part fitting noise, and then performs the adaptive weight learning over the embedded sparse representations. Third, our AELP-WL computes the projected soft labels by trading-off the manifold smoothness and label fitness errors over the adaptive weights and the embedded representations for enhancing the label estimation power. By including a regressive label approximation error for simultaneous minimization to correlate sample features with the embedded soft labels, the out-of-sample issue is naturally solved. By minimizing the reconstruction errors over features and embedded soft labels, classification error and label approximation error jointly, state-of-the-art results are delivered.

In machine learning community, graph-based semi-supervised learning (GSSL) approaches have attracted more extensive research due to their elegant mathematical formulation and good performance. However, one of the reasons affecting the performance of the GSSL method is that the training data and test data need to be independently identically distributed (IID); any individual user may show a completely different encephalogram (EEG) data in the same situation. The EEG data may be non-IID. In addition, noise/outlier sensitiveness still exist in GSSL approaches. To these ends, we propose in this paper a novel clustering method based on structure risk minimization model, called multi-model adaptation learning with possibilistic clustering assumption for EEG-based emotion recognition (MA-PCA). It can effectively minimize the influence from the noise/outlier samples based on different EEG-based data distribution in some reproduced kernel Hilbert space. Our main ideas are as follows: (1) reducing the negative impact of noise/outlier patterns through fuzzy entropy regularization, (2) considering the training data and test data are IID and non-IID to obtain a better performance by multi-model adaptation learning, and (3) the algorithm implementation and convergence theorem are also given. A large number of experiments and deep analysis on real DEAP datasets and SEED datasets was carried out. The results show that the MA-PCA method has superior or comparable robustness and generalization performance to EEG-based emotion recognition.

Named entity recognition (NER) plays a vital role in extracting critical medical entities from health care records, facilitating applications such as clinical decision support and data mining. Developing robust NER models for low-resource languages, such as Estonian, remains a challenge due to the scarcity of annotated data and domain-specific pretrained models. Large language models (LLMs) have proven to be promising in understanding text from any language or domain. This study addresses the development of medical NER models for low-resource languages, specifically Estonian. We propose a novel approach by generating synthetic health care data and using LLMs to annotate them. These synthetic data are then used to train a high-performing NER model, which is applied to real-world medical texts, preserving patient data privacy. Our approach to overcoming the shortage of annotated Estonian health care texts involves a three-step pipeline: (1) synthetic health care data are generated using a locally trained GPT-2 model on Estonian medical records, (2) the synthetic data are annotated with LLMs, specifically GPT-3.5-Turbo and GPT-4, and (3) the annotated synthetic data are then used to fine-tune an NER model, which is later tested on real-world medical data. This paper compares the performance of different prompts; assesses the impact of GPT-3.5-Turbo, GPT-4, and a local LLM; and explores the relationship between the amount of annotated synthetic data and model performance. The proposed methodology demonstrates significant potential in extracting named entities from real-world medical texts. Our top-performing setup achieved an F1-score of 0.69 for drug extraction and 0.38 for procedure extraction. These results indicate a strong performance in recognizing certain entity types while highlighting the complexity of extracting procedures. This paper demonstrates a successful approach to leveraging LLMs for training NER models using synthetic data, effectively preserving patient privacy. By avoiding reliance on human-annotated data, our method shows promise in developing models for low-resource languages, such as Estonian. Future work will focus on refining the synthetic data generation and expanding the method's applicability to other domains and languages.

To assess the validity of privacy-preserving synthetic data by comparing results from synthetic versus original EHR data analysis. A published retrospective cohort study on real-world effectiveness of COVID-19 vaccines by Maccabi Healthcare Services in Israel was replicated using synthetic data generated from the same source, and the results were compared between synthetic versus original datasets. The endpoints included COVID-19 infection, symptomatic COVID-19 infection and hospitalization due to infection and were also assessed in several demographic and clinical subgroups. In comparing synthetic versus original data estimates, several metrices were utilized: standardized mean differences (SMD), decision agreement, estimate agreement, confidence interval overlap, and Wald test. Synthetic data were generated five times to assess the stability of results. The distribution of demographic and clinical characteristics demonstrated very small difference (< 0.01 SMD). In the comparison of vaccine effectiveness assessed in relative risk reduction between synthetic versus original data, there was a 100% decision agreement, 100% estimate agreement, and a high level of confidence interval overlap (88.7%-99.7%) in all five replicates across all subgroups. Similar findings were achieved in the assessment of vaccine effectiveness against symptomatic COVID-19 Infection. In the comparison of hazard ratios for COVID 19-related hospitalization and odds ratio for symptomatic COVID-19 Infection, the Wald tests suggested no significant difference between respective effect estimates in all five replicates for all patient subgroups but there were disagreements in estimate and decision metrices in some subgroups and replicates. Overall, comparison of synthetic versus original real-world data demonstrated good validity and reliability. Transparency on the process to generate high fidelity synthetic data and assurances of patient privacy are warranted.

We propose a new transductive label propagation method, termed Adaptive Neighborhood Propagation (Adaptive-NP) by joint L2,1-norm regularized sparse coding, for semi-supervised classification. To make the predicted soft labels more accurate for predicting the labels of samples and to avoid the tricky process of choosing the optimal neighborhood size or kernel width for graph construction, Adaptive-NP seamlessly integrates sparse coding and neighborhood propagation into a unified framework. That is, the sparse reconstruction error and classification error are combined for joint minimization, which clearly differs from traditional methods that explicitly separate graph construction and label propagation into independent steps, which may result in inaccurate predictions. Note that our Adaptive-NP alternately optimize the sparse codes and soft labels matrices, where the sparse codes are used as adaptive weights for neighborhood propagation at each iteration, so the tricky process of determining neighborhood size or kernel width is avoided. Besides, for enhancing sparse coding, we use the L2,1-norm constraint on the sparse coding coefficients and the reconstruction error at the same time for delivering more accurate and robust representations. Extensive simulations show that our model can deliver state-of-the-art performances on several public datasets for classification.

Semi-supervised ranking is a relatively new and important learning problem. Some semi-supervised learning to rank algorithms have been proposed by adapting existing pair-wise ranking algorithms into the semi-supervised setting. In contrast, there is comparatively little work in developing semi-supervised ranking algorithm based on the list-wise approach. In this paper, we proposed a semi-supervised extension of one of the state-of-the art list-wise ranking algorithm--AdaRank, for the task of learning to rank with partially labeled data. The proposed algorithm is a two-staged approach that combines the process of label propagation and a regularized version of AdaRank. The label propagation process is used to propagate the relevance labels from labeled instances to other unlabeled ones, so that more training data are available to learn the ranking function. The regularized version of AdaRank optimizes a novel performance measure, which incorporates both the usual Information Retrieval IR metrics and the smoothness of the ranking function. Experimental results on benchmark datasets show the advantage of our methods over existing pair-wise algorithms for the Information Retrieval measures.

Cross-domain aspect extraction for sentiment analysis: A transductive learning approach