Due to the widespread applications of heterogeneous graphs in the real world, heterogeneous graph neural networks (HGNNs) have developed rapidly and made a great success in recent years. To effectively capture the complex interactions in heterogeneous graphs, various attention mechanisms are widely used in designing HGNNs. However, the employment of these attention mechanisms brings two key problems: high computational complexity and poor robustness. To address these problems, we propose a <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Fast</b> and <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Ro</b> bust attention-free <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">H</b> eterogeneous <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G</b> raph <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">C</b> onvolutional <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</b> etwork (FastRo-HGCN) without any attention mechanisms. Specifically, we first construct virtual links based on the topology similarity and feature similarity of the nodes to strengthen the connections between the target nodes. Then, we design type normalization to aggregate and transfer the intra-type and inter-type node information. The above methods are used to reduce the interference of noisy information. Finally, we further enhance the robustness and relieve the negative effects of oversmoothing with the self-loops of nodes. Extensive experimental results on three real-world datasets fully demonstrate that the proposed FastRo-HGCN significantly outperforms the state-of-the-art models. The codes and data of this work are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/ZZY-GraphMiningLab/FastRo-HGCN</uri> .

The research on complex networks has achieved significant progress in revealing the mesoscopic features of networks. Community detection is an important aspect of understanding real-world complex systems. We present in this paper a High-order node proximity Spectral Clustering on Ratios-of-Eigenvectors (SCOREH+) algorithm for locating communities in complex networks. The algorithm improves SCORE and SCORE+ and preserves high-order transitivity information of the network affinity matrix. We optimize the high-order proximity matrix from the initial affinity matrix using the Radial Basis Functions (RBFs) and Katz index. In addition to the optimization of the Laplacian matrix, we implement a procedure that joins an additional eigenvector (the (k + 1)th leading eigenvector) to the spectrum domain for clustering if the network is considered to be a “weak signal” graph. The algorithm has been successfully applied to both real-world and synthetic data sets. The proposed algorithm is compared with state-of-art algorithms, such as ASE, Louvain, Fast-Greedy, Spectral Clustering (SC), SCORE, and SCORE+. To demonstrate the high efficacy of the proposed method, we conducted comparison experiments on eleven real-world networks and a number of synthetic networks with noise. The experimental results in most of these networks demonstrate that SCOREH+ outperforms the baseline methods. Moreover, by tuning the RBFs and their shaping parameters, we may generate state-of-the-art community structures on all real-world networks and even on noisy synthetic networks.

In recent years, split learning (SL) has emerged as a promising distributed learning framework that can utilize big data in parallel without privacy leakage while reducing client-side computing resources. In the initial implementation of SL, however, the server serves multiple clients sequentially incurring high latency. Parallel implementation of SL can alleviate this latency problem, but existing Parallel SL algorithms compromise scalability due to its fundamental structural problem. To this end, our previous works have proposed two scalable Parallel SL algorithms, dubbed SGLR and LocFedMix-SL, by solving the aforementioned fundamental problem of the Parallel SL structure. In this article, we propose a novel Parallel SL framework, coined Mix2SFL, that can ameliorate both accuracy and communication-efficiency while still ensuring scalability. Mix2SFL first supplies more samples to the server through a manifold mixup between the smashed data uploaded to the server as in SmashMix of LocFedMix-SL, and then averages the split-layer gradient as in GradMix of SGLR, followed by local model aggregation as in SFL. Numerical evaluation corroborates that Mix2SFL achieves improved performance in both accuracy and latency compared to the state-of-the-art SL algorithm with scalability guarantees. Moreover, its convergence speed as well as privacy guarantee are validated through the experimental results.

We present a novel approach to detecting fake news in Internet of Things (IoT) applications. By investigating federated learning and trusted authority methods, we address the issue of data security during training. Simultaneously, by investigating convolution transformers and user clustering, we deal with multi-modality in fake news data. Firstly, we use dense embedding and the k-means algorithm to cluster users into groups that are similar to one another. We then develop a local model for each user using their local data. The server then receives the local models of the users along with the clustering information, and a trusted authority verifies their integrity there. We use two different types of aggregation in place of conventional federated learning systems. The initial step is to combine all the users' models to create a single global model. The second step entails compiling each user's model into a local model of comparable users. Both models are supplied to the users, who then select the most suitable model for identifying fake news. By conducting extensive experiments using Twitter data, we demonstrate that the proposed method outperforms various baselines, where it achieves an average accuracy of 0.85 in comparison to others that do not exceed 0.81.

Deep Learning has been applied to many applications across different domains. However, the distribution shift between the test data and training data is a major factor impacting the quality of deep neural networks (DNNs). To address this issue, existing research mainly focuses on enhancing DNN models by retraining them using labeled test data. However, labeling test data is costly, which seriously reduces the efficiency of DNN testing. To solve this problem, test selection strategically selected a small set of tests to label. Unfortunately, existing test selection methods seldom focus on the data distribution shift. To address the issue, this paper proposes an approach for test selection named Feature Distribution Analysis-Based Test Selection (FATS). FATS analyzes the distributions of test data and training data and then adopts learning to rank (a kind of supervised machine learning to solve ranking tasks) to intelligently combine the results of analysis for test selection. We conduct an empirical study on popular datasets and DNN models, and then compare FATS with seven test selection methods. Experiment results show that FATS effectively alleviates the impact of distribution shifts and outperforms the compared methods with the average accuracy improvement of 19.6%<inline-formula><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula>69.7% for DNN model enhancement.

A data publishing deal conducted with anonymous microdata can preserve the privacy of people. However, anonymizing data with multiple records of an individual (1:M dataset) is still a challenging problem. After anonymizing the 1:M microdata, the vertical correlation can be exploited to launch privacy attacks. In this paper, a novel privacy preserving model <inline-formula><tex-math notation="LaTeX">$l_{c}, l_{s}$</tex-math></inline-formula>-ANGEL is proposed. To validate the new model, two privacy attacks are presented, namely, a Vertical correlation attack (<inline-formula><tex-math notation="LaTeX">$V_{c0}$</tex-math></inline-formula>) and a Vulnerable sensitive attribute attack (<inline-formula><tex-math notation="LaTeX">$V_{sa}$</tex-math></inline-formula>) on 1:M datasets, which breach the privacy of individuals. Furthermore, the proposed model is examined through High-Level Petri Nets (HLPNs). Our experiments on three real-world datasets;“INFORMS”,“YOUTUBE”, and “IMDb” demonstrate that the proposed model outperforms the state-of-the-art models. Our practices and lessons learned in this work can direct future concrete steps towards Multiple Sensitive Attributes, where we can expand the proposed model to dynamic datasets.

Fact verification tasks aim to identify the integrity of textual contents according to the truthful corpus. Existing fact verification models usually build a fully connected reasoning graph, which regards claim-evidence pairs as nodes and connects them with edges. They employ the graph to propagate the semantics of the nodes. Nevertheless, the noisy nodes usually propagate their semantics via the edges of the reasoning graph, which misleads the semantic representations of other nodes and amplifies the noise signals. To mitigate the propagation of noisy semantic information, we introduce a Confidential Graph Attention Network (CO-GAT), which proposes a node masking mechanism for modeling the nodes. Specifically, CO-GAT calculates the node confidence score by estimating the relevance between the claim and evidence pieces. Then, the node masking mechanism uses the node confidence scores to control the noise information flow from the vanilla node to the other graph nodes. CO-GAT achieves a 73.59% FEVER score on the FEVER dataset and shows the generalization ability by broadening the effectiveness to the science-specific domain.

In the era of Big Data, artificial intelligence and information science are the key technologies to extract the value of data and enhance the competitiveness of enterprises. The characteristics of distributed, small-scale, and sparse lead to the isolated data island problem. To solve these problems, Federated Learning is proposed. However, a large number of terminal models need to be uploaded to the server in Federated Learning, especially for the actual scenario of Internet of Things. Therefore, huge communication costs are required which dramatically increases the pressure on the backbone network. Furthermore, the low quality of the local model will lead to decreased accuracy and convergence rates of the model. To overcome the above limitations, we propose heterogeneous device collaboration based federated learning (HDCFL), which constructs a three-layer structure for Federated Learning by leveraging edge computing and designs a heterogeneous device collaboration method that groups the terminals based on their computing power, communication time, and data volume to train the model. Then, we conduct a theoretical analysis of the proposed algorithm which verifies its advantage. At last, the experimental result demonstrates that the proposed algorithm consistently achieves superior performance in terms of both convergence speed and accuracy compared with state-of-the-art baselines.

This paper studies how AI-assisted programming and large language models (LLM) improve software developers' ability via AI tools (LLM agents) like Github Copilot and Amazon CodeWhisperer, while integrating human feedback to enhance reinforcement learning (RLHF) with crowd-sourced computation to enhance text-to-code generation. Additionally, we demonstrate that our Bayesian optimization framework supports AI alignment in code generation by distributing the feedback collection burden, highlighting the value of collecting human feedback of good quality. Our empirical evaluations demonstrate the efficacy of this approach, showcasing how LLM agents can be effectively trained for improved text-to-code generation. Our Bayesian optimization framework can be designed for general domain-specific languages, promoting the alignment of large language model capabilities with human feedback in AI-assisted programming for code generation.

Federated learning can facilitate multiple parties to train a shared model on their own private data in a communication-efficient manner. It offers significant benefits for fine-tuning pre-trained large language models, as it supports distributed fine-tuning with a wider range of diverse data while preserving data privacy. However, recent research has revealed a potential privacy vulnerability in federated learning, specifically in the sharing of gradients from clients to server. This vulnerability can lead to the leakage of training data for Transformer-based large language models, thereby allowing the recovery of textual data. In this paper, we conduct a comprehensive evaluation of the effectiveness of the state-of-the-art gradient leakage attacks on textual data within the context of fine-tuning large language models. Our findings reveal that the key element for the attack's success — the target gradient — is not as readily obtainable for the adversary as previously assumed, particularly in regards to the Transformer architecture and practical federated learning settings. A technical error in their implementations has inadvertently caused the gradient to become more associated with the target data than intended. With the error fixed and when following the conventional federated learning framework, gradient leakage attacks pose minimal threats to large language models.