Deep Graph Convolutional Networks Based on Contrastive Learning: Alleviating Over-smoothing Phenomenon

Graph convolutional network (GCN) algorithms have been employed to learn graph embedding due to its inductive inference property, which is extended to GCN with higher-order information. However, the application of these higher-order information is confusing and not effectively distinguished. In addition, single channel GCN using higher-order information is weak for robust feature learning, and existing dual-channel GCNs rarely take higher-order information into account. To alleviate the above problems, we propose a dual-channel GCN with higher-order information for robust feature learning, denoted as HDGCN. Firstly, features of positive and negative higher-order graphs are extracted that fully exploits the self-contained attributes and higher-order geometric information. Meantime, the features of original graph structure are extracted by a conventional GCN that utilizes the self-contained feature attributes. Then, node features are represented as edge features by a feature fusion function. For the selection of negative samples, a fractional staggered negative sampling method is applied, by which the trainable graph model gains better topological features. Finally, the performance on seven real-world datasets demonstrates that HDGCN obtains the state-of-the-art performance on pairwise link prediction, higher-order structure prediction, and node classification tasks. By changing the attributes of multiple tasks, it can be proved that HDGCN has good robustness.

Deep Graph Convolutional Networks (GCNs) with multiple layers have been used for applications such as point cloud classification and semantic segmentation and achieved state-of-the-art results. However, they are computationally expensive and have a high run-time latency. In this paper, we propose AgileGCN, a novel framework to compress and accelerate deep GCN models with residual connections using structured pruning. Specifically, in each residual structure of a deep GCN, channel sampling and padding are applied to the input and output channels of a convolutional layer, respectively, to significantly reduce its floating point operations (FLOPs) and number of parameters. Experimental results on two benchmark point cloud datasets demonstrate that AgileGCN achieves significant FLOPs and parameters reduction while maintaining the performance of the unpruned models for both point cloud classification and segmentation.

A graph convolutional network (GCN) employs a graph filtering kernel tailored for data with irregular structures. However, simply stacking more GCN layers does not improve performance; instead, the output converges to an uninformative low-dimensional subspace, where the convergence rate is characterized by the graph spectrum— this is the known over-smoothing problem in GCN. In this paper, we propose a sparse graph learning algorithm incorporating a new spectrum prior to compute a graph topology that circumvents over-smoothing while preserving pairwise correlations inherent in data. Specifically, based on a spectral analysis of multilayer GCN output, we derive a spectrum prior for the graph Laplacian matrix L to robustify the model expressiveness against over-smoothing. Then, we formulate a sparse graph learning problem with the spectrum prior, solved efficiently via block coordinate descent (BCD). Moreover, we optimize the weight parameter trading off the fidelity term with the spectrum prior, based on data smoothness on the original graph learned without spectrum manipulation. The output L is then normalized for supervised GCN training. Experiments show that our proposal produced deeper GCNs and higher prediction accuracy for regression and classification tasks compared to competing schemes.

Exploiting node-feature bipartite graph in graph convolutional networks

Graph Convolutional Network (GCN) is an emerging technique that performs learning and reasoning on graph data. It operates feature learning on the graph structure, through aggregating the features of the neighbor nodes to obtain the embedding of each target node. Owing to the strong representation power, recent research shows that GCN achieves state-of-the-art performance on several tasks such as recommendation and linked document classification. Despite its effectiveness, we argue that existing designs of GCN forgo modeling cross features, making GCN less effective for tasks or data where cross features are important. Although neural network can approximate any continuous function, including the multiplication operator for modeling feature crosses, it can be rather inefficient to do so (i.e., wasting many parameters at the risk of overfitting) if there is no explicit design. To this end, we design a new operator named Cross-feature Graph Convolution, which explicitly models the arbitrary-order cross features with complexity linear to feature dimension and order size. We term our proposed architecture as Cross-GCN, and conduct experiments on three graphs to validate its effectiveness. Extensive analysis validates the utility of explicitly modeling cross features in GCN, especially for feature learning at lower layers.

The generalization error of graph convolutional networks may enlarge with more layers

Graph convolutional network (GCN) based approaches have achieved significant progress for solving complex, graph-structured problems. GCNs incorporate the graph structure information and the node (or edge) features through message passing and computes ‘deep’ node representations. Despite significant progress in the field, designing GCN architectures for heterogeneous graphs still remains an open challenge. Due to the schema of a heterogeneous graph, useful information may reside multiple hops away. A key question is how to perform message passing to incorporate information of neighbors multiple hops away while avoiding the well-known over-smoothing problem in GCNs. To address this question, we propose our GCN framework Deep Heterogeneous Graph Convolutional Network (DHGCN), which takes advantage of the schema of a heterogeneous graph and uses a hierarchical approach to effectively utilize information many hops away. It first computes representations of the target nodes based on their schema-derived ego-network (SEN). It then links the nodes of the same type with various pre-defined metapaths and performs message passing along these links to compute final node representations. Our design choices naturally capture the way a heterogeneous graph is generated from the schema. The experimental results on real and synthetic datasets corroborate the design choice and illustrate the performance gains relative to competing alternatives.

Neuroimaging plays an significant role in diagnosing and pathological study of brain diseases. Considering that both functional and structural abnormalities may lead to brain dis-eases and disorders, single modal neuroimaging approach may not fully characterize brain activities and working modes. Fusion of multimodal neuroimaging data is expected to provide more comprehensive characterization of brain diseases, given that the different modalities contain more complementary information. Recently, Graph Convolutional Networks (GCNs) is shown to have powerful capacity in representation learning for graph-structure data, which is considered to integrate both graph se-mantic structure and node information. Therefore, in this paper, we propose the Weighted Graph AutoEncoder (WGAE), a GCN- driven multimodal fusion model, to learn the combinational latent node representation of fMRI and DTI neuroimaging data, which are used as node features and graph structure respectively in the graph in unsupervised manner. Experimental results on two real-world datasets show the superiority of the proposed model over other existing single-modal or multi-modal methods in learning representations for disease prediction as the downstream task. Furthermore, ablation experiments also show the collaborative contribution of multimodal neuroimaging fusion in the proposed model, and also show the feasibility of assessing the respective importance of the two modalities during the disease prediction.

LightGCAN: A lightweight graph convolutional attention network for user preference modeling and personalized recommendation

Cross-subject emotion recognition in brain-computer interface based on frequency band attention graph convolutional adversarial neural networks

Coronary heart disease (CHD) has become one of the most serious public health issues due to its high morbidity and mortality rates. Most of the existing coronary heart disease risk prediction models manually extract features based on shallow machine learning methods. It only focuses on the differences between local patient features and ignores the interaction modeling between global patients. Its accuracy is still insufficient for individualized patient management strategies. In this paper, we propose CHD prediction as a graph node classification task for the first time, where nodes can represent individuals in potentially diseased populations and graphs intuitively represent associations between populations. We used an adaptive multi-channel graph convolutional neural network (AM-GCN) model to extract graph embeddings from topology, node features, and their combinations through graph convolution. Then, the adaptive importance weights of the extracted embeddings are learned by using an attention mechanism. For different situations, we model the relationship of the CHD population with the population graph and the K-nearest neighbor graph method. Our experimental evaluation explored the impact of the independent components of the model on the CHD disease prediction performance and compared it to different baselines. The experimental results show that our new model exhibits the best experimental results on the CHD dataset, with a 1.3% improvement in accuracy, a 5.1% improvement in AUC, and a 4.6% improvement in F1-score compared to the nongraph model.

WiFi fingerprinting positioning is one of the main indoor localization methods because of its reliable accuracy and low cost of deployment. The traditional methods conduct the localization by comparing the online RSSI with the fingerprint database using the various distance metrics while ignoring the structure information of fingerprints in the spatial and signal spaces. To tackle the problem, we proposes a graph learning based method with the combination of geographical location and node features. Our graph regards the WIFI fingerprints as graph nodes. Then we uses the semi-supervised graph convolutional neural network to predict the locations of the online the RSSI. After that, the prediction is preprocessed by uses the smoothing prediction with label propagation. Due to the experimental results, the proposed algorithm achieves good positioning results on both the UJIIndoorLoc public dataset and the self-collected museum dataset.

A Novel Privacy-Preserving Graph Convolutional Network via Secure Matrix Multiplication

The objective of extracting relations between document components lies in identifying the Within a single document, the focus often lies on understanding the connections between different entities. This type of analysis goes beyond individual sentences, as it demands an understanding of how information from multiple sentences interacts to form these connections. Over recent years, the importance of exploring relationships involving several entities simultaneously has grown significantly. To advance the field of studying such connections across entire documents, a novel collection of data points, known as DocRED, has been introduced. Currently, the standard approach for this task involves using BiLSTM networks to process the entire document as a whole. However, this method struggles to effectively capture the intricate relationships that exist among various entities. To overcome this limitation, a new model designed for document-level relationship identification has been developed, which leverages Graph Convolutional Networks (GCN). GCNs are particularly useful here because they can gather information from surrounding entities, allowing for a more detailed modeling of their interactions. The proposed approach starts by identifying coreferential links to gather features that represent the relationships between pairs of entities. These features are then analyzed using GCN to construct a graph structure that represents the entire document, ultimately revealing the complex interactions between different entities. Testing this model on the large-scale DocRED dataset from Tsinghua University has demonstrated its strong performance in this challenging task.

Single-cell RNA sequencing (scRNAseq) makes it possible to analyze gene expression profiles at the individual cell scale and to discover intrinsic and extrinsic cellular processes in biological research. Cell clustering is one of the most important steps in analyzing scRNAseq data. With rapid developments of single cell sequencing technologies, scRNAseq data grow in size and heterogeneity. However, traditional clustering methods like Kmeans with or without dimension reduction methods, cannot handle high sparse and massive scRNAseq data. Although some deep learning based methods have been proposed to denoise the data and cluster cells simultaneously, learning informative representations of cells for accurate cell clustering is still a challenging problem to be solved. In this work, we propose a deep learning model that combines a deep graph convolutional network (GCN) and a self-supervised mechanism. The GCN considers not only the gene expressions but also the relationship between cells to represent cells. The self-supervised mechanism is employed to provide the clustering assignments of cells. Moreover, we utilize the negative log-likelihood of the negative binomial (NB) function as loss in the data reconstruction due to the assumption that genes expression values can be represented by the NB model. We compared the performance of our proposed method with those of the existing clustering methods for scRNAseq data and conventional clustering methods. Results show that our method achieves better performance in terms of accuracy, adjusted random index (ARI), and normalized mutual information (NMI).