Adaptive Attributed Network Embedding for Community Detection

Recent research on community detection focuses on learning representations of nodes using different network embedding methods, and then feeding them as normal features to clustering algorithms. However, we find that though one may have good results by direct clustering based on such network embedding features, there is ample room for improvement. More seriously, in many real networks, some statisticallysignificant nodes which play pivotal roles are often divided into incorrect communities using network embedding methods. This is because while some distance measures are used to capture the spatial relationship between nodes by embedding, the nodes after mapping to feature vectors are essentially not coupled any more, losing important structural information. To address this problem, we propose a general Markov Random Field (MRF) framework to incorporate coupling in network embedding which allows better detecting network communities. By smartly utilizing properties of MRF, the new framework not only preserves the advantages of network embedding (e.g. low complexity, high parallelizability and applicability for traditional machine learning), but also alleviates its core drawback of inadequate representations of dependencies via making up the missing coupling relationships. Experiments on real networks show that our new approach improves the accuracy of existing embedding methods (e.g. Node2Vec, DeepWalk and MNMF), and corrects most wrongly-divided statistically-significant nodes, which makes network embedding essentially suitable for real community detection applications. The new approach also outperforms other state-of-the-art conventional community detection methods.

Network embedding which aims to embed a given network into a low-dimensional vector space has been proved effective in various network analysis and mining tasks such as node classification, link prediction and network visualization. The emerging network embedding methods have shifted of emphasis in utilizing mature deep learning models. The neural-network based network embedding has become a mainstream solution because of its high efficiency and capability of preserving the nonlinear characteristics of the network. In this paper, we propose Adversarial Network Embedding using Structural Similarity (ANESS), a novel, versatile, low-complexity GAN-based network embedding model which utilizes the inherent vertex-to-vertex structural similarity attribute of the network. ANESS learns robustness and effective vertex embeddings via a adversarial training procedure. Specifically, our method aims to exploit the strengths of generative adversarial networks in generating high-quality samples and utilize the structural similarity identity of vertexes to learn the latent representations of a network. Meanwhile, ANESS can dynamically update the strategy of generating samples during each training iteration. The extensive experiments have been conducted on the several benchmark network datasets, and empirical results demonstrate that ANESS significantly outperforms other state-of-the-art network embedding methods.

Network embedding assigns nodes in a network to low-dimensional representations and effectively preserves the network structure. Recently, a significant amount of progresses have been made toward this emerging network analysis paradigm. In this survey, we focus on categorizing and then reviewing the current development on network embedding methods, and point out its future research directions. We first summarize the motivation of network embedding. We discuss the classical graph embedding algorithms and their relationship with network embedding. Afterwards and primarily, we provide a comprehensive overview of a large number of network embedding methods in a systematic manner, covering the structure- and property-preserving network embedding methods, the network embedding methods with side information and the advanced information preserving network embedding methods. Moreover, several evaluation approaches for network embedding and some useful online resources, including the network data sets and softwares, are reviewed, too. Finally, we discuss the framework of exploiting these network embedding methods to build an effective system and point out some potential future directions.

Spatial re-use TDMA (STDMA) is a medium access control (MAC) protocol which allows multiple transmitter-receiver pairs to communicate simultaneously, as long as they do not interfere too much with each other. Node and link assignment strategies for spatial re-use TDMA networks have previously been studied independently. The metric chosen in these studies is the uniform capacity, the minimum rate at which any node can communicate with any other node in the network. It has been shown that for large frame lengths, node assignment leads to lower spatial re-use than link assignment, resulting in lower uniform capacity than link assignment. In this paper, we first explore the behaviour of node and link assignment at small frame lengths. We show that node assignment achieves non-zero throughput values at lower frame lengths than link assignment, and that it outperforms link assignment at small frame lengths. This motivates the need for an assignment strategy that performs well for all values of the frame length. We propose a joint node-link assignment strategy and show that it performs as well as node assignment at small frame lengths and as well as link assignment at large frame lengths, while exceeding the performance of both node and link assignment at intermediate frame lengths.

Many real-world optimization problems are characterized by multiple conflicting objectives, which are known as multi-objective optimization problems (MOPs). In the last two decades, evolutionary algorithms have shown promising performance in solving various MOPs, and a large number of multi-objective evolutionary algorithms (MOEAs) have been proposed. In order to determine the most suitable MOEA for a specific MOP, it is usually necessary to perform experiments to compare the performance of multiple candidate MOEAs. In 2017, an evolutionary multi-objective optimization platform was proposed by us, called PlatEMO, which provides the source codes of many state-of-the-art MOEAs and helps researchers perform batch experiments on these MOEAs. In this work, we illustrate the method of using the newest version of PlatEMO to solve MOPs in applications, by means of a case study on the feature selection problem, which is an important and difficult task in machine learning and data mining. This paper details the method of adding the feature selection problem to PlatEMO, and presents the experimental results of eight MOEAs on nine datasets in feature selection.

Classification is a common task in Machine Learning and Data Mining. Some classification problems are called hierarchical classification problems because they need to take into account a hierarchical taxonomy which establishes an order between involved classes. The protein's function prediction is considered a hierarchical classification problem because their functions are arranged in a hierarchical taxonomy of classes. This paper presents an algorithm for hierarchical classification using the jumping emerging patterns approach. Jumping emerging patterns have been used to flat classification and in this work we explore its adoption in a hierarchical classification scenario. The proposed algorithm was evaluated in eight real datasets, compared against two other recent hierarchical classification algorithms from the literature and also with flat k-nearest neighbor classification algorithm. Preliminary results showed that the proposed approach is an alternative for hierarchical classification, having as main advantages the simplicity and understandability with good accuracy.

Clustering is a fundamental task in machine learning and data mining. The success of deep learning, especially deep generative models, has given birth to the next generation of clustering - deep unsupervised clustering featured by the capability of clustering large-scale and high-dimensional data. One typical example is the Gaussian mixture variational autoencoder (GMVAE), where the prior of the latent variable is a mixture of Gaussian distributions (GMM). However, the mixture number manually given in GMVAE greatly impacts on the final clustering performance and also limits the adaptation ability when new clusters come. To solve this issue, we propose an Infinite Gaussian Mixture Autoencoder (IGMVAE), which replaces the plain Gaussian prior in VAE with an infinite Gaussian mixture distribution. Our method adaptively learns the number of mixtures from data, overcoming the manual setting limitation in GMVAE, and dynamically expands clusters for novel categories. Experimental results on three datasets show that our proposed IGMVAE achieves 5.5%–14.25% higher clustering accuracy compared to GMVAE. Moreover, our model is capable of generating high-quality data in a fully unsupervised manner.

Classification is one of the main tasks in machine learning, data mining, and pattern recognition. Compared with the extensively studied automation approaches, the interactive approaches, centered on human users, are less explored. This chapter studies interactive classification at 3 levels. At the philosophical level, the motivations and a process-based framework of interactive classification are proposed. At the technical level, a granular computing model is suggested for re-examining not only existing classification problems, but also interactive classification problems. At the application level, an interactive classification system (ICS), using a granule network as the search space, is introduced. ICS allows multi-strategies for granule tree construction, and enhances the understanding and interpretation of the classification process. Interactive classification is complementary to the existing classification methods.

Feature selection is an important task in machine learning, pattern recognition and data mining. This paper proposed a new feature selection method for classification, named SD, which is based on scatter matrix used in linear discriminant analysis. The main feature of SD is its simplicity and independency of learning algorithms. High-dimensional data samples are first projected into a lower dimensional subspace of the original feature space by means of a linear transformation matrix, which can be attained according to the scatter degree of each feature, and then the scatter degree is used to measure the importance of each feature. A comparison of SD and some popular feature selection methods (information gain and chi <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -test) is conducted, and the results of experiment carried out on 19 data sets show the advantages of SD.

Learning classifiers from imbalanced data is still one of challenging tasks in machine learning and data mining. Data difficulty factors referring to internal and local characteristics of class distributions deteriorate performance of standard classifiers. Many of these factors may be approximated by analyzing the neighbourhood of the learning examples and identifying different types of examples from the minority class. In this paper, we follow recent research on developing such methods for assessing the types of examples which exploit either k-nearest neighbours or kernels. We discuss the approaches to tune the size of both kinds of neighborhoods depending on the data set characteristics and evaluate their usefulness in series of experiments with real-world and synthetic data sets. Furthermore, we claim that the proper analysis of these neighborhoods could be the basis for developing new specialized algorithms for imbalanced data. To illustrate it, we study generalizations of over-sampling in pre-processing methods and neighbourhood based ensembles.

Deep clustering is a fundamental task in machine learning and data mining that aims at learning clustering-oriented feature representations. In previous studies, most of deep clustering methods follow the idea of self-supervised representation learning by maximizing the consistency of all similar instance pairs while ignoring the effect of feature redundancy on clustering performance. In this paper, to address the above issue, we design a dual mutual information constrained clustering method named DMICC which is based on deep contrastive clustering architecture, in which the dual mutual information constraints are particularly employed with solid theoretical guarantees and experimental validations. Specifically, at the feature level, we reduce the redundancy among features by minimizing the mutual information across all the dimensionalities to encourage the neural network to extract more discriminative features. At the instance level, we maximize the mutual information of the similar instance pairs to obtain more unbiased and robust representations. The dual mutual information constraints happen simultaneously and thus complement each other to jointly optimize better features that are suitable for the clustering task. We also prove that our adopted mutual information constraints are superior in feature extraction, and the proposed dual mutual information constraints are clearly bounded and thus solvable. Extensive experiments on five benchmark datasets show that our proposed approach outperforms most other clustering algorithms. The code is available at https://github.com/Li-Hyn/DMICC.

Many statistical inference problems correspond to recovering the values of a set of hidden variables from sparse observations on them. For instance, in a planted constraint satisfaction problem such as planted 3-SAT, the clauses are sparse observations from which the hidden assignment is to be recovered. In the problem of community detection in a stochastic block model, the community labels are hidden variables that are to be recovered from the edges of the graph. Inspired by ideas from statistical physics, the presence of a stable fixed point for belief propogation has been widely conjectured to characterize the computational tractability of these problems. For community detection in stochastic block models, many of these predictions have been rigorously confirmed. In this work, we consider a general model of statistical inference problems that includes both community detection in stochastic block models, and all planted constraint satisfaction problems as special cases. We carry out the cavity method calculations from statistical physics to compute the regime of parameters where detection and recovery should be algorithmically tractable. At precisely the predicted tractable regime, we give: (i) a general polynomial-time algorithm for the problem of detection: distinguishing an input with a planted signal from one without; (ii) a general polynomial-time algorithm for the problem of recovery: outputting a vector that correlates with the hidden assignment significantly better than a random guess would. Analogous to the spectral algorithm for community detection [1], [2], the detection and recovery algorithms are based on the spectra of a matrix that arises as the derivatives of the belief propagation update rule. To devise a spectral algorithm in our general model, we obtain bounds on the spectral norms of certain families of random matrices with correlated and matrix valued entries. We then demonstrate how eigenvectors of various powers of the matrix can be used to partially recover the hidden variables.

Discretization of the value range of a numerical feature is a common task in data mining and machine learning. Optimal multivariate discretization is in general computationally intractable. We have proposed approximation algorithms with performance guarantees for training error minimization by axis-parallel hyperplanes. This work studies their efficiency and practicability. We give efficient implementations to both greedy set covering and linear programming approximation of optimal multivariate discretization. We also contrast the algorithms empirically to an efficient heuristic discretization method.

It is essential to identify similarity between graphs for various tasks in data mining, machine learning and pattern recognition. Graph edit distance (GED) is the most popular graph similarity measure thanks to its flexibility and versatility. In this paper, we study the problem of top-<i>k</i> graph similarity search, which finds <i>k</i> graphs most similar to a given query graph under the GED measure. We propose incremental GED computation algorithms that compute desired GED lower and upper bounds. Based on the algorithms, we develop novel search frameworks to address the top-<i>k</i> search problem. Our frameworks are also designed to use a state-of-the art indexing technique to speed up top-<i>k</i> search. By conducting extensive experiments on real datasets, we show that the proposed frameworks significantly improve the performance of top-<i>k</i> graph similarity search.

Application of Chi-square discretization algorithms to ensemble classification methods