A Comparative Analysis of Community Detection Methods in Massive Datasets

Community detection in social networks based on fire propagation

This thesis focuses on the pressing issue of illicit drug trafficking and its impact on public health and safety at a global level. With the advent of digital technologies and social media platforms, combating drug trafficking has become increasingly challenging for law enforcement and researchers alike. Among these platforms, Instagram, a popular photo and video-sharing social networking platform, has emerged as a prominent hub for drug trafficking activities. In this study, we delve into the effectiveness of community and key player detection algorithms in identifying and disrupting illicit drug supply networks on Instagram. To conduct our research, we collected real Instagram data spanning from June to August 2022. We examined several community detection algorithms, including Louvain, Newman-Girvan, Infomap, Label Propagation, and Hierarchical Clustering. Additionally, we explored key player algorithms, namely CDKPE, TopRank, K-core, and KPEI, the latter being a novel algorithm introduced in this study. Our objective was to assess the performance of these algorithms in accurately identifying and targeting drug trafficking networks. Our findings reveal that the Louvain and Newman-Girvan algorithms outperformed others in terms of community detection, demonstrating their effectiveness in identifying cohesive groups involved in drug trafficking on Instagram. In terms of key player detection, the CDKPE and KPEI algorithms emerged as the most effective, highlighting the individuals who play pivotal roles within these networks. These algorithms offer practical applications for law enforcement agencies seeking to disrupt drug trafficking operations on Instagram. By emphasizing the importance of leveraging advanced analytical tools, our study underscores the significance of combating drug trafficking on social media platforms.

Many real world systems or web services can be represented as a network such as social networks and transportation networks. In the past decade, many algorithms have been developed to detect the communities in a network. However, the impact of locations on community has not been fully investigated by the research literature. In this paper, we propose a method to determine if a location-based community detection method is suitable for a given network and provide a new community detection algorithm that pushes the location information into the community detection. We test our proposed method on both synthetic data and real world network datasets. The results show that the communities detected by our method distribute in a smaller area compared with the traditional methods and have the similar or higher tightness on network connections.

Most community detection algorithms are trying to obtain the global information of the network. But increasingly large scale of the current network makes accessing to global information very difficult. In the meanwhile, the network shows power-law distribution and sparse features. And local community mining algorithms which use these features have more advantages over global mining methods. In this paper, we proposed a local community detection algorithm based on the core members named LLCDA (Leader based Local Community Detecting Algorithm) which uses local structural information in the network to optimize a local objective function. A local community can be detected through continuous optimization of the function by expanding from an initial core member computed by a modified PageRank sorting algorithm. The proposed LLCDA algorithm has been tested on both synthetic and real world networks, and it has been compared with other community detecting algorithms. The experimental results validated our proposed LLCDA and showed that significant improvements have been achieved by this technique.

Communities are virtually ubiquitous in real-world networks, and the statistic of modularity index Q is the classical measurement for community detection algorithms. However, the relationship between the modularity property and network multilever micro-scale structures is still not clear. In this paper, we study community detection results both in artificial and real-life complex networks by constructing different order null networks, and the results uncover that how micro-structures (such as degree distribution, assortativity and clustering coefficient) affect community properties. Meanwhile, we also propose two novel null networks (increasing or decreasing community structures) to verify the robustness of different community detection algorithms. Our results indicate that the modularity index Q is not a suitable statistic to measure the weak community property which is widely available in empirical networks. Our findings can not only be used to test the robustness of different community detection methods, but also be helpful to uncover the correlation of network structures between microcosmic and mesoscopic scales.

Incorporating Community Detection and Clustering Techniques into Collaborative Filtering Model

Community detection is an effective approach to unveil relationships among individuals in online social networks. In the literature, quite a few algorithms have been proposed to conduct community detection by exploiting the topology of social networks and the attributes of social actors. In practice, community detection is usually conducted by third parties, such as advertisement companies and hospitals, with access to social networks for different purposes, which can easily lead to a privacy breach. In this paper, we investigate community detection in social networks aiming to protect the privacy of both the network topology and the users' attributes. We show that with additional prior knowledge, community detection can be performed by querying the information of only a fraction of instead of the entire population. In particular, we first propose a new scheme called differentially private community detection (DPCD). DPCD detects communities in social networks via a probabilistic generative model, which can be decomposed into subproblems solved by individual users. The private social relationships and attributes of each user are protected by objective perturbation with differential privacy guarantees. Then, we propose a parsimonious node affiliation recovery (NAR) algorithm, which is also differentially private, to unveil the community affiliation information of the whole population based on that of the limited number of queried individuals by solving a sparse optimization problem. Through both theoretical analysis and experimental validation using synthetic and real-world social networks, we demonstrate that the proposed DPCD scheme detects social communities under the modest privacy budget. In addition, we show the effectiveness of NAR to perform community detection by querying a limited number of individuals in social networks.

Community detection is crucial in the analysis of social networks. Its main goal is to find groups of users densely connected among them, and sparsely connected between themselves. There are so many algorithms which makes it difficult for researchers to choose the best one for a specific dataset. In this paper, we provide a thorough comparison between five Community Detection (CD) algorithms- Girman-Newman (GN), Clauset-Newman-Moore (CNM), Label Propagation Algorithm (LPA), Louvain and Leiden. To evaluate in real social network datasets like Zachary's Karate Club, Dolphin networks and bigger ones such as Facebook, Twitter, LinkedIn along with citation networks we used different metrics modularity and NMI. We employed Normalized Mutual Information (NMI) to quantify the agreement of detected communities with their true ground-truth score and modularity for evaluating the overall quality of partitions given by diverse methods. Our experiments show that greedy and modularity-optimization algorithms are particularly well-suited. Notably, the Leiden algorithm had a better modularity value than Louvain (Q = 0.9141 and Q = 0.9051 of LinkedIn Network respectively) in most of the dataset. The NMI plot provided more explanation about Clauset-Newman-Moore (CNM), Louvain and Label Propagation Algorithm (LPA)which are in good agreement on community detection and their NMI score. These results would allow us to choose more rationally among the different community detection algorithms for social network analysis, by providing accurate quantitative benchmarks.

Community detection in complex networks has increasingly become an important topic in the network, but in most community detection methods, a single node only belongs to one community. In fact, there is often overlap among real-world online communities. In this paper, an overlapping community detection algorithm based on subgraph structure and multi-optimization method is designed. In this algorithm, the maximum clique mined by k-core decomposition is used as the clique node, thus the overlap characteristic is transformed into the inherent characteristic of the new graph. After that, a population initialization method based on k-core decomposition is designed, and the discrete framework of multi-objective particle swarm optimization algorithm is used to optimize the two objectives on the basis of maximal clique graph to solve the problem of overlapping community detection. In the real-world network, this algorithm is compared with similar community detection algorithms. The comparison of the evaluation indexe shows that the community detection effect of this algorithm is similar to that of similar algorithms, and has a good application prospect.

Community detection is an advanced graph operation that is used to reveal tightly-knit groups of vertices (aka. communities) in real-world networks. Given the intractability of the problem, efficient heuristics are used in practice. Yet, even the best of these state-of-the-art heuristics can become computationally demanding over large inputs and can generate workloads that exhibit inherent irregularity in data movement on manycore platforms. In this paper, we posit that effective acceleration of the graph community detection operation can be achieved by reducing the cost of data movement through a combined innovation at both software and hardware levels. More specifically, we first propose an efficient software-level parallelization of community detection that uses approximate updates to cleverly exploit a diminishing returns property of the algorithm. Secondly, as a way to augment this innovation at the software layer, we design an efficient Wireless Network on Chip (WiNoC) architecture that is suited to handle the irregular on-chip data movements exhibited by the community detection algorithm under both unicast- and broadcast-heavy cache coherence protocols. Experimental results show that our resulting WiNoC-enabled manycore platform achieves on average 52% savings in execution time, without compromising on the quality of the outputs, when compared to a traditional manycore platform designed with a wireline mesh NoC and running community detection without employing approximate updates.

How to efficiently detect communities for dynamic graphs has attracted significant attention due to its widespread applications, such as "Recommended System" in social networking or "Anti-Money Laundry" for banks. However, with the growing size and the increasing of changing frequency for dynamic graphs, it is better to detect communities incrementally than applying community detection algorithm for each of the graph. In this paper, we propose a generalized bottom-up community detection framework to help standard community detection algorithms to detect communities in highly dynamic graphs incrementally. The evaluations on real data sets show that our generalized framework does have the ability to accelerate standard community detection methods for highly dynamic graphs (up to 96%). In addition, for super hubs in large scale graph, we also propose an approximate process to meet the requirement of real-time processing. By giving "mathematical proof", "efficiency" and "accuracy" evaluations from real data sets, we demonstrate this approximate process can not only accelerate the community detection process but also can preserve the accuracy, as the normalized mutual information for regular communities and approximate communities is larger than 92%.

The technology is making its road of development continuously and all the users are enjoying the benefits from it and leading an easy life. With the rise in usage of internet, many people join online social networks to connect with the world, share their views, and interact with others. Thus, the increasing sizes of social networks are becoming complex and are involving a huge amount of data. Mining these data has become one of the domains in information technology. The fundamental application and/or also active research from this data is discovering meaningful communities from interaction among different people of a huge network. The algorithms on community detection are very useful & necessary to find the pattern or structure of the social network, analyze the social network, find out the relationship between two people and also control the sentiments of the people. The quality of detected communities is also measured by using different parameters. This work starts with an explanation on the concepts of online social networks, communities and community detections then survey on the different community detection algorithms have been done based on the different categories.

Community detection is one of the primary tools to discover useful information that is hidden in complex networks. Some community detection algorithms for bipartite networks have been proposed from various viewpoints. However, the performance of these algorithms deteriorates when the community structure becomes unclear. Enhancing community structure remains a nontrivial task. In this paper, we propose a community detection algorithm, called ECD, that enhances community structure in bipartite networks. In the proposed ECD, the topology of a network is modified by reducing unnecessary edges that are connected to neighboring low-weight communities. Therefore, an ambiguous community structure is converted into a structure that is much clearer than the original structure. The experimental results on both artificial and real-world networks verify the accuracy and reliability of our algorithm. Compared with existing community detection algorithms using state-of-the-art methods, our algorithm has better performance.

The research on complex networks has achieved significant progress in revealing the mesoscopic features of networks. Community detection is an important aspect of understanding real-world complex systems. We present in this paper a High-order node proximity Spectral Clustering on Ratios-of-Eigenvectors (SCOREH+) algorithm for locating communities in complex networks. The algorithm improves SCORE and SCORE+ and preserves high-order transitivity information of the network affinity matrix. We optimize the high-order proximity matrix from the initial affinity matrix using the Radial Basis Functions (RBFs) and Katz index. In addition to the optimization of the Laplacian matrix, we implement a procedure that joins an additional eigenvector (the (k + 1)th leading eigenvector) to the spectrum domain for clustering if the network is considered to be a “weak signal” graph. The algorithm has been successfully applied to both real-world and synthetic data sets. The proposed algorithm is compared with state-of-art algorithms, such as ASE, Louvain, Fast-Greedy, Spectral Clustering (SC), SCORE, and SCORE+. To demonstrate the high efficacy of the proposed method, we conducted comparison experiments on eleven real-world networks and a number of synthetic networks with noise. The experimental results in most of these networks demonstrate that SCOREH+ outperforms the baseline methods. Moreover, by tuning the RBFs and their shaping parameters, we may generate state-of-the-art community structures on all real-world networks and even on noisy synthetic networks.

Previous studies on network mining have focused primarily on learning a single task (such as classification or community detection) on a given network. This paper considers the problem of multi-task learning on heterogeneous network data. Specifically, we present a novel framework that enables one to perform classification on one network and community detection in another related network. Multi-task learning is accomplished by introducing a joint objective function that must be optimized to ensure the classes in one network are consistent with the link structure, nodal attributes, as well as the communities detected in another network. We provide both theoretical and empirical analysis of the framework. We also show that the framework can be extended to incorporate prior information about the correspondences between the clusters and classes in different networks. Experiments performed on both real-world and synthetic data sets demonstrate the effectiveness of the joint framework compared to applying classification and community detection algorithms on each network separately.