A Unifying Framework for Mining Approximate Top-<inline-formula> <tex-math notation="LaTeX">$k$ </tex-math> </inline-formula> Binary Patterns

An important step in unveiling the relation between network structure and dynamics defined on networks is to detect communities, and numerous methods have been developed separately to identify community structure in different classes of networks, such as unipartite networks, bipartite networks, and directed networks. Here, we show that the finding of communities in such networks can be unified in a general framework-detection of community structure in bipartite networks. Moreover, we propose an evolutionary method for efficiently identifying communities in bipartite networks. To this end, we show that both unipartite and directed networks can be represented as bipartite networks, and their modularity is completely consistent with that for bipartite networks, the detection of modular structure on which can be reformulated as modularity maximization. To optimize the bipartite modularity, we develop a modified adaptive genetic algorithm (MAGA), which is shown to be especially efficient for community structure detection. The high efficiency of the MAGA is based on the following three improvements we make. First, we introduce a different measure for the informativeness of a locus instead of the standard deviation, which can exactly determine which loci mutate. This measure is the bias between the distribution of a locus over the current population and the uniform distribution of the locus, i.e., the Kullback-Leibler divergence between them. Second, we develop a reassignment technique for differentiating the informative state a locus has attained from the random state in the initial phase. Third, we present a modified mutation rule which by incorporating related operations can guarantee the convergence of the MAGA to the global optimum and can speed up the convergence process. Experimental results show that the MAGA outperforms existing methods in terms of modularity for both bipartite and unipartite networks.

Network Intrusion Detection Systems (NIDSs) are an increasingly important tool for the prevention and mitigation of cyber attacks. Over the past years, a lot of research efforts have aimed at leveraging the increasingly powerful models of Machine Learning (ML) for this purpose. A number of labelled synthetic datasets have been generated and made publicly available by researchers, and they have become the benchmarks via which new ML-based NIDS classifiers are being evaluated. Recently published results show excellent classification performance with these datasets, increasingly approaching 100 percent performance across key evaluation metrics such as Accuracy, F1 score, AUC, etc. Unfortunately, we have not yet seen these excellent academic research results translated into practical NIDS systems with such near-perfect performance. This motivated our research presented in this paper, where we analyse the statistical properties of the benign traffic in three of the more recent and relevant NIDS datasets, (CIC_IDS, UNSW_NB15, TON_IOT), by converting them into a common flow format. As a comparison, we consider two datasets obtained from real-world production networks, one from a university network and one from a medium size Internet Service Provider (ISP). Our results show that the two real-world datasets are quite similar among themselves in regards to most of the considered statistical features. Equally, the three synthetic datasets are also relatively similar within their group. However, and most importantly, our results show a distinct difference of most of the considered statistical features between the three synthetic datasets and the two real-world datasets. Since ML relies on the basic assumption of training and test datasets being sampled from the same distribution, this raises the question of how well the performance results of ML-classifiers trained on the considered synthetic datasets can translate and generalise to real-world networks. We believe this is an interesting and relevant question which provides motivation for further research in this space.

Background The utility of synthetic data in benchmark studies depends on its ability to closely mimic real-world conditions and to reproduce results obtained from experimental data. Here, we evaluate the performance of differential abundance tests for 16S metagenomic data. Building on the benchmark study by Nearing et al. (1), who assessed 14 differential abundance tests using 38 experimental datasets in a case-control design, we validate their findings by generating synthetic datasets that mimics the experimental data. We will employ statistical tests to rigorously assess the similarity between synthetic and experimental data and to validate the conclusions on the performance of these tests drawn by Nearing et al. (1). This protocol adheres to the SPIRIT guidelines and is, to our knowledge, the first of its kind in computational benchmark studies. Methods We replicate Nearing et al.’s (1) methodology, incorporating synthetic data simulated using two distinct tools, mirroring each of the 38 experimental datasets. Equivalence tests will be conducted on 43 data characteristics comparing synthetic and experimental data, complemented by principal component analysis for overall similarity assessment. The 14 differential abundance tests will be applied to both synthetic and experimental datasets, evaluating the consistency of significant feature identification and the number of significant features per tool. Correlation analysis and multiple regression will explore how differences between synthetic and experimental data characteristics may affect the results. Conclusions Synthetic data enables the validation of findings through controlled experiments. We assess how well synthetic data replicates experimental data, validate previous findings and delineate the strengths and limitations of synthetic data in benchmark studies. Moreover, to our knowledge this is the first computational benchmark study to systematically incorporate synthetic data for validating differential abundance methods while strictly adhering to a pre-specified study protocol following SPIRIT guidelines, contributing significantly to transparency, reproducibility, and unbiased research.

Synthetic data's utility in benchmark studies depends on its ability to closely mimic real-world conditions and reproduce results obtained from experimental data. Building on Nearing et al.'s study (1), who assessed 14 differential abundance tests using 38 experimental 16S rRNA datasets in a case-control design, we are generating synthetic datasets that mimic the experimental data to verify their findings. We will employ statistical tests to rigorously assess the similarity between synthetic and experimental data and to validate the conclusions on the performance of these tests drawn by Nearing et al. (1). This protocol adheres to the SPIRIT guidelines, demonstrating how established reporting frameworks can support robust, transparent, and unbiased study planning. We replicate Nearing et al.'s (1) methodology, incorporating synthetic data simulated using two distinct tools, mirroring the 38 experimental datasets. Equivalence tests will be conducted on a non-redundant subset of 46 data characteristics comparing synthetic and experimental data, complemented by principal component analysis for overall similarity assessment. The 14 differential abundance tests will be applied to synthetic and experimental datasets, evaluating the consistency of significant feature identification and the number of significant features per tool. Correlation analysis and multiple regression will explore how differences between synthetic and experimental data characteristics may affect the results. Synthetic data enables the validation of findings through controlled experiments. We assess how well synthetic data replicates experimental data, try to validate previous findings with the most recent versions of the DA methods and delineate the strengths and limitations of synthetic data in benchmark studies. Moreover, to our knowledge this is the first computational benchmark study to systematically incorporate synthetic data for validating differential abundance methods while strictly adhering to a pre-specified study protocol following SPIRIT guidelines, contributing to transparency, reproducibility, and unbiased research.

This article proposes a new network approach toward the implementation of Takagi–Sugeno (T–S) fuzzy models referred to as disjunctive fuzzy neural networks (DJFNNs). The proposed DJFNN involves a novel network architecture and a greedy learning algorithm. Being different from the existing grid-based and clustering-based network architectures, the proposed architecture adds an OR neural layer positioned between the fuzzification layer and the rule layer. In this way, the implied constraint between the number of rules and the number of fuzzy labels is excluded so that a curse of dimensionality can be overcome and more interpretable models are formed. Furthermore, inspired by the core algorithm for building a decision tree, a top–down, nonbacktracking, and greedy algorithm is proposed to learn the unknown parameters of the networks. The input space splits into smaller and smaller subspace along the predefined fuzzy grids in a supervised manner meanwhile the associated conditions of the T–S fuzzy model are identified. The greedy algorithm is applicable to high-dimensional problems since there is no exponential growth in time or space as the dimensionality increases. The new network architecture and greedy learning algorithm make the proposed DJFNN a regression model of high interpretability and good prediction capability, particularly suitable for solving the high-dimensional problems. The DJFNN was experimented with using a synthetic dataset and 28 real-world datasets and compared with classical and state-of-the-art methods through nonparametric statistical tests. The results confirmed the effectiveness of the DJFNN in terms of accuracy, interpretability, and computational cost.

Mapping network flows provides insight into the organization of networks, but even though many real networks are bipartite, no method for mapping flows takes advantage of the bipartite structure. What do we miss by discarding this information and how can we use it to understand the structure of bipartite networks better? The map equation models network flows with a random walk and exploits the information-theoretic duality between compression and finding regularities to detect communities in networks. However, it does not use the fact that random walks in bipartite networks alternate between node types, information worth 1 bit. To make some or all of this information available to the map equation, we developed a coding scheme that remembers node types at different rates. We explored the community landscape of bipartite real-world networks from no node-type information to full node-type information and found that using node types at a higher rate generally leads to deeper community hierarchies and a higher resolution. The corresponding compression of network flows exceeds the amount of extra information provided. Consequently, taking advantage of the bipartite structure increases the resolution and reveals more network regularities.

To assess the validity of privacy-preserving synthetic data by comparing results from synthetic versus original EHR data analysis. A published retrospective cohort study on real-world effectiveness of COVID-19 vaccines by Maccabi Healthcare Services in Israel was replicated using synthetic data generated from the same source, and the results were compared between synthetic versus original datasets. The endpoints included COVID-19 infection, symptomatic COVID-19 infection and hospitalization due to infection and were also assessed in several demographic and clinical subgroups. In comparing synthetic versus original data estimates, several metrices were utilized: standardized mean differences (SMD), decision agreement, estimate agreement, confidence interval overlap, and Wald test. Synthetic data were generated five times to assess the stability of results. The distribution of demographic and clinical characteristics demonstrated very small difference (< 0.01 SMD). In the comparison of vaccine effectiveness assessed in relative risk reduction between synthetic versus original data, there was a 100% decision agreement, 100% estimate agreement, and a high level of confidence interval overlap (88.7%-99.7%) in all five replicates across all subgroups. Similar findings were achieved in the assessment of vaccine effectiveness against symptomatic COVID-19 Infection. In the comparison of hazard ratios for COVID 19-related hospitalization and odds ratio for symptomatic COVID-19 Infection, the Wald tests suggested no significant difference between respective effect estimates in all five replicates for all patient subgroups but there were disagreements in estimate and decision metrices in some subgroups and replicates. Overall, comparison of synthetic versus original real-world data demonstrated good validity and reliability. Transparency on the process to generate high fidelity synthetic data and assurances of patient privacy are warranted.

Synthetic face datasets for training Face Recognition (FR) models have gained significant interest in recent years due to privacy concerns associated with real facial data. Recent advances in generative techniques, based mainly on Generative Adversarial Networks (GANs) and Diffusion Models, have led to the proliferation of various synthetic datasets. Although the performance of FR models trained on synthetic data is becoming similar to those of models trained on real data when evaluated on standard benchmarks such as LFW, CFP-FP, AgeDB, and CALFW, little is known about their effectiveness in more challenging scenarios involving individuals with high facial similarity, such as doppelgängers (lookalikes) and identical twins. This work addresses this gap by evaluating state-of-the-art FR models trained on real or synthetic datasets across four testing datasets that feature high facial similarity: HDA Doppelgänger, DoppelVer, 3D-TEC, and ND-Twins-2009-2010. We analyze verification performances, including subgroup analysis by ethnicity, gender, and age, to better understand demographic disparities. Our results show that the gap between synthetic and real face datasets is larger in these challenging scenarios than on typical benchmarks. We also observe similar biases in FR models trained on synthetic or real data, suggesting that current synthetic datasets may inherit biases through the use of real data at some stage in their pipeline. Reproducibility code, trained models, and instructions are available at.

BackgroundThe exponential growth in patient data collection by healthcare providers, governments, and private industries is yielding large and diverse datasets that offer new insights into critical medical questions. Leveraging extensive computational resources, Machine Learning and Artificial Intelligence are increasingly utilized to address health-related issues, such as predicting outcomes from Electronic Health Records and detecting patterns in multi-omics data. Despite the proliferation of medical devices based on Artificial Intelligence, data accessibility for research is limited due to privacy concerns. Efforts to de-identify data have met challenges in maintaining effectiveness, particularly with large datasets. As an alternative, synthetic data, that replicate main statistical properties of real patient data, are proposed. However, the lack of standardized evaluation metrics complicates the selection of appropriate synthetic data generation methods. Effective evaluation of synthetic data must consider resemblance, utility and privacy, tailored to specific applications. Despite available metrics, benchmarking efforts remain limited, necessitating further research in this area.ResultsWe present SynthRO (Synthetic data Rank and Order), a user-friendly tool for benchmarking health synthetic tabular data across various contexts. SynthRO offers accessible quality evaluation metrics and automated benchmarking, helping users determine the most suitable synthetic data models for specific use cases by prioritizing metrics and providing consistent quantitative scores. Our dashboard is divided into three main sections: (1) Loading Data section, where users can locally upload real and synthetic datasets; (2) Evaluation section, in which several quality assessments are performed by computing different metrics and measures; (3) Benchmarking section, where users can globally compare synthetic datasets based on quality evaluation.ConclusionsSynthetic data mitigate concerns about privacy and data accessibility, yet lacks standardized evaluation metrics. SynthRO provides an accessible dashboard helping users select suitable synthetic data models, and it also supports various use cases in healthcare, enhancing prognostic scores and enabling federated learning. SynthRO’s accessible GUI and modular structure facilitate effective data evaluation, promoting reliability and fairness. Future developments will include temporal data evaluation, further broadening its applicability.

Bipartite networks that characterize complex relationships among data arise in various domains. The existing bipartite network models are mainly based on a type of relationship between objects, and cannot effectively describe multiple relationships in the real world. In this paper, we propose a multi-relationship bipartite network (MBN) model, which can describe multiple relationships between two types of objects, and realizes simple weighted bipartite network reconstruction. Our model contains three major modules, namely multi-relationship bipartite network modeling (MBNM), multi-relationship aggregation module (MAM) and network reconstruction module (NRM). In MBNM, a multi-relationship bipartite network is proposed to describe multiple relationships between two types of objects. In the MAM, considering that different relationships have different information for the model, we introduce a novel relationship-level attention mechanism, and the aggregation of multiple relationships is carried out through the importance of each relationship. Based on the learning framework, the NRM can learn the potential representations of nodes after multi-relationship aggregation, and design a nonlinear fusion mechanism to reconstruct weighted bipartite network. We conducted extensive experiments on three real-world datasets and the results show that multi-relationship aggregation can effectively improve the performance of the model. In addition, experiments also show that our model can outperform existing competitive baseline method.

Detecting multimodal misinformation, especially in the form of image-text pairs, is crucial.Obtaining large-scale, high-quality real-world fact-checking datasets for training detectors is costly, leading researchers to use synthetic datasets generated by AI technologies.However, the generalizability of detectors trained on synthetic data to real-world scenarios remains unclear due to the distribution gap.To address this, we propose learning from synthetic data for detecting real-world multimodal misinformation through two model-agnostic data selection methods that match synthetic and real-world data distributions.Experiments show that our method enhances the performance of a small MLLM (13B) on real-world fact-checking datasets, enabling it to even surpass GPT-4V (OpenAI, 2023b).

This paper performs comprehensive analysis on datasets for occlusion-aware face segmentation, a task that is crucial for many downstream applications. The collection and annotation of such datasets are time-consuming and labor-intensive. Although some efforts have been made in synthetic data generation, the naturalistic aspect of data remains less explored. In our study, we propose two occlusion generation techniques, Naturalistic Occlusion Generation (NatOcc), for producing high-quality naturalistic synthetic occluded faces; and Random Occlusion Generation (RandOcc), a more general synthetic occluded data generation method (Figure 1). We empirically show the effectiveness and robustness of both methods, even for unseen occlusions. To facilitate model evaluation, we present two high-resolution real-world occluded face datasets with finegrained annotations, RealOcc and RealOcc-Wild, featuring both careful alignment preprocessing and an in-the-wild setting for robustness test. We further conduct a comprehensive analysis on a newly introduced segmentation benchmark, offering insights for future exploration. Our code and dataset are available at https://github.com/kennyvoo/face-occlusion-generation.

Many of the real-world data sets can be portrayed as bipartite networks. Since connections between nodes of the same type are lacking, they need to be inferred. The standard way to do this is by converting the bipartite networks to their monopartite projection. However, this simple approach renders an incomplete representation of all the information in the original network. To this end, we propose a new statistical method to identify the most critical links in the bipartite network projection. Our method takes into account the heterogeneity of node connections. Moreover, it can handle situations where links of different types are present. We compare our method against the state-of-the-art and illustrate the findings with synthetic data and empirical examples of investor and political data.

For the extraction of spatially important regions from mass spectrometry imaging (MSI) data, different clustering methods have been proposed. These clustering methods are based on certain assumptions and use different criteria to assign pixels into different classes. For high-dimensional MSI data, the curse of dimensionality also limits the performance of clustering methods which are usually overcome by pre-processing the data using dimension reduction techniques. In summary, the extraction of spatial patterns from MSI data can be done using different unsupervised methods, but the robust evaluation of clustering results is what is still missing. In this study, we have performed multiple simulations on synthetic and real MSI data to validate the performance of unsupervised methods. The synthetic data were simulated mimicking important spatial and statistical properties of real MSI data. Our simulation results confirmed that K-means clustering with correlation distance and Gaussian Mixture Modeling clustering methods give optimal performance in most of the scenarios. The clustering methods give efficient results together with dimension reduction techniques. From all the dimension techniques considered here, the best results were obtained with the minimum noise fraction (MNF) transform. The results were confirmed on both synthetic and real MSI data. However, for successful implementation of MNF transform the MSI data requires to be of limited dimensions.

Generation of synthetic full-scale burst test data for corroded pipelines using the tabular generative adversarial network