Diffusion Models for Tabular Data Imputation and Synthetic Data Generation

Preserving logical and functional dependencies in synthetic tabular data

High-quality tabular data is a crucial requirement for developing data-driven applications, especially healthcare-related ones, because most of the data nowadays collected in this context is in tabular form. However, strict data protection laws complicates the access to medical datasets. Thus, synthetic data has become an ideal alternative for data scientists and healthcare professionals to circumvent such hurdles. Although many healthcare institutions still use the classical de-identification and anonymization techniques for generating synthetic data, deep learning-based generative models such as generative adversarial networks (GANs) have shown a remarkable performance in generating tabular datasets with complex structures. This paper examines the GANs’ potential and applicability within the healthcare industry, which often faces serious challenges with insufficient training data and patient records sensitivity. We investigate several state-of-the-art GAN-based models proposed for tabular synthetic data generation. Healthcare datasets with different sizes, numbers of variables, column data types, feature distributions, and inter-variable correlations are examined. Moreover, a comprehensive evaluation framework is defined to evaluate the quality of the synthetic records and the viability of each model in preserving the patients’ privacy. The results indicate that the proposed models can generate synthetic datasets that maintain the statistical characteristics, model compatibility and privacy of the original data. Moreover, synthetic tabular healthcare datasets can be a viable option in many data-driven applications. However, there is still room for further improvements in designing a perfect architecture for generating synthetic tabular data.

<p><strong>Because of its wide range of applications, generative artificial intelligence (Generative AI) has received a lot of attention in academia and industry. Despite significant advances in computer vision and natural language processing, the use of Generative AI in tabular data is still underexplored. This gap is especially significant given the prevalence of tabular data as the primary data modality. This thesis seeks to close this gap by focusing on the efficient generation, evaluation, refinement, and application of tabular data, while addressing the challenges inherent in its heterogeneous nature.</strong></p><p>The primary goal of this thesis is to develop efficient algorithms for tabular data synthesis, advancing the field of Generative AI in the tabular domain. To achieve this goal, four specific research objectives have been outlined.</p><p>First, this thesis addresses gaps in evaluation metrics by unifying a framework for the comprehensive and consistent assessment of synthetic data. The proposed framework is designed to meet diverse business requirements across various downstream tasks by incorporating a wide range of advanced metrics. These metrics cover various types of evaluations, including univariate, bivariate, multivariate, cluster, and record-level evaluations. Additionally, standardized visualizations are provided to facilitate qualitative assessments. The results indicate that the proposed evaluation framework not only enables consistent comparisons, rankings, and the selection of different data synthesis approaches but also acts as a valuable tool to promptly assess the reliability of results obtained from synthetic data.</p><p>Second, this thesis presents innovative algorithms for the generation of tabular data, aiming to significantly enhance the quality of data synthesis. One of the contributions is the development of a reversible feature engineering pipeline designed to automatically represent tabular data in an efficient format while also ensuring that the transformed data can be easily converted back to its original format. Additionally, the thesis proposes novel deep learning-based tabular data generation models that are capable of learning the joint distribution of multivariate datasets without relying on predefined distribution assumptions. Experimental results highlight the effectiveness of the proposed algorithms in handling heterogeneous data, demonstrating superior performance in most scenarios within the same training duration when compared to similar alternatives.</p><p>Third, this thesis introduces innovative synthetic data prototype selection algorithms with the goal of refining the generated samples. This approach aims to leverage the advantages of Deep Generative Models, which, once trained, can generate unlimited and diverse synthetic data. By carefully identifying and selecting high-quality samples or removing unrealistic ones, the quality of the synthetic data can be enhanced from a post-processing perspective. Building on this hypothesis, we recognize the iterative nature inherent in the data synthesis procedure, where the processes of data generation, evaluation, and refinement should operate repeatedly in a cyclical flow. During the data generation phase, the evaluation framework assists in identifying potential issues and risks based on use cases, while the refinement (post-processing) step iteratively improves synthetic data in alignment with the evaluation outcomes.</p><p>Last, this thesis applies and validates the proposed models across various domains, including business, healthcare, and government, each requiring distinct downstream tasks. Specifically, we assess our algorithms from the standpoint of data balancing using churn data, evaluate our models focusing on data augmentation with health data, and test our algorithms from the perspective of data representation considering data privacy concerns within sensitive citizen data. These applications serve as practical illustrations, demonstrating the effectiveness and utility of the proposed Generative AI models in real-world scenarios.</p><p>In summary, this thesis makes a substantial contribution to the advancement of Generative AI within the domain of tabular data. It not only presents innovative algorithms and evaluation methods but also introduces practical frameworks applicable to real-world scenarios. The findings demonstrate the considerable capacity of Generative AI to fundamentally transform tabular data in various fields, ultimately leading to improved data availability, quality, and quantity.</p>

<p>High data collection costs and complicated data access regulations increase the demand for synthetic data. Generative Adversarial Networks (GANs) are a novel generative framework with great potential for high quality synthetic data generation. GANs formulate the true distribution of data implicitly, and the success of GANs are often measured based on the similarity of generated data to this true distribution. GANs were originally designed to work with continuous data. However, many important real-world datasets such as medical images involve discontinuous distributions. GAN training for discontinuous distributions is relatively more challenging, as the training procedure often suffers from instability and mode collapse issues. This dissertation focuses on designing novel GAN architectures to generate representative synthetic image data, and proposes new structures to alleviate GANs' mode collapse issue. As part of this thesis, novel applications of image data generation with GANs have been also investigated for important problems arising in the telecommunication industry and medical domain. Specifically, we first explore various GAN structures to generate engineered electromagnetic surfaces. We consider the continuous approximation of the data and explore the capabilities of feed-forward and convolutional GANs for synthetic data generation. Next, we introduce a novel GAN architecture to address the problem of mode collapse in GAN training. The proposed structure incorporates a third network that penalizes the generator for generating low diversity samples. Lastly, we study the challenging problem of object generation in 3D space using GANs, and we propose extensions to existing 3D GAN structures to generate connected 3D volumes. Additionally, we explore a more challenging version of this 3D volume generation problem by generating connected volumes packed with spheres. This research has applications in radiosurgery treatment planning, and the proposed 3D GAN structure can help generate rare, unseen 3D tumor volumes and information on how to treat these tumors. Accordingly, our analysis contributes to overcoming data scarcity issues (e.g., due to privacy considerations) for an important practical problem in the medical domain.</p>

Data-driven models function admirably in solving real-world problems. However, obtaining relevant data is difficult. Also, sometimes more diverse data is needed to identify the limitations of trained Machine Learning models. Creating such data samples based on earlier known metadata is a common practice. However, this process can induce bias in the dataset unknowingly. Generative Adversarial Networks (GAN) based data generation models generate more data based on initial data distribution. Thus, data generation models may reflect bias in the generated synthetic data. In this study, the authors have proposed an interactive synthetic data generation Graphical User Interface (GUI) tool. The tool is equipped with Bias detection and mitigation algorithms which will notify users about the pre-existing bias and provide methods to mitigate it. Similarly, this tool can be used to evaluate synthetic data generated using GAN-based models against fairness metrics. The authors have found that Learning Fair Representation (LFR) bias mitigation method has performed 62% & 17.5% better than Prejudice remover and Disparate impact remover for German Credit & Adult original datasets. These results were concluded based on bias detection metrics such as Statistical Parity Difference (SPD) and Disparate Impact (DI). The proposed data generation tool used with LFR method can reduced SPD metric by 93% on original German Credit data. The authors conclude that both original and synthetic datasets had a bias. Therefore, the fairness level of any dataset should be checked vigilantly.

The increasing use of electronic health records (EHRs) in medical research and AI-driven healthcare necessitates high-fidelity synthetic data that balances patient privacy with statistical and clinical utility. Traditional generative models, such as generative adversarial networks (GANs) and variational autoencoders (VAEs), struggle with mode collapse, limited sample diversity, and difficulties in modelling complex dependencies in high-dimensional tabular data. This study introduces a diffusion model-based approach for generating synthetic EHR data and evaluates its utility in clustering multimorbidity patterns using Dirichlet process mixture models (DPMMs). Denoising diffusion probabilistic models (DDPMs) iteratively refine noise through a structured denoising process, producing diverse, high-fidelity synthetic records. The DPMM framework, a Bayesian nonparametric clustering method, dynamically determines the number of clusters, effectively handling heterogeneous, imbalanced datasets. Model evaluation incorporates statistical similarity measures, feature correlation analysis, privacy risk assessments, and predictive performance metrics. Results demonstrate that DDPM-generated data surpasses GANs and VAEs in fidelity (Jensen–Shannon divergence (JSD) = 0.020, Pearson pairwise correlation (PPC) = 0.94), and privacy preservation (membership inference attack (MIA) Risk = 0.25). DPMM clustering reveals clinically meaningful disease patterns, outperforming traditional clustering models. These findings highlight the potential of diffusion models for privacy-preserving synthetic EHR generation and robust multimorbidity clustering in healthcare analytics.

The use of electronic health record (EHR) tabular data for clustering has garnered global attention for its potential to uncover meaningful clusters, improve patient care and advance healthcare research. Applications of EHR data extend beyond healthcare to include financial modeling, fraud detection, differential diagnosis and rare disease identification. However, critical challenges persist, including privacy concerns, limited access to real EHR data, underexplored generative methods and inadequate validation frameworks. This study aims to generate and validate synthetic EHR tabular data and demonstrate its utility in clustering, addressing existing gaps in privacy, data quality and validation. Advanced deep learning architectures, including Generative Adversarial Networks (GANs), were developed to generate high-quality synthetic datasets. We implemented robust validation frameworks that incorporate fidelity, utility and privacy-preserving mechanisms. Dirichlet Process Mixture Models (DPMMs) used synthetic data to group similar items and measured the differences between real and synthetic data distributions by estimating divergence. GANs effectively replicated complex datasets while balancing fidelity, utility and privacy. Validation confirmed the robustness of DPMMs in identifying latent structures. Cluster analysis revealed significant multimorbidity patterns, facilitating personalized treatment strategies, resource allocation and preventive care. This study demonstrates the transformative potential of synthetic EHR data in clustering, advancing healthcare delivery, planning and research.

Clustering of the single-cell RNA-seq (scRNA-seq) transcriptome profiles is able to identify cell types, which is beneficial to improve the understanding of disease progression. However, in practice, the single-cell expression data often contains a significant number of missing values as a result of technical variability. Missing data is a critical challenge in scRNA-seq clustering analysis since the unknown value does not reflect the underlying true expression level and makes it difficult to discovering cell types by applying clustering algorithms directly. Various approaches have been developed to overcome missing data issue in scRNA-seq clustering. Most of them recover missing expression values by borrowing observed data from similar cells or synthesizing data via generative adversarial networks. Such that the biologically meaningful cluster structure has not been sufficiently exploited. In this work, we introduce ColImpute, a collaborative structure-preserved missing data imputation approach for the scRNA-seq clustering. Specifically, a cluster structure-preserved imputation module and a subspace clustering module, which respectively perform missing data imputation and cell subtypes identification, are integrated into a unified optimization framework to train the two networks in a collaborative manner. Consequently, the clustering module effectively contributes cluster-structure information to guide the trainning process of the missing data imputation module. Simultaneously, the cluster structure-preserved imputation module reciprocally enhances the performance of the clustering module by generating more precise recovered samples. Promising experimental results show that the proposed method is effective for both the data imputation and the cell types identification.

The generation of high-quality, long-sequenced time-series data is essential due to its wide range of applications. In the past, standalone Recurrent and Convolutional Neural Network-based Generative Adversarial Networks (GAN) were used to synthesize time-series data. However, they are inadequate for generating long sequences of time-series data due to limitations in the architecture, such as difficulties in capturing long-range dependencies, limited temporal coherence, and scalability challenges. Furthermore, GANs are well known for their training instability and mode collapse problem. To address this, we propose TransFusion , a diffusion, and transformers-based generative model to generate high-quality long-sequence time-series data. We extended the sequence length to 384, surpassing the previous limit, and successfully generated high-quality synthetic data. Also, we introduce two evaluation metrics to evaluate the quality of the synthetic data as well as its predictive characteristics. TransFusion is evaluated using a diverse set of visual and empirical metrics, consistently outperforming the previous state-of-the-art by a significant margin.

Widespread adoption of artificial intelligence (AI) in civilian and defense government agencies requires the stakeholders to have trust in AI solutions. One of the five principles of ethical AI, identified by the Department of Defense, emphasizes that AI solutions be equitable. The AI system involves a series of choices from data selection to model definition, each of which is subject to human and algorithmic biases and can lead to unintended consequences. This paper focuses on allowing AI bias mitigation with the use of synthetic data. The proposed technique, named Fair-GAN, builds upon the recently developed Fair-SMOTE approach, which used synthesized data to fix class and other imbalances caused by protected attributes such as race and gender. Fair-GAN uses Generative Adversarial Networks (GAN) instead of the Synthetic Minority Oversampling Technique (SMOTE). While SMOTE can only synthesize tabular and numerical data, GAN can synthesize tabular data with numerical, binary, and categorical variables. GAN can also synthesize other data forms such as images, audio and text. In our experiments, we use the Synthetic Data Vault (SDV), which implements approaches such as conditional tabular GAN (CTGAN) and tabular variational autoencoders (TVAE). We show the applicability of Fair-GAN to several benchmark problems, which are used to evaluate the efficacy of AI bias mitigation algorithms. It is shown that Fair-GAN leads to significant improvements in metrics used for evaluating AI fairness such as the statistical parity difference, disparate impact, average odds difference, and equal opportunities difference.

Synthetic data generation is a strategy used to address the lack and complex process to acquire clinical data and information, in particular in type 2 diabetes mellitus (T2DM) research. T2DM is characterized by chronic hyperglycemia with macrovascular and microvascular complications. Nevertheless, despite the importance of data to improve diagnostic accuracy, better treatments, and personalized patient care, medical datasets are often restricted by ethical and privacy constraints. In this sense, this chapter evaluates four synthetic data generation techniques, Gaussian Mixture Models (GMM), Generative Adversarial Networks (GAN), Wasserstein GAN (WGAN), and Variational Autoencoders (VAE). The quality of the generated data was assessed through statistical divergence metrics-specifically Jensen-Shannon (JSD) and Kullback-Leibler (KLD)and by analyzing their impact on classification performance. The results indicate that GMM achieved the lowest JSD, showing the best overall distributional similarity, while WGAN obtained the lowest KLD, suggesting a closer alignment in information content with real data. Additionally, GAN and WGAN demonstrated the highest predictive performance in classification tasks, indicating that they better preserved essential relationships within the data. These findings confirm that generative strategies of using synthetic data to improve T2DM research are feasible, offering an alternative to develop diagnosis tools without compromising patient confidentiality. It is possible to conclude that the generation method selection depends on the type of data and research objective, maximizing statistical similarity, optimizing performance, or balancing both aims. Synthetic data generation approaches represent a feasible approach to expand balanced and quality datasets to advance in personalized healthcare for diabetes patients.

Electronic Health Records often suffer from missing data, which poses a major problem in clinical practice and clinical studies. A novel approach for dealing with missing data are Generative Adversarial Nets (GANs), which have been generating huge research interest in image generation and transformation. Recently, researchers have attempted to apply GANs to missing data generation and imputation for EHR data: a major challenge here is the categorical nature of the data. State-of-the-art solutions to the GAN-based generation of categorical data involve either reinforcement learning, or learning a bidirectional mapping between the categorical and the real latent feature space, so that the GANs only need to generate real-valued features. However, these methods are designed to generate complete feature vectors instead of imputing only the subsets of missing features. In this paper we propose a simple and yet effective approach that is based on previous work on GANs for data imputation. We first motivate our solution by discussing the reason why adversarial training often fails in case of categorical features. Then we derive a novel way to re-code the categorical features to stabilize the adversarial training. Based on experiments on two real-world EHR data with multiple settings, we show that our imputation approach largely improves the prediction accuracy, compared to more traditional data imputation approaches.

This study presents a novel approach using Wasserstein Generative Adversarial Networks with Gradient Penalty (WGAN-GP) to generate synthetic electroencephalography (EEG) and electrocardiogram (ECG) waveforms. The synthetic EEG data represent concentration and relaxation mental states, while the synthetic ECG data correspond to normal and abnormal states. By addressing the challenges of limited biophysical data, including privacy concerns and restricted volunteer availability, our model generates realistic synthetic waveforms learned from real data. Combining real and synthetic datasets improved classification accuracy from 92% to 98.45%, highlighting the benefits of dataset augmentation for machine learning performance. The WGAN-GP model achieved 96.84% classification accuracy for synthetic EEG data representing relaxation states and optimal accuracy for concentration states when classified using a fusion of convolutional neural networks (CNNs). A 50% combination of synthetic and real EEG data yielded the highest accuracy of 98.48%. For EEG signals, the real dataset consisted of 60-s recordings across four channels (TP9, AF7, AF8, and TP10) from four individuals, providing approximately 15,000 data points per subject per state. For ECG signals, the dataset contained 1200 real samples, each comprising 140 data points, representing normal and abnormal states. WGAN-GP outperformed a basic generative adversarial network (GAN) in generating reliable synthetic data. For ECG data, a support vector machine (SVM) classifier achieved an accuracy of 98% with real data and 95.8% with synthetic data. Synthetic ECG data improved the random forest (RF) classifier’s accuracy from 97% with real data alone to 98.40% when combined with synthetic data. Statistical significance was assessed using the Wilcoxon signed-rank test, demonstrating the robustness of the WGAN-GP model. Techniques such as discrete wavelet transform, downsampling, and upsampling were employed to enhance data quality. This method shows significant potential in addressing biophysical data scarcity and advancing applications in assistive technologies, human-robot interaction, and mental health monitoring, among other medical applications.

Generation of synthetic data with low-dimensional features for condition monitoring utilizing Generative Adversarial Networks

As the demand for data increases for generative artificial intelligence, many organizations have begun to use generative data to train their models. This increase in demand raises concerns regarding model collapse, a phenomenon where model performance declines when trained on AI-generated data. This study aims to analyze the effect of contamination on three image generation models and observe each model’s resiliency toward model collapse: Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), and Denoising Diffusion Implicit Models (DDIMs). Each model was trained on a CIFAR-10 dataset with contamination ratios of 0%, 0.2%, 2%, and 20%, and the datasets were contaminated with generated images from a pre-trained Stable Diffusion model. Each model was evaluated under identical conditions and evaluated using Fréchet Inception Distance (FID), Inception Score (IS), Kernel Inception Distance (KID), precision, and recall. Results indicate that synthetic data contamination leads to performance degradation across all models in different ways. GAN exhibited the fastest and most severe degradation. VAE demonstrated high resilience in maintaining color semantics and metric stability, at the cost of reduced diversity. DDIM-based diffusion models maintain object structural coherence but suffer significant color corruption. These findings emphasize the value of controlling synthetic data usage in generative model training to ensure the long-term reliability of models.