Differentially private synthetic mixed-type data generation for unsupervised learning

<p>High data collection costs and complicated data access regulations increase the demand for synthetic data. Generative Adversarial Networks (GANs) are a novel generative framework with great potential for high quality synthetic data generation. GANs formulate the true distribution of data implicitly, and the success of GANs are often measured based on the similarity of generated data to this true distribution. GANs were originally designed to work with continuous data. However, many important real-world datasets such as medical images involve discontinuous distributions. GAN training for discontinuous distributions is relatively more challenging, as the training procedure often suffers from instability and mode collapse issues. This dissertation focuses on designing novel GAN architectures to generate representative synthetic image data, and proposes new structures to alleviate GANs' mode collapse issue. As part of this thesis, novel applications of image data generation with GANs have been also investigated for important problems arising in the telecommunication industry and medical domain. Specifically, we first explore various GAN structures to generate engineered electromagnetic surfaces. We consider the continuous approximation of the data and explore the capabilities of feed-forward and convolutional GANs for synthetic data generation. Next, we introduce a novel GAN architecture to address the problem of mode collapse in GAN training. The proposed structure incorporates a third network that penalizes the generator for generating low diversity samples. Lastly, we study the challenging problem of object generation in 3D space using GANs, and we propose extensions to existing 3D GAN structures to generate connected 3D volumes. Additionally, we explore a more challenging version of this 3D volume generation problem by generating connected volumes packed with spheres. This research has applications in radiosurgery treatment planning, and the proposed 3D GAN structure can help generate rare, unseen 3D tumor volumes and information on how to treat these tumors. Accordingly, our analysis contributes to overcoming data scarcity issues (e.g., due to privacy considerations) for an important practical problem in the medical domain.</p>

High-quality tabular data is a crucial requirement for developing data-driven applications, especially healthcare-related ones, because most of the data nowadays collected in this context is in tabular form. However, strict data protection laws complicates the access to medical datasets. Thus, synthetic data has become an ideal alternative for data scientists and healthcare professionals to circumvent such hurdles. Although many healthcare institutions still use the classical de-identification and anonymization techniques for generating synthetic data, deep learning-based generative models such as generative adversarial networks (GANs) have shown a remarkable performance in generating tabular datasets with complex structures. This paper examines the GANs’ potential and applicability within the healthcare industry, which often faces serious challenges with insufficient training data and patient records sensitivity. We investigate several state-of-the-art GAN-based models proposed for tabular synthetic data generation. Healthcare datasets with different sizes, numbers of variables, column data types, feature distributions, and inter-variable correlations are examined. Moreover, a comprehensive evaluation framework is defined to evaluate the quality of the synthetic records and the viability of each model in preserving the patients’ privacy. The results indicate that the proposed models can generate synthetic datasets that maintain the statistical characteristics, model compatibility and privacy of the original data. Moreover, synthetic tabular healthcare datasets can be a viable option in many data-driven applications. However, there is still room for further improvements in designing a perfect architecture for generating synthetic tabular data.

This study presents a novel approach using Wasserstein Generative Adversarial Networks with Gradient Penalty (WGAN-GP) to generate synthetic electroencephalography (EEG) and electrocardiogram (ECG) waveforms. The synthetic EEG data represent concentration and relaxation mental states, while the synthetic ECG data correspond to normal and abnormal states. By addressing the challenges of limited biophysical data, including privacy concerns and restricted volunteer availability, our model generates realistic synthetic waveforms learned from real data. Combining real and synthetic datasets improved classification accuracy from 92% to 98.45%, highlighting the benefits of dataset augmentation for machine learning performance. The WGAN-GP model achieved 96.84% classification accuracy for synthetic EEG data representing relaxation states and optimal accuracy for concentration states when classified using a fusion of convolutional neural networks (CNNs). A 50% combination of synthetic and real EEG data yielded the highest accuracy of 98.48%. For EEG signals, the real dataset consisted of 60-s recordings across four channels (TP9, AF7, AF8, and TP10) from four individuals, providing approximately 15,000 data points per subject per state. For ECG signals, the dataset contained 1200 real samples, each comprising 140 data points, representing normal and abnormal states. WGAN-GP outperformed a basic generative adversarial network (GAN) in generating reliable synthetic data. For ECG data, a support vector machine (SVM) classifier achieved an accuracy of 98% with real data and 95.8% with synthetic data. Synthetic ECG data improved the random forest (RF) classifier’s accuracy from 97% with real data alone to 98.40% when combined with synthetic data. Statistical significance was assessed using the Wilcoxon signed-rank test, demonstrating the robustness of the WGAN-GP model. Techniques such as discrete wavelet transform, downsampling, and upsampling were employed to enhance data quality. This method shows significant potential in addressing biophysical data scarcity and advancing applications in assistive technologies, human-robot interaction, and mental health monitoring, among other medical applications.

In terms of electric vehicles (EVs), electric kickboards are crucial elements of smart transportation networks for short-distance travel that is risk-free, economical, and environmentally friendly. Forecasting the daily demand can improve the local service provider's access to information and help them manage their short-term supply more effectively. This study developed the forecasting model using real-time data and weather information from Jeju Island, South Korea. Cluster analysis under the rental pattern of the electric kickboard is a component of the forecasting processes. We cannot achieve noticeable results at first because of the low amount of training data. We require a lot of data to produce a solid prediction result. For the sake of the subsequent experimental procedure, we created synthetic time-series data using a generative adversarial networks (GAN) approach and combined the synthetic data with the original data. The outcomes have shown how the GAN-based synthetic data generation approach has the potential to enhance prediction accuracy. We employ an ensemble model to improve prediction results that cannot be achieved using a single regressor model. It is a weighted combination of several base regression models to one meta-regressor. To anticipate the daily demand in this study, we create an ensemble model by merging three separate base machine learning algorithms, namely CatBoost, Random Forest (RF), and Extreme Gradient Boosting (XGBoost). The effectiveness of the suggested strategies was assessed using some evaluation indicators. The forecasting outcomes demonstrate that mixing synthetic data with original data improves the robustness of daily demand forecasting and outperforms other models by generating more agreeable values for suggested assessment measures. The outcomes further show that applying ensemble techniques can reasonably increase the forecasting model's accuracy for daily electric kickboard demand.

Generation of synthetic data with low-dimensional features for condition monitoring utilizing Generative Adversarial Networks

Training data is an essential ingredient within supervised learning, but time consuming, expensive and for some applications impossible to acquire. A possible solution is to use synthetic training data. However, the domain shift of synthetic data makes it challenging to obtain good results when used as training data for deep learning models. It is therefore of interest to refine synthetic data, e.g. using image-to-image translation, to improve results. The aim of this work is to compare different methods to do image-to-image translation of synthetic training data of thermal IR-images using generative adversarial networks (GANs). Translation is done both using synthetic thermal IR-images alone, as well as including pixelwise depth and/or semantic information. To evaluate, we propose a new measure based on the Frechet Inception Distance, adapted to work for thermal IR-images. We show that by adapting a GAN model to also include corresponding pixelwise depth data to each synthetic IR-image, the performance is improved compared to using only IR-images.

Accurate disease phenotype prediction plays an important role in the treatment of heterogeneous diseases like cancer in the era of precision medicine. With the advent of high throughput technologies, more comprehensive multi-omics data is now available that can effectively link the genotype to phenotype. However, the interactive relation of multi-omics datasets makes it particularly challenging to incorporate different biological layers to discover the coherent biological signatures and predict phenotypic outcomes. In this study, we introduce omicsGAN, a generative adversarial network model to integrate two omics data and their interaction network. The model captures information from the interaction network as well as the two omics datasets and fuse them to generate synthetic data with better predictive signals. Large-scale experiments on The Cancer Genome Atlas breast cancer, lung cancer and ovarian cancer datasets validate that (i) the model can effectively integrate two omics data (e.g. mRNA and microRNA expression data) and their interaction network (e.g. microRNA-mRNA interaction network). The synthetic omics data generated by the proposed model has a better performance on cancer outcome classification and patients survival prediction compared to original omics datasets. (ii) The integrity of the interaction network plays a vital role in the generation of synthetic data with higher predictive quality. Using a random interaction network does not allow the framework to learn meaningful information from the omics datasets; therefore, results in synthetic data with weaker predictive signals. Source code is available at: https://github.com/CompbioLabUCF/omicsGAN. Supplementary data are available at Bioinformatics online.

BackgroundPrivacy restrictions limit access to protected patient-derived health information for research purposes. Consequently, data anonymization is required to allow researchers data access for initial analysis before granting institutional review board approval. A system installed and activated at our institution enables synthetic data generation that mimics data from real electronic medical records, wherein only fictitious patients are listed.ObjectiveThis paper aimed to validate the results obtained when analyzing synthetic structured data for medical research. A comprehensive validation process concerning meaningful clinical questions and various types of data was conducted to assess the accuracy and precision of statistical estimates derived from synthetic patient data.MethodsA cross-hospital project was conducted to validate results obtained from synthetic data produced for five contemporary studies on various topics. For each study, results derived from synthetic data were compared with those based on real data. In addition, repeatedly generated synthetic datasets were used to estimate the bias and stability of results obtained from synthetic data.ResultsThis study demonstrated that results derived from synthetic data were predictive of results from real data. When the number of patients was large relative to the number of variables used, highly accurate and strongly consistent results were observed between synthetic and real data. For studies based on smaller populations that accounted for confounders and modifiers by multivariate models, predictions were of moderate accuracy, yet clear trends were correctly observed.ConclusionsThe use of synthetic structured data provides a close estimate to real data results and is thus a powerful tool in shaping research hypotheses and accessing estimated analyses, without risking patient privacy. Synthetic data enable broad access to data (eg, for out-of-organization researchers), and rapid, safe, and repeatable analysis of data in hospitals or other health organizations where patient privacy is a primary value.

Traditional datasets are often limited, which can affect the accuracy of analyses. Additionally, the use of students’ real data raises privacy concerns. Generative Adversarial Networks (GANs) offer a solution by generating synthetic data that closely mirrors real-world data without compromising sensitive information. The research explored the application of GANs to enhance student performance datasets by addressing challenges related to data scarcity and privacy in educational research. In the research, GANs were utilized to generate synthetic student performance data. The accuracy of the data was assessed using Mean Absolute Percentage Error (MAPE), with values ranging from 0.004% to 19.92% across various statistical measures and means. These results demonstrated the reliability of the synthetic data, making it suitable for further analysis. The synthetic datasets were then analyzed using the Apriori algorithm, a well-known method in data mining for discovering significant patterns and relationships. A lower bound minimum support of 0.1 (10%) and a minimum confidence threshold of 0.6 (60%) were applied, ensuring the identification of meaningful associations. The analysis reveals important patterns and relationships among student attributes and behaviors. The research highlights the potential of GANs to advance data-driven educational research. By generating high-quality synthetic data, GANs allow researchers to conduct comprehensive analyses while addressing privacy concerns. The research contributes to the methodological approach to data augmentation in education, offering new opportunities for ethical and robust research.

Data-driven models function admirably in solving real-world problems. However, obtaining relevant data is difficult. Also, sometimes more diverse data is needed to identify the limitations of trained Machine Learning models. Creating such data samples based on earlier known metadata is a common practice. However, this process can induce bias in the dataset unknowingly. Generative Adversarial Networks (GAN) based data generation models generate more data based on initial data distribution. Thus, data generation models may reflect bias in the generated synthetic data. In this study, the authors have proposed an interactive synthetic data generation Graphical User Interface (GUI) tool. The tool is equipped with Bias detection and mitigation algorithms which will notify users about the pre-existing bias and provide methods to mitigate it. Similarly, this tool can be used to evaluate synthetic data generated using GAN-based models against fairness metrics. The authors have found that Learning Fair Representation (LFR) bias mitigation method has performed 62% & 17.5% better than Prejudice remover and Disparate impact remover for German Credit & Adult original datasets. These results were concluded based on bias detection metrics such as Statistical Parity Difference (SPD) and Disparate Impact (DI). The proposed data generation tool used with LFR method can reduced SPD metric by 93% on original German Credit data. The authors conclude that both original and synthetic datasets had a bias. Therefore, the fairness level of any dataset should be checked vigilantly.

This research examines the utilization of Generative Adversarial Networks (GANs) to produce synthetic financial data that ensures privacy while adhering to stringent regulatory frameworks, such as the General Data Protection Regulation (GDPR) (European Union, 2016) [4] and the California Consumer Privacy Act (CCPA). Financial institutions handle extensive sensitive data, necessitating stringent privacy safeguards. Conventional anonymization techniques frequently reduce data utility, thereby limiting their effectiveness for machine learning, research, and analysis. Conversely, GANs offer an innovative alternative by generating realistic synthetic datasets that maintain the statistical properties of original data without containing personally identifiable information. This paper introduces a GAN-based framework designed specifically for financial data, enhanced with differential privacy (Dwork et al., 2006) [3] to provide robust privacy assurances. Through evaluation on real-world financial datasets, the framework demonstrates its capability to generate high-quality synthetic data applicable in areas such as fraud detection and customer segmentation. Findings suggest that this approach effectively maintains a balance between privacy and utility, presenting a scalable solution for financial institutions to leverage data while ensuring compliance with legal mandates. This study advances privacy-preserving data generation and delivers actionable insights for the financial sector.

Synthetic data is a promising tool for improving the accessibility of datasets which are too sensitive to be shared publicly.To this end, we introduce metasyn, a Python package for generating synthetic data from tabular datasets.Unlike existing synthetic data generation software, metasyn is built on a simple generative model that omits multivariate information.This choice enables transparency and auditability, keeps information leakage to a minimum, and enables privacy guarantees through a plug-in system.While the analytical validity of the generated data is thus intentionally limited, its potential uses are broad, including exploratory analyses, code development and testing, and external communication and teaching (van Kesteren, 2024).

e19554 Background: Adding daratumumab to standard multiple myeloma (MM) regimens improves response rates and clinical outcomes. Systematic reviews (meta-analyses) allow further evaluation of its effectiveness in MM patients with high-risk cytogenetic factors but are limited by model assumptions. his study addresses these limitations using a novel synthetic individual patient data (SynIPD) generation algorithm to enable a more robust analysis. Methods: Advances in AI have enabled the creation of synthetic data that mimics the statistical properties of real patient data, empowering aggregated analyses of clinical trials. This study focuses on newly diagnosed MM (NDMM) and relapsed/refractory MM (RRMM) patients with high-risk cytogenetic profiles. Six studies were included: ALCYONE, CASSIOPEIA, and MAIA for NDMM, and CANDOR, CASTOR, and POLLUX for RRMM. Large language models extracted clinical evidence, and SynIPD was generated, incorporating cytogenetic profiles. Random-effects meta-analyses were conducted, and results were compared with pooled analyses using synthetic data. Results: Synthetic data were generated for three studies for NDMM (ALCYONE, CASSIOPEIA, MAIA) and three studies for RRMM (CANDOR, CASTOR, POLLUX). The generated hazard ratios (HRs) for each study closely aligned with reported HRs for high-risk patients, with a maximum relative error of less than 2%. By pooling these datasets, we obtained synthetic datasets for NDMM and RRMM patients. Stratified Cox regression analyses yielded HRs (95% confidence interval) of 0.764 (0.529-1.104) for NDMM and 0.431 (0.294-0.613) for RRMM. Random-effects meta-analyses showed HRs (95% confidence interval) of 0.764 (0.529-1.104) for NDMM and 0.425 (0.287-0.621) for RRMM. Compared with traditional meta-analysis techniques, the synthetic data approach provides additional clinical insights that were not available in the original publications. For instance, the clinical results for the three NDMM studies are summarized as KM curves in Figure 1, which were not reported in the original studies. Conclusions: Our results suggest that the addition of daratumumab to standard regimens improves PFS for RRMM patients with high-risk cytogenetic factors, although its benefit for NDMM patients is not statistically significant. Pooling synthetic data enhances the robustness of conventional meta-analysis outcomes. The identification of HRs and corresponding confidence intervals through SynIPD improves the credibility of our proposed method, offering a reliable approach to evaluating the efficacy of daratumumab in high-risk patients. Since SynIPD includes individual covariate information, it provides deep clinical insights beyond merely enhancing meta-analysis. In other words, any statistical results derived from the original IPD can be effectively approximated using SynIPD.

Generation of synthetic full-scale burst test data for corroded pipelines using the tabular generative adversarial network

Hybrid microdata using microaggregation