Evaluating the Impact of Health Care Data Completeness for Deep Generative Models.

Automated design synthesis has the potential to revolutionize the modern engineering design process and improve access to highly optimized and customized products across countless industries. Successfully adapting generative machine learning to design engineering may enable such automated design synthesis and is a research subject of great importance. We present a review and analysis of deep generative machine learning models in engineering design. Deep generative models (DGMs) typically leverage deep networks to learn from an input dataset and synthesize new designs. Recently, DGMs such as feedforward neural networks (NNs), generative adversarial networks (GANs), variational autoencoders (VAEs), and certain deep reinforcement learning (DRL) frameworks have shown promising results in design applications like structural optimization, materials design, and shape synthesis. The prevalence of DGMs in engineering design has skyrocketed since 2016. Anticipating the continued growth, we conduct a review of recent advances to benefit researchers interested in DGMs for design. We structure our review as an exposition of the algorithms, datasets, representation methods, and applications commonly used in the current literature. In particular, we discuss key works that have introduced new techniques and methods in DGMs, successfully applied DGMs to a design-related domain, or directly supported the development of DGMs through datasets or auxiliary methods. We further identify key challenges and limitations currently seen in DGMs across design fields, such as design creativity, handling constraints and objectives, and modeling both form and functional performance simultaneously. In our discussion, we identify possible solution pathways as key areas on which to target the future work.

The design and operation of modern energy systems are heavily influenced by time-dependent and uncertain parameters, e.g., renewable electricity generation, load-demand, and electricity prices. These are typically represented by a set of discrete realizations known as scenarios. A popular scenario generation approach uses deep generative models (DGM) that allow scenario generation without prior assumptions about the data distribution. However, the validation of generated scenarios is difficult, and a comprehensive discussion about appropriate validation methods is currently lacking. To start this discussion, we provide a critical assessment of the currently used validation methods in the energy scenario generation literature. In particular, we assess validation methods based on probability density, auto-correlation, and power spectral density. Furthermore, we propose using the multifractal detrended fluctuation analysis (MFDFA) as an additional validation method for non-trivial features like peaks, bursts, and plateaus. As representative examples, we train generative adversarial networks (GANs), Wasserstein GANs (WGANs), and variational autoencoders (VAEs) on two renewable power generation time series (photovoltaic and wind from Germany in 2013 to 2015) and an intra-day electricity price time series form the European Energy Exchange in 2017 to 2019. We apply the four validation methods to both the historical and the generated data and discuss the interpretation of validation results as well as common mistakes, pitfalls, and limitations of the validation methods. Our assessment shows that no single method sufficiently characterizes a scenario but ideally validation should include multiple methods and be interpreted carefully in the context of scenarios over short time periods.

Editors’ Summary Raquel Bernal, Marcela Eslava, Ugo Panizza, and Roberto Rigobón This issue of Economía consists of four papers. The first paper by Gladys López-Acevedo and Mónica Tinajero-Bravo looks at the effects of several programs aimed at small and medium-sized enterprises in Mexico on firm performance in the long term. The second paper by Eduardo Lora and Johanna Fajardo compares perception of one’s social ranking with objective measures of social ranking to find that there is a large difference between the two, partly because individuals incorporate a wide variety of wealth measures in their assessment apart from income. Verónica Amarante, Mery Ferrando, and Andrea Vigorito study the effects of conditional cash transfers on school attendance and child labor among teenagers in Uruguay in the third paper. Finally, Rodrigo Taborda discusses possible media bias in the reporting of exchange rate–related news during revaluation episodes in comparison with reporting during devaluation episodes in the fourth and last paper of this volume. In the first paper, Gladys López-Acevedo and Mónica Tinajero-Bravo estimate the effects of a variety of support programs aimed at small and medium-sized enterprises in Mexico by using a novel ten-year panel data set of firm-level data from 1994 to 2005. These programs comprise primarily business development services such as consulting services, training programs for workers, quality control practices, and export promotion, among others, and programs that support R&D activities. Rigorous evaluations of such programs are scarce, particularly in developing countries, owing to lack of appropriate data sets that contain longitudinal information and allow some correction for self-selection into program participation. López-Acevedo and Tinajero-Bravo link a firm panel data set with program participation information from a different source to estimate treatment effects by using fixed-effects models. The longitudinal dimension of the data proves to be extremely useful in allowing identification of effects that take a long time to realize after program participation begins. [End Page vii] The results indicate that three of the four programs for small and mediumsized enterprises assessed have been successful in improving firm outcomes such as sales, value added per worker, and exports. Programs that promote R&D activities have been particularly effective in these dimensions. However, these effects only show after three to four years following the start of program participation, so they are undetectable in cross-section data sets or panel data sets with a shorter span. In some cases, effects continue to increase (as much as threefold) from year 3 to year 10 of program participation. Alessandro Maffioli highlights several contributions of the paper. He notes that the construction of a long-term panel data set allows López-Acevedo and Tinajero-Bravo to address a variety of questions unanswered in the literature, owing to lack of appropriate data. However, Maffioli also indicates that it would have been extremely useful to understand not only individual program effects but also complementarities among them. In particular, the possibility of understanding coordination and sequencing of different policy instruments available for small and medium-sized enterprises is important for policy design. In the second paper, Eduardo Lora and Johanna Fajardo use a rich data set for sixteen Latin American countries to assess the gap between perceived social ranking and objective social ranking based on income measures. After showing that the gap is large, the authors proceed to explore the factors that most explain this difference. By using the usual definitions of middle class (income ranges from US$2 to US$13 per day or from 0.5 to 1.5 times the median income), the authors show that only 69 percent of middle-income individuals classify themselves as such according to the first definition and only about 45 percent do so according to the second one. The authors indicate that individuals consider a wide variety of measures of wealth apart from income when assessing their social ranking. These include human capital (education), access to financial services, access to public utilities, and ownership of durable goods such as a car, washing machine, television, or computer, and they also consider the presence of children in the household...

Synthetic data offers a promising avenue for addressing privacy, scarcity, and fairness challenges in healthcare datasets. However, there is limited evaluation of how different generation methods balance fidelity, utility, and fairness, particularly for underrepresented subgroups. This study addresses this gap by comparing representative generative modelling techniques, both probabilistic and deep approaches, that are popular in the research literature. We empirically evaluate BayesBoost, CTGAN, TVAE, CopulaGAN, and DECAF on two healthcare datasets containing numerical, binary, and categorical features. Each model’s performance is assessed along three axes: data fidelity, machine learning utility, and fairness, using Accuracy Parity, Equalised Odds, and Predictive Rate Parity. Results show that BayesBoost consistently achieved superior fidelity, utility, and fairness preservation, particularly when paired with Random Forest classifiers, achieving around 60 – 63 % higher downstream utility than GAN-based deep generative baselines (e.g., Random Forest accuracy up to 0.88 with BayesBoost versus 0.54 to − 0.55 for GAN-based methods). Deep generative models, while effective in capturing complex structures, often degraded fairness, especially for underrepresented groups, with equalised odds deviating by over 100 % from the ideal parity value of 1.0 in some settings. The Variational Autoencoder outperformed other deep generative models in fairness preservation, especially for equalised odds, although with some reduction in fidelity and utility. Overall, these findings suggest that synthetic data generation for healthcare must move beyond fidelity evaluations to explicitly assess fairness and subgroup impacts, with probabilistic models such as BayesBoost showing strong potential for ethical deployment, while deep generative models require further adaptation for fairness-sensitive applications.

A method for evaluating deep generative models of images for hallucinations in high-order spatial context

Deep generative models have gained much attention given their ability to generate data for applications as varied as healthcare to financial technology to surveillance, and many more - the most popular models being generative adversarial networks (GANs) and variational auto-encoders (VAEs). Yet, as with all machine learning models, ever is the concern over security breaches and privacy leaks and deep generative models are no exception. In fact, these models have advanced so rapidly in recent years that work on their security is still in its infancy. In an attempt to audit the current and future threats against these models, and to provide a roadmap for defense preparations in the short term, we prepared this comprehensive and specialized survey on the security and privacy preservation of GANs and VAEs. Our focus is on the inner connection between attacks and model architectures and, more specifically, on five components of deep generative models: the training data, the latent code, the generators/decoders of GANs/VAEs, the discriminators/encoders of GANs/VAEs, and the generated data. For each model, component and attack, we review the current research progress and identify the key challenges. The paper concludes with a discussion of possible future attacks and research directions in the field.

High-quality tabular data is a crucial requirement for developing data-driven applications, especially healthcare-related ones, because most of the data nowadays collected in this context is in tabular form. However, strict data protection laws complicates the access to medical datasets. Thus, synthetic data has become an ideal alternative for data scientists and healthcare professionals to circumvent such hurdles. Although many healthcare institutions still use the classical de-identification and anonymization techniques for generating synthetic data, deep learning-based generative models such as generative adversarial networks (GANs) have shown a remarkable performance in generating tabular datasets with complex structures. This paper examines the GANs’ potential and applicability within the healthcare industry, which often faces serious challenges with insufficient training data and patient records sensitivity. We investigate several state-of-the-art GAN-based models proposed for tabular synthetic data generation. Healthcare datasets with different sizes, numbers of variables, column data types, feature distributions, and inter-variable correlations are examined. Moreover, a comprehensive evaluation framework is defined to evaluate the quality of the synthetic records and the viability of each model in preserving the patients’ privacy. The results indicate that the proposed models can generate synthetic datasets that maintain the statistical characteristics, model compatibility and privacy of the original data. Moreover, synthetic tabular healthcare datasets can be a viable option in many data-driven applications. However, there is still room for further improvements in designing a perfect architecture for generating synthetic tabular data.

Localization in reverberant environments remains an open challenge. Recently, supervised learning approaches have demonstrated very promising results in addressing reverberation. However, even with large data volumes, the number of labels available for supervised learning in such environments is usually small. We propose to address this issue with a semi-supervised learning (SSL) approach, based on deep generative modeling. Our chosen deep generative model, the variational autoencoder (VAE), is trained to generate the phase of relative transfer functions (RTFs) between microphones. In parallel, a direction of arrival (DOA) classifier network based on RTF-phase is also trained. The joint generative and discriminative model, deemed VAE-SSL, is trained using labeled and unlabeled RTF-phase sequences. In learning to generate and classify the sequences, the VAE-SSL extracts the physical causes of the RTF-phase (i.e., source location) from distracting signal characteristics such as noise and speech activity. This facilitates effective end-to-end operation of the VAE-SSL, which requires minimal preprocessing of RTF-phase. VAE-SSL is compared with two signal processing-based approaches, steered response power with phase transform (SRP-PHAT) and MUltiple SIgnal Classification (MUSIC), as well as fully supervised CNNs. The approaches are compared using data from two real acoustic environments - one of which was recently obtained at Technical University of Denmark specifically for our study. We find that VAE-SSL can outperform the conventional approaches and the CNN in label-limited scenarios. Further, the trained VAE-SSL system can generate new RTF-phase samples which capture the physics of the acoustic environment. Thus, the generative modeling in VAE-SSL provides a means of interpreting the learned representations. To the best of our knowledge, this paper presents the first approach to modeling the physics of acoustic propagation using deep generative modeling.

Deep convolutional neural networks (CNNs) have proven highly effective for visual recognition, where learning a universal representation from activations of convolutional layer plays a fundamental problem. In this paper, we present Fisher Vector encoding with Variational Auto-Encoder (FV-VAE), a novel deep architecture that quantizes the local activations of convolutional layer in a deep generative model, by training them in an end-to-end manner. To incorporate FV encoding strategy into deep generative models, we introduce Variational Auto-Encoder model, which steers a variational inference and learning in a neural network which can be straightforwardly optimized using standard stochastic gradient method. Different from the FV characterized by conventional generative models (e.g., Gaussian Mixture Model) which parsimoniously fit a discrete mixture model to data distribution, the proposed FV-VAE is more flexible to represent the natural property of data for better generalization. Extensive experiments are conducted on three public datasets, i.e., UCF101, ActivityNet, and CUB-200-2011 in the context of video action recognition and fine-grained image classification, respectively. Superior results are reported when compared to state-of-the-art representations. Most remarkably, our proposed FV-VAE achieves to-date the best published accuracy of 94.2% on UCF101.

This paper contributes to the field of deep generative learning applied to solar photovoltaic (PV) synthetic data generation problems by exploring Deep Generative Model (DGM) that combines Variational Autoencoders (VAE) and Generative Adversarial Networks (GAN), i.e., VAEGAN. We build upon knowledge in the area of deep learning to incorporate our Hybrid Deep Neural Network (HDNN), combining convolutional and Long Short-Term Memory (LSTM) layers at the encoding level for producing robust latent representations and subsequently high-quality synthetic PV data samples. The major advantage of these approaches is that it allows the DGMs to perform better feature extraction as well as to capture the historical trends in data effectively. The simulations on actual data acquired from a real PV system demonstrate the effectiveness of the DGMs to produce high-quality samples for multiple seasons of the year.

Accurate forecasting of climate extremes such as floods, heatwaves, and severe storms is vital for risk mitigation and climate resilience planning. Traditional statistical models often fail to capture the nonlinear dynamics and tail dependencies inherent in such events. This paper proposes a novel hybrid framework that integrates Extreme Value Theory (EVT) with Deep Generative Models (DGMs), specifically Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs), for probabilistic modeling and simulation of rare climate phenomena. EVT is employed to model the marginal distributions of extreme events using the Generalized Pareto and Generalized Extreme Value distributions. Meanwhile, DGMs learn latent representations from high-dimensional climate data and synthesize realistic, tail-aware samples. The proposed model captures both the statistical rigor of EVT and the expressive power of deep learning. Empirical evaluations are conducted using ERA5 reanalysis and satellite datasets, focusing on extreme precipitation and temperature anomalies across diverse regions. Results show that the hybrid EVT–DGM framework significantly improves tail risk estimation, return level prediction, and generative quality compared to conventional models. This approach provides a robust tool for data-driven climate risk forecasting under uncertainty.

We explore methods of producing adversarial examples on deep generative models such as the variational autoencoder (VAE) and the VAE-GAN. Deep learning architectures are known to be vulnerable to adversarial examples, but previous work has focused on the application of adversarial examples to classification tasks. Deep generative models have recently become popular due to their ability to model input data distributions and generate realistic examples from those distributions. We present three classes of attacks on the VAE and VAE-GAN architectures and demonstrate them against networks trained on MNIST, SVHN and CelebA. Our first attack leverages classification-based adversaries by attaching a classifier to the trained encoder of the target generative model, which can then be used to indirectly manipulate the latent representation. Our second attack directly uses the VAE loss function to generate a target reconstruction image from the adversarial example. Our third attack moves beyond relying on classification or the standard loss for the gradient and directly optimizes against differences in source and target latent representations. We also motivate why an attacker might be interested in deploying such techniques against a target generative network.

This paper proposes a set of methods that enables low precision posit ™ arithmetic to be successfully used for the training of generative adversarial networks (GANs) with minimal quality loss. We show that ultra low precision posits, as small as 6 bits, can achieve high quality output for the generation phase after training. We also evaluate floating-point (float) formats and compare them to 8-bit posits in the context of GAN training. Our scaling and adaptive calibration techniques are capable of producing superior training quality for 8-bit posits that surpasses 8-bit floats and matches the results of 16-bit floats. Hardware simulation results indicate that our methods have higher energy efficiency compared to both 16- and 8-bit float training systems.

<p>High data collection costs and complicated data access regulations increase the demand for synthetic data. Generative Adversarial Networks (GANs) are a novel generative framework with great potential for high quality synthetic data generation. GANs formulate the true distribution of data implicitly, and the success of GANs are often measured based on the similarity of generated data to this true distribution. GANs were originally designed to work with continuous data. However, many important real-world datasets such as medical images involve discontinuous distributions. GAN training for discontinuous distributions is relatively more challenging, as the training procedure often suffers from instability and mode collapse issues. This dissertation focuses on designing novel GAN architectures to generate representative synthetic image data, and proposes new structures to alleviate GANs' mode collapse issue. As part of this thesis, novel applications of image data generation with GANs have been also investigated for important problems arising in the telecommunication industry and medical domain. Specifically, we first explore various GAN structures to generate engineered electromagnetic surfaces. We consider the continuous approximation of the data and explore the capabilities of feed-forward and convolutional GANs for synthetic data generation. Next, we introduce a novel GAN architecture to address the problem of mode collapse in GAN training. The proposed structure incorporates a third network that penalizes the generator for generating low diversity samples. Lastly, we study the challenging problem of object generation in 3D space using GANs, and we propose extensions to existing 3D GAN structures to generate connected 3D volumes. Additionally, we explore a more challenging version of this 3D volume generation problem by generating connected volumes packed with spheres. This research has applications in radiosurgery treatment planning, and the proposed 3D GAN structure can help generate rare, unseen 3D tumor volumes and information on how to treat these tumors. Accordingly, our analysis contributes to overcoming data scarcity issues (e.g., due to privacy considerations) for an important practical problem in the medical domain.</p>

Electrical resistivity tomography (ERT) is a geophysical method used to imaging the subsurface and widely used in hydrogeological studies due to its sensitivity to electrical resistivity, which is directly related to rock type, porosity, ionic strength of the pore fluids, and surface conductivity of geologic materials. The prediction of subsurface properties from the recorded data at the surface requires solving a challenging geophysical inversion problem. For near-surface characterization studies, this is often accomplished with deterministic electrical resistivity inversion methods. Deterministic geophysical inversion approaches linearize the problem around an initial solution, resulting in a single smooth representation of the subsurface. Deterministic models are unable to capture the natural variability of the subsurface. Moreover, a single solution does not yield enough information for accurate uncertainty assessment. Contrary to deterministic approaches, stochastic inversion methods predict multiple model realizations that fit similarly the recorded geophysical data and allow assessing uncertainties. Lately, deep learning algorithms based on deep generative models have been used to re-parametrize model and data spaces into low-dimensional domains and solve geophysical inverse problems in a more efficient way.We propose an ERT inversion methodology in which a deep convolutional variational autoencoder (VAE) network is trained with a set of electrical resistivity models generated using geostatistical simulation. After training the VAE, the latent space is perturbed and updated iteratively with adaptive stochastic sampling to generate electrical resistivity models by inputting the optimized latent vectors to the decoder part of the VAE. From the set of decoded models, we use a finite volume approximation of Poisson’s equation to compute synthetic apparent resistivity models. The misfit between predicted and observed apparent resistivity data is used to drive the convergence of the iterative procedure and condition the optimization of new models in the subsequent iterations.The proposed methodology is illustrated by applying it to both a two-dimensional synthetic case and to a two-dimensional profile obtained from an ERT survey carried out in an area located in the Southern region of Portugal. In both application examples, the predicted models generate synthetic geophysical data that match the observed one. We show the ability of the model to assess spatial uncertainty and compare the results obtained in the real data set against commercial deterministic ERT inversion methodology.The work presented herein is supported by the PRIMA programme under grant agreement No. 1923, project Innovative and Sustainable Groundwater Management in the Mediterranean (InTheMED). The PRIMA programme is supported by the European Union.