A Sequentially-Adaptive Deep Variational Model for Multirate Process Anomaly Detection

Automated design synthesis has the potential to revolutionize the modern engineering design process and improve access to highly optimized and customized products across countless industries. Successfully adapting generative machine learning to design engineering may enable such automated design synthesis and is a research subject of great importance. We present a review and analysis of deep generative machine learning models in engineering design. Deep generative models (DGMs) typically leverage deep networks to learn from an input dataset and synthesize new designs. Recently, DGMs such as feedforward neural networks (NNs), generative adversarial networks (GANs), variational autoencoders (VAEs), and certain deep reinforcement learning (DRL) frameworks have shown promising results in design applications like structural optimization, materials design, and shape synthesis. The prevalence of DGMs in engineering design has skyrocketed since 2016. Anticipating the continued growth, we conduct a review of recent advances to benefit researchers interested in DGMs for design. We structure our review as an exposition of the algorithms, datasets, representation methods, and applications commonly used in the current literature. In particular, we discuss key works that have introduced new techniques and methods in DGMs, successfully applied DGMs to a design-related domain, or directly supported the development of DGMs through datasets or auxiliary methods. We further identify key challenges and limitations currently seen in DGMs across design fields, such as design creativity, handling constraints and objectives, and modeling both form and functional performance simultaneously. In our discussion, we identify possible solution pathways as key areas on which to target the future work.

Early warning for learning performance requires to identify the maximum number of at-risk students as early as possible within a semester. However, educational data often suffer from the issue of data imbalanced, making it challenging to simultaneously achieve both high precision (accurate identification) and high recall (comprehensive coverage) in at-risk student detection. Deep generative models and oversampling models are effective methods to solve data imbalanced issues, which can improve classification performance. This paper proposes a method that combines the advantages of deep generative models and oversampling models to build a blending model for dealing with the imbalanced education data, which can effectively improve the precision, recall, f1-score and auc for online learning early warning. First, we compare baseline models to select the best classifier, then choose the highest-precision deep generative model and the highest-recall oversampling model to construct blending models, which are shown to improve early warning prediction metrics. Finally, interpretable models are used to analyze differences in at-risk student prediction between the blending model, deep generative model, and oversampling model. The proposed models are validated on both extremely imbalanced datasets and new semester datasets. Results show that: (1) Compared to the baseline model, both the base learners build by deep generative model and oversampling model can improve the evaluation metrics of the model, the deep generative base learners achieve higher precision than oversampling model, the oversampling base learners achieve higher recall than the deep generative base learners. (2) The blending model composed of deep generative base learner and oversampling base learner can further improve the f1-score and auc based on their individual strengths, the proposed blending model can also conduct effective early warning 3 units earlier than baseline models. (3) Compared to its base learners, blending model G-B-Blending changed the key variables for prediction, and the at-risk students identified by the blending model come from the union set of at-risk students identified by GAN+GB and B-SMOTE+GB individually. (4) The blending model proposed in this paper achieves better prediction results than the baseline on both extremely imbalanced datasets and new semesters datasets, it can identify more at-risk students more accurately at earlier units, allowing teachers to save more energy and time for teaching interventions. This research provides significant insights for dealing with imbalanced datasets by blending with deep generative model and oversampling model in education.

The de novo molecule generation problem involves generating novel or modified molecular structures with desirable properties. Taking advantage of the great representation learning ability of deep learning models, deep generative models, which differ from discriminative models in their traditional machine learning approach, provide the possibility of generation of desirable molecules directly. Although deep generative models have been extensively discussed in the machine learning community, a specific investigation of the computational issues related to deep generative models for de novo molecule generation is needed. A concise and insightful discussion of recent advances in applying deep generative models for de novo molecule generation is presented, with particularly emphasizing the most important challenges for successful application of deep generative models in this specific area.This article is categorized under: Computer and Information Science > Chemoinformatics Computer and Information Science > Computer Algorithms and Programming

Deep generative models have gained much attention given their ability to generate data for applications as varied as healthcare to financial technology to surveillance, and many more - the most popular models being generative adversarial networks (GANs) and variational auto-encoders (VAEs). Yet, as with all machine learning models, ever is the concern over security breaches and privacy leaks and deep generative models are no exception. In fact, these models have advanced so rapidly in recent years that work on their security is still in its infancy. In an attempt to audit the current and future threats against these models, and to provide a roadmap for defense preparations in the short term, we prepared this comprehensive and specialized survey on the security and privacy preservation of GANs and VAEs. Our focus is on the inner connection between attacks and model architectures and, more specifically, on five components of deep generative models: the training data, the latent code, the generators/decoders of GANs/VAEs, the discriminators/encoders of GANs/VAEs, and the generated data. For each model, component and attack, we review the current research progress and identify the key challenges. The paper concludes with a discussion of possible future attacks and research directions in the field.

A method for evaluating deep generative models of images for hallucinations in high-order spatial context

Deep generative models are a developing branch of generative artificial intelligence (AI), which is growing rapidly across various sectors due to current advancements. Its application has now extended to the medical and healthcare domains. Deep generative model technology is bringing new possibilities in medical and healthcare applications. This technology is driving revolutionary advancements across a wide range of medical facilities, specializations, and innovative research. Deep generative models and related tools aid in generating and supporting medical and healthcare applications by facilitating innovative research, in medical reporting and analysis, medical images, disease and risk prediction, and associated drug discovery. Nevertheless, these models face serious obstacles in their implementation in real time applications, including challenges related to accuracy, cost-effectiveness, privacy, security, and authentication, as well as ethical concerns. This chapter provides a detailed discussion of the challenges involved in building the robust deep generative models, focusing on aspects such as handling the clinical trials, clinical relevance, working with heterogeneous datasets, addressing dataset scarcity, and overcoming biological complexity.

Deep generative models are a developing branch of generative artificial intelligence (AI), which is growing rapidly across various sectors due to current advancements. Its application has now extended to the medical and healthcare domains. Deep generative model technology is bringing new possibilities in medical and healthcare applications. This technology is driving revolutionary advancements across a wide range of medical facilities, specializations, and innovative research. Deep generative models and related tools aid in generating and supporting medical and healthcare applications by facilitating innovative research, in medical reporting and analysis, medical images, disease and risk prediction, and associated drug discovery. Nevertheless, these models face serious obstacles in their implementation in real time applications, including challenges related to accuracy, cost-effectiveness, privacy, security, and authentication, as well as ethical concerns. This chapter provides a detailed discussion of the challenges involved in building the robust deep generative models, focusing on aspects such as handling the clinical trials, clinical relevance, working with heterogeneous datasets, addressing dataset scarcity, and overcoming biological complexity.

Synthetic data offers a promising avenue for addressing privacy, scarcity, and fairness challenges in healthcare datasets. However, there is limited evaluation of how different generation methods balance fidelity, utility, and fairness, particularly for underrepresented subgroups. This study addresses this gap by comparing representative generative modelling techniques, both probabilistic and deep approaches, that are popular in the research literature. We empirically evaluate BayesBoost, CTGAN, TVAE, CopulaGAN, and DECAF on two healthcare datasets containing numerical, binary, and categorical features. Each model’s performance is assessed along three axes: data fidelity, machine learning utility, and fairness, using Accuracy Parity, Equalised Odds, and Predictive Rate Parity. Results show that BayesBoost consistently achieved superior fidelity, utility, and fairness preservation, particularly when paired with Random Forest classifiers, achieving around 60 – 63 % higher downstream utility than GAN-based deep generative baselines (e.g., Random Forest accuracy up to 0.88 with BayesBoost versus 0.54 to − 0.55 for GAN-based methods). Deep generative models, while effective in capturing complex structures, often degraded fairness, especially for underrepresented groups, with equalised odds deviating by over 100 % from the ideal parity value of 1.0 in some settings. The Variational Autoencoder outperformed other deep generative models in fairness preservation, especially for equalised odds, although with some reduction in fidelity and utility. Overall, these findings suggest that synthetic data generation for healthcare must move beyond fidelity evaluations to explicitly assess fairness and subgroup impacts, with probabilistic models such as BayesBoost showing strong potential for ethical deployment, while deep generative models require further adaptation for fairness-sensitive applications.

This paper presents a novel compressed sensing (CS) approach to high dimensional wireless channel estimation by optimizing the input to a deep generative network. Channel estimation using generative networks relies on the assumption that the reconstructed channel lies in the range of a generative model. Channel reconstruction using generative priors outperforms conventional CS techniques and requires fewer pilots. It also eliminates the need of a priori knowledge of the sparsifying basis, instead using the structure captured by the deep generative model as a prior. Using this prior, we also perform channel estimation from one-bit quantized pilot measurements, and propose a novel optimization objective function that attempts to maximize the correlation between the received signal and the generator's channel estimate while minimizing the rank of the channel estimate. Our approach significantly outperforms sparse signal recovery methods such as Orthogonal Matching Pursuit (OMP) and Approximate Message Passing (AMP) algorithms such as EM-GM-AMP for narrowband mmWave channel reconstruction, and its execution time is not noticeably affected by the increase in the number of received pilot symbols.

Quantile regression and conditional density estimation can reveal structure that is missed by mean regression, such as multimodality and skewness. In this article, we introduce a deep learning generative model for joint quantile estimation called Penalized Generative Quantile Regression (PGQR). Our approach simultaneously generates samples from many random quantile levels, allowing us to infer the conditional distribution of a response variable given a set of covariates. Our method employs a novel variability penalty to avoid the problem of vanishing variability, or memorization, in deep generative models. Further, we introduce a new family of partial monotonic neural networks (PMNN) to circumvent the problem of crossing quantile curves. A major benefit of PGQR is that it can be fit using a single optimization, thus, bypassing the need to repeatedly train the model at multiple quantile levels or use computationally expensive cross-validation to tune the penalty parameter. We illustrate the efficacy of PGQR through extensive simulation studies and analysis of real datasets. Code to implement our method is available at https://github.com/shijiew97/PGQR.

Optimal transport has a long history in mathematics which was proposed by Gaspard Monge in the eighteenth century (Monge, 1781). However, until recently, advances in optimal transport theory pave the way for its use in the AI community, particularly for formulating deep generative models. In this paper, we provide a comprehensive overview of the literature in the field of deep generative models using optimal transport theory with an aim of providing a systematic review as well as outstanding problems and more importantly, open research opportunities to use the tools from the established optimal transport theory in the deep generative model domain.

Image security in visual data is growing in importance due to the increasing use of digital images in many applications. This paper describes a novel image encrypting system using deep generative models with a self-attention mechanism to improve encrypting speed, decrypting speed, or both. Based on the CycleGAN models, the encrypting generator encrypts images while a dedicated decryption network decrypts them properly. The self-attention module improves the dispersibility of visual data to capture global dependencies among images at variable scales to achieve a highly secure image transformation with high reconstruction accuracy. Performance analysis has been done using two color image sets, brain MRI images to detect brain tumors, and images to detect skin cancer. Analysis indicates high security (Entropy = 7.9996, NPCR ≈ 99.99%), high reconstruction accuracy (SSIM ≈ 0.99, PSNR > 40 dB), and high resistance to differential (UACI ≈ 33.46) and occlusion attacks to images. This paper opens up novel avenues in applying deep models to visual cryptography schemes.

Asymmetric deep generative models

Synthetic images generated from deep generative models have the potential to address data scarcity and data privacy issues. The selection of synthesis models is mostly based on image quality measurements, and most researchers favor synthetic images that produce realistic images, i.e., images with good fidelity scores, such as low Fr\'echet Inception Distance (FID) and high Peak Signal-To-Noise Ratio (PSNR). However, the quality of synthetic images is not limited to fidelity, and a wide spectrum of metrics should be evaluated to comprehensively measure the quality of synthetic images. In addition, quality metrics are not truthful predictors of the utility of synthetic images, and the relations between these evaluation metrics are not yet clear. In this work, we have established a comprehensive set of evaluators for synthetic images, including fidelity, variety, privacy, and utility. By analyzing more than 100k chest X-ray images and their synthetic copies, we have demonstrated that there is an inevitable trade-off between synthetic image fidelity, variety, and privacy. In addition, we have empirically demonstrated that the utility score does not require images with both high fidelity and high variety. For intra- and cross-task data augmentation, mode-collapsed images and low-fidelity images can still demonstrate high utility. Finally, our experiments have also showed that it is possible to produce images with both high utility and privacy, which can provide a strong rationale for the use of deep generative models in privacy-preserving applications. Our study can shore up comprehensive guidance for the evaluation of synthetic images and elicit further developments for utility-aware deep generative models in medical image synthesis.

This paper mainly applies deep generative models for short-term load forecasting on the enterprise electricity consumption dataset. After data cleaning on the electricity use dataset with the help of related weather data, we complement missing data and improve the data quality to better implement neural network generative prediction models. We build DeepAR and Wavenet as the representative of deep generative models. The main result is that deep generative models perform better compared with other baseline models, such as ARIMA, machine learning and baseline neural networks, no matter what accuracy metric and prediction horizon. Further improvement is to test in higher frequency electricity dataset with better quality.