Generative Data Models for Validation and Evaluation of Visualization Techniques

With the significant increase in the amount of solar energy sources, the variation in the production caused by the stochastic nature of these sources becomes a larger problem for the electrical grid. Having a large percentage of installed units, therefore, requires some type of both short- and long-term planning of energy production. Most planning and forecasting tools require a large amount of solar energy data for the location in question and oftentimes that data is difficult to obtain. In this paper, we propose a method for generating synthetic solar irradiance data and test its applicability on forecasting and long-term planning problems. Synthetic data are generated using various generative adversarial networks and masked autoencoder based models, which are all trained on a dataset of solar irradiance data measured on the territory of the city of Banja Luka in Bosnia and Herzegovina. The synthetic data are evaluated both on different types of short-term forecasting tools and a long-term solar plant simulation tool, and the obtained results are compared with the cases where real data is used. By comparing the results, we have concluded that by using generative models we are able to generate synthetic data which is not only realistic, but applicable for both of these uses.

<p><strong>Because of its wide range of applications, generative artificial intelligence (Generative AI) has received a lot of attention in academia and industry. Despite significant advances in computer vision and natural language processing, the use of Generative AI in tabular data is still underexplored. This gap is especially significant given the prevalence of tabular data as the primary data modality. This thesis seeks to close this gap by focusing on the efficient generation, evaluation, refinement, and application of tabular data, while addressing the challenges inherent in its heterogeneous nature.</strong></p><p>The primary goal of this thesis is to develop efficient algorithms for tabular data synthesis, advancing the field of Generative AI in the tabular domain. To achieve this goal, four specific research objectives have been outlined.</p><p>First, this thesis addresses gaps in evaluation metrics by unifying a framework for the comprehensive and consistent assessment of synthetic data. The proposed framework is designed to meet diverse business requirements across various downstream tasks by incorporating a wide range of advanced metrics. These metrics cover various types of evaluations, including univariate, bivariate, multivariate, cluster, and record-level evaluations. Additionally, standardized visualizations are provided to facilitate qualitative assessments. The results indicate that the proposed evaluation framework not only enables consistent comparisons, rankings, and the selection of different data synthesis approaches but also acts as a valuable tool to promptly assess the reliability of results obtained from synthetic data.</p><p>Second, this thesis presents innovative algorithms for the generation of tabular data, aiming to significantly enhance the quality of data synthesis. One of the contributions is the development of a reversible feature engineering pipeline designed to automatically represent tabular data in an efficient format while also ensuring that the transformed data can be easily converted back to its original format. Additionally, the thesis proposes novel deep learning-based tabular data generation models that are capable of learning the joint distribution of multivariate datasets without relying on predefined distribution assumptions. Experimental results highlight the effectiveness of the proposed algorithms in handling heterogeneous data, demonstrating superior performance in most scenarios within the same training duration when compared to similar alternatives.</p><p>Third, this thesis introduces innovative synthetic data prototype selection algorithms with the goal of refining the generated samples. This approach aims to leverage the advantages of Deep Generative Models, which, once trained, can generate unlimited and diverse synthetic data. By carefully identifying and selecting high-quality samples or removing unrealistic ones, the quality of the synthetic data can be enhanced from a post-processing perspective. Building on this hypothesis, we recognize the iterative nature inherent in the data synthesis procedure, where the processes of data generation, evaluation, and refinement should operate repeatedly in a cyclical flow. During the data generation phase, the evaluation framework assists in identifying potential issues and risks based on use cases, while the refinement (post-processing) step iteratively improves synthetic data in alignment with the evaluation outcomes.</p><p>Last, this thesis applies and validates the proposed models across various domains, including business, healthcare, and government, each requiring distinct downstream tasks. Specifically, we assess our algorithms from the standpoint of data balancing using churn data, evaluate our models focusing on data augmentation with health data, and test our algorithms from the perspective of data representation considering data privacy concerns within sensitive citizen data. These applications serve as practical illustrations, demonstrating the effectiveness and utility of the proposed Generative AI models in real-world scenarios.</p><p>In summary, this thesis makes a substantial contribution to the advancement of Generative AI within the domain of tabular data. It not only presents innovative algorithms and evaluation methods but also introduces practical frameworks applicable to real-world scenarios. The findings demonstrate the considerable capacity of Generative AI to fundamentally transform tabular data in various fields, ultimately leading to improved data availability, quality, and quantity.</p>

Recent developments in generative models have demonstrated that with the right data set, techniques, computational infrastructure, and network architectures, it is possible to generate seemingly intelligent outputs, without explicitly reckoning with underlying cognitive processes. The ability to generate novel, plausible behaviour could be a boon to cognitive modellers. However, insights for cognition are limited, given that generative models' blackbox nature does not provide readily interpretable hypotheses about underlying cognitive mechanisms. On the other hand, cognitive architectures make very strong hypotheses about the nature of cognition, explicitly describing the subjects and processes of reasoning. Unfortunately, the formal framings of cognitive architectures can make it difficult to generate novel or creative outputs. We propose to show that cognitive architectures that rely on certain Vector Symbolic Algebras (VSAs) are, in fact, naturally understood as generative models. We discuss how memories of VSA representations of data form distributions, which are necessary for constructing distributions used in generative models. Finally, we discuss the strengths, challenges, and future directions for this line of work.

Although a plethora of synthetic data generation models have been proposed to validate biomarkers of brain and cardiovascular dynamics separately, a limited number of computational methods estimating directed brain-heart information flow are currently available in the scientific literature. This study introduces a computational framework exploiting existing generative models for a novel time-resolved quantification of causal brain-heart interplay. Exemplarily, having electroencephalographic signals and heart rate variability series as inputs, respective synthetic data models are coupled through parametrised functions defined in accordance with current central autonomic network (CAN) knowledge. We validate this concept using data from 30 healthy volunteers undergoing notable sympathetic elicitation through a cold-pressor test, and further compare the obtained results with a state-of-the-art method as maximal information coefficient. Although our findings are in agreement with previous CAN findings, we report new insights into the role of fronto-parietal region activity and lateralisation mechanisms over the temporal cortices during prolonged peripheral elicitation, which occur with specific time delays. Additionally, the afferent autonomic outflow maps to brain oscillations in the δ and γ bands, whereas complementary cortical dynamics in the θ, α, and β bands act on efferent autonomic control. The proposed framework paves the way towards novel biomarker definitions for the assessment of complex physiological networks using existing data generation models for brain and peripheral dynamics.

A critical task for system operators is the precise identification of the root causes underlying an error situation. This identification is fundamental in deciding optimal maintenance actions, such as replacing a component versus calibrating it. However, the actual causes of an error are often neither measured nor unique. The measured quantities are the result of complex interactions between different error causes and system variables. Root cause identification in this context becomes a matter of inferring hidden causes from their measurable effects. This challenge is notably pronounced in cyber-physical systems comprising control loops. Control mechanisms, integral to maintaining system performance, introduce a layer of complexity in diagnostics and ultimately complicate the isolation of the underlying causes of errors. To address this challenge, we introduce a two-step approach to derive the hidden causes as a statistical inference task. First, we develop a generative model leveraging existing control software and expert-based insights into the mechanisms of errors, i.e., a simulator of synthetic data given some hidden error causes. Then, we transform the generative model into a probabilistic program on which statistical inference can be executed within a probabilistic programming language framework. This inference effectively estimates the hidden causes given some measured data from the system. Being intrinsically a statistical approach, these inferences come with a confidence interval. We applied this methodology to an industrial printer’s sheet transport belt, operating in a closed-loop configuration. Our approach successfully discerned the contributions of three distinct hidden causes to the belt’s deviation from its intended position. This paper highlights the efficacy of generative modeling followed by a probabilistic programming approach in unraveling complex interactions within cyber-physical systems for optimal maintenance.

Generative models, such as diffusion models, have made significant advancements in recent years, enabling the synthesis of high‐quality realistic data across various domains. Here, the adaptation and training of a diffusion model on super‐resolution microscopy images are explored. It is shown that the generated images resemble experimental images, and that the generation process does not exhibit a large degree of memorization from existing images in the training set. To demonstrate the usefulness of the generative model for data augmentation, the performance of a deep learning‐based single‐image super‐resolution (SISR) method trained using generated high‐resolution data is compared against training using experimental images alone, or images generated by mathematical modeling. Using a few experimental images, the reconstruction quality and the spatial resolution of the reconstructed images are improved, showcasing the potential of diffusion model image generation for overcoming the limitations accompanying the collection and annotation of microscopy images. Finally, the pipeline is made publicly available, runnable online, and user‐friendly to enable researchers to generate their own synthetic microscopy data. This work demonstrates the potential contribution of generative diffusion models for microscopy tasks and paves the way for their future application in this field.

In this paper, we present a generative sketch model for human hair analysis and synthesis. We treat hair images as2D piecewise smooth vector (flow) fields and, thus, our representation is view-based in contrast to the physically-based 3D hair models in graphics. The generative model has three levels. The bottom level is the high-frequency band of the hair image. The middle level is a piecewise smooth vector field for the hair orientation, gradient strength, and growth directions. The top level is an attribute sketch graph for representing the discontinuities in the vector field.Asketch graph typically has a number of sketch curves which are divided into 11 types of directed primitives. Each primitive is a small window (say 5 _ 7 pixels) where the orientations and growth directions are defined in parametric forms, for example, hair boundaries, occluding lines between hair strands, dividing lines on top of the hair, etc. In addition to the three level representation, we model the shading effects, i.e., the low-frequency band of the hair image, by a linear superposition of some Gaussian image bases andwe encode the hair color by a color map. The inference algorithm is divided into two stages: 1) We compute the undirected orientation field and sketch graph from an input image and 2) we compute the hair growth direction for the sketch curves and the orientation field using a Swendsen-Wang cut algorithm. Both steps maximize a joint Bayesian posterior probability. The generative model provides a straightforward way for synthesizing realistic hair images and stylistic drawings (rendering) from a sketch graph and a few Gaussian bases. The latter can be either inferred from a real hair image or input(edited) manually using a simple sketching interface. We test our algorithm on a large data set of hair images with diverse hair styles.

As a new way of training generative models, Generative Adversarial Net (GAN) that uses a discriminative model to guide the training of the generative model has enjoyed considerable success in generating real-valued data. However, it has limitations when the goal is for generating sequences of discrete tokens. A major reason lies in that the discrete outputs from the generative model make it difficult to pass the gradient update from the discriminative model to the generative model. Also, the discriminative model can only assess a complete sequence, while for a partially generated sequence, it is non-trivial to balance its current score and the future one once the entire sequence has been generated. In this paper, we propose a sequence generation framework, called SeqGAN, to solve the problems. Modeling the data generator as a stochastic policy in reinforcement learning (RL), SeqGAN bypasses the generator differentiation problem by directly performing gradient policy update. The RL reward signal comes from the GAN discriminator judged on a complete sequence, and is passed back to the intermediate state-action steps using Monte Carlo search. Extensive experiments on synthetic data and real-world tasks demonstrate significant improvements over strong baselines.

Sequence classification is critical in the data mining communities. It becomes more challenging when the class distribution is imbalanced, which occurs in many real-world applications. Oversampling algorithms try to re-balance the skewed class by generating synthetic data for minority classes, but most of existing oversampling approaches could not consider the temporal structure of sequences, or handle multivariate and long sequences. To address these problems, this paper proposes a novel oversampling algorithm based on the 'generative' models of sequences. In particular, a recurrent neural network was employed to learn the generative mechanics for sequences as representations for the corresponding sequences. These generative models are then utilized to form a kernel to capture the similarity between different sequences. Finally, oversampling is performed in the kernel feature space to generate synthetic data. The proposed approach can handle highly imbalanced sequential data and is robust to noise. The competitiveness of the proposed approach is demonstrated by experiments on both synthetic data and benchmark data, including univariate and multivariate sequences.

Many applications require precise estimates of high-dimensional covariance matrices. The standard estimator is the sample covariance matrix, which is conceptually simple, fast to compute and has favorable properties in the limit of infinitely many observations. The picture changes when the dimensionality is of the same order as the number of observations. In such cases, the eigenvalues of the sample covariance matrix are highly biased, the condition number becomes large and the inversion of the matrix gets numerically unstable.
\nA number of alternative estimators are superior in the high-dimensional setting, which include as subcategories structured estimators, regularized estimators and spectrum correction methods. In this thesis I contribute to all three areas. 
\nIn the area of structured estimation, I focus on models with low intrinsic dimensionality. I analyze the bias in Factor Analysis, the state-of-the-art factor model and propose Directional Variance Adjustment (DVA) Factor Analysis, which reduces bias and yields improved estimates of the covariance matrix.
\nAnalytical shrinkage of Ledoit and Wolf (LW-Shrinkage) is the most popular regularized estimator. I contribute in three aspects: first, I provide a theoretical analysis of the behavior of LW-Shrinkage in the presence of pronounced eigendirections, a case of great practical relevance. I show that LW-Shrinkage does not perform well in this setting and propose aoc-Shrinkage which yields significant improvements. Second, I discuss the effect of autocorrelation on LW-Shrinkage and review the Sancetta-Estimator, an extension of LW-Shrinkage to autocorrelated data. I show that the Sancetta-Estimator is biased and propose a theoretically and empirically superior estimator with reduced bias. Third, I propose an extension of shrinkage to multiple shrinkage targets. Multi-Target Shrinkage is not restricted to covariance estimation and allows for many interesting applications which go beyond regularization, including transfer learning. I provide a detailed theoretical and empirical analysis.
\nSpectrum correction approaches the problem of covariance estimation by improving the estimates of the eigenvalues of the sample covariance matrix. I discuss the state-of-the-art approach, Nonlinear Shrinkage, and propose a cross-validation based covariance (CVC) estimator which yields competitive performance at increased numerical stability and greatly reduced complexity and computational cost. On all data sets considered, CVC is on par or superior in comparison to the regularized and structured estimators.
\nIn the last chapter, I conclude with a discussion of the advantages and disadvantages of all covariance estimators presented in this thesis and give situation-specific recommendations. In addition, the appendix contains a systematic analysis of Linear Discriminant Analysis as a model application, which sheds light on the interdependency between the generative model of the data and various covariance estimators.

Machine Learning (ML) research in healthcare remains challenging as large, privacy preserving open data sets are lacking. Synthetic data could offer a solution – but the value of synthetic data depends on diverse and conflicting criteria such as utility, fidelity, and privacy, which are rarely evaluated comprehensively. To close this gap, we explore the trade-off between these metrics in an empirical evaluation across a broad spectrum of generative models, datasets and metrics. Our results indicate that no single generative model excels across all metrics and datasets. In contrast, we find that for every dataset, different generative methods work best -- highlighting the need for automation for synthetic data methods. We investigate the dependency between privacy and utility metrics and demonstrate that the first two main variance directions of all metrics capture the trade-offs between fidelity, utility, and privacy well enough to support design choices of generative models in healthcare.

The article considers the possibility of using synthetic data to model a portfolio in conditions of instability in financial markets. Methods based on the analysis of historical data need to be revised to account for rare events that are missing from historical records. The use of generative models makes it possible to generate synthetic data, creating conditions for modeling situations that go beyond the known scenarios. The purpose of the article is to consider the impact of synthetic data on asset management methods in conditions of high market volatility. The methods of creating synthetic data using generative adversarial networks and their role in modeling situations with limited access to necessary information are described. The use of synthetic data in scientific papers confirms their effectiveness in adapting asset management strategies, which contributes to improving results. This underscores the need for their application and guarantees the stability of financial systems in the external conditions that determine the present. In conclusion, it is noted that generative models that create synthetic data increase the accuracy and flexibility of financial portfolio management strategies. The considered approaches open up opportunities for forecasting and decision-making. Due to this, the information contained in the work will be useful to investors and bank employees.

We have witnessed rapid progress on 3D-aware image synthesis, leveraging recent advances in generative visual models and neural rendering. Existing approaches however fall short in two ways: first, they may lack an underlying 3D representation or rely on view-inconsistent rendering, hence synthesizing images that are not multi-view consistent; second, they often depend upon representation network architectures that are not expressive enough, and their results thus lack in image quality. We propose a novel generative model, named Periodic Implicit Generative Adversarial Networks ($π$-GAN or pi-GAN), for high-quality 3D-aware image synthesis. $π$-GAN leverages neural representations with periodic activation functions and volumetric rendering to represent scenes as view-consistent 3D representations with fine detail. The proposed approach obtains state-of-the-art results for 3D-aware image synthesis with multiple real and synthetic datasets.

Preserving logical and functional dependencies in synthetic tabular data

Applications for machine learning (ML) and deep learning (DL) are constantly growing and have already been adopted in the field of particle measurement technology. Even though analytical (ultra-)centrifugation (AC/AUC) is a widely used technique for characterizing dispersed particle systems, ML and DL have not yet been applied in this area. Data evaluation and interpretation in AC/AUC can be challenging and often requires expert knowledge. DL models can help, but their development is limited by a lack of annotated training data. One solution is to generate and use synthetic data instead. In the first part of this study, a model was trained to synthesize data from experiments using a combination of Variational Autoencoder (VAE) and Generative Adversarial Networks (GANs). The results appear highly realistic. Novice users could distinguish real from synthetic samples with only 63% accuracy. Then, a classifier was trained on experimental AC data to categorize real-world examples based on their underlying separation kinetics, testing different DL architectures. After initial training, the models were further fine-tuned with synthetic AC data. ResNet34 models achieved the best performance with 94% accuracy, comparable to an AC expert (91%), while inexperienced users reached only 53%. In the second part of our study, a regression model was trained for the analysis of sedimentation coefficients. Therefore, various generative models were developed and evaluated for synthesizing AUC data based on numerically simulated sedimentation boundaries. The best results were achieved by combining VAE and GAN architectures with embedded physical constraints. However, the generative networks have so far led to additional smearing of the profiles, resulting in a broadening of the sedimentation coefficient distribution and indicating that further refinement is necessary.