Noise adaptive training for subspace Gaussian mixture models

The subspace Gaussian mixture model (SGMM) has been exploited for cross-lingual speech recognition. The general motivation is that the subspace parameters can be estimated on multiple source languages and then transferred to the target language. In this work, we investigate an extension to SGMM, referred to as subspace mixture model (SMM), in which subspace parameters on the target language are casted as a linear mixture of the subspaces derived from source languages. This approach reduces the number of SGMM model parameters, while retaining the flexibility of subspace learning on the target language. Experiments show that the proposed SMM method outperforms SGMM significantly when the target language has limited training data.

Adaptive training is a widely used technique for building speech recognition systems on nonhomogeneous training data. Recently, there has been interest in applying these approaches for situations where there is significant levels of background noise in the training data. Various schemes for adaptive training are based on noise-, or speaker-, specific transforms of features to yield estimates of the clean speech. However, when there are high levels of background noise, these clean speech estimates may be poor resulting in degradations in performance. In this paper, a new approach for adaptive training on noise-corrupted training data is presented. It extends a popular form of linear transform for model-based adaptation and adaptive training, constrained MLLR (CMLLR), to reflect additional uncertainty from noise-corrupted observations. This new form of adaptation transform is called noisy CMLLR (NCMLLR). NCMLLR uses a modified version of generative model between clean speech and noisy observation, similar to factor analysis (FA). However, in contrast to FA here the generative model describes an adaptation transform, rather than a covariance matrix structure. The use of NCMLLR for adaptive training using an expectation-maximization approach is described. Discriminative adaptive training with NCMLLR is also described based on the minimum phone error criterion. Experimental results comparing NCMLLR with standard adaptive training schemes are given on a noise-corrupted version of Resource Management, the ARPA 1994 CSRNAB Spoke 10 task, and in-car recorded data.

This paper concerns cross-lingual acoustic modeling in the case when there are limited target language resources. We build on an approach in which a subspace Gaussian mixture model (SGMM) is adapted to the target language by reusing the globally shared parameters estimated from out-of-language training data. In current cross-lingual systems, these parameters are fixed when training the target system, which can give rise to a mismatch between the source and target systems. We investigate a maximum a posteriori (MAP) adaptation approach to alleviate the potential mismatch. In particular, we focus on the adaptation of phonetic subspace parameters using a matrix variate Gaussian prior distribution. Experiments on the GlobalPhone corpus using the MAP adaptation approach results in word error rate reductions, compared with the cross-lingual base-line systems and systems updated using maximum likelihood, for training conditions with 1 hour and 5 hours of target language data.

Subspace Gaussian mixture based language modeling for large vocabulary continuous speech recognition

In this paper, the improvements in the recently implemented Kannada speech recognition system is demonstrated in detail. The Kannada automatic speech recognition (ASR) system consists of ASR models which are created by using Kaldi, IVRS call flow and weather and agricultural commodity prices information databases. The task specific speech data used in the recently developed spoken dialogue system had high level of different background noises. The different types of noises present in collected speech data had an adverse effect on the on line and off line speech recognition performances. Therefore, to improve the speech recognition accuracy in Kannada ASR system, a noise reduction algorithm is developed which is a fusion of spectral subtraction with voice activity detection (SS-VAD) and minimum mean square error spectrum power estimator based on zero crossing (MMSE-SPZC) estimator. The noise elimination algorithm is added in the system before the feature extraction part. An alternative ASR models are created using subspace Gaussian mixture models (SGMM) and deep neural network (DNN) modeling techniques. The experimental results show that, the fusion of noise elimination technique and SGMM/DNN based modeling gives a better relative improvement of 7.68% accuracy compared to the recently developed GMM-HMM based ASR system. The least word error rate (WER) acoustic models could be used in spoken dialogue system. The developed spoken query system is tested from Karnataka farmers under uncontrolled environment.

DBMiP: A pre-training method for information propagation over deep networks

Mechanical metamaterials - engineered materials whose behavior is defined by their internal structure rather than their base material - have received great at- tention for the unusual properties they can exhibit. However, while the forward problem of predicting the mechanical behavior of a given metamaterial can be reliably done through simulation or experiments, the inverse problem - designing metamaterials with specific properties - remains difficult, especially for heteroge- neous metamaterials composed of multiple unit cells. This dissertation explores the use of generative machine learning to address this inverse design problem. The first part of the work investigates how inverse and generative models can be conditioned to produce unit cells with prescribed mechanical properties and how they can be used to generate multiple valid solutions for the same target. This is examined using a curved-beam metamaterial, which allows a compact geometric representation without complex design restrictions. The effect these restrictions have on the generative models is investigated in the second part. For this, a number of generative models—Variational Autoencoders, Generative Adversarial Networks, and Diffusion Models—are evaluated on a kirigami metamaterial where dependencies between the cuts cause intricate design constraints. The final part covers the challenges that arise when generative models are applied to heteroge- neous metamaterials, in which multiple unit cells are combined to create spatially varying mechanical properties. The results show that the applicability of generative models for the design of metamaterials strongly depends on the choice of geometric parameterization. Generative models make assumptions about their input data that generally hold for images, but can be invalid for metamaterials when complex dependencies be- tween the input parameters exist. B-splines can provide a representation without those dependencies and are therefore well suited to be paired up with generative models. Furthermore, it is also important how these geometrical parameters are sampled to create the training data for the generative models, as this sampling also affects the distribution of the properties in the training data. Overall, while generative machine learning models prove effective for the inverse design of mechanical metamaterials their successful application requires careful parameterization and selection of the training data.

With the recent advancements in the field of diffusion generative models, it has been shown that defining the generative process in the latent space of a powerful pretrained autoencoder can offer substantial advantages. This approach, by abstracting away imperceptible image details and introducing substantial spatial compression, renders the learning of the generative process more manageable while significantly reducing computational and memory demands. In this work, we propose to replace autoencoder coding with a model-based coding scheme based on traditional lossy image compression techniques; this choice not only further diminishes computational expenses but also allows us to probe the boundaries of latent-space image generation. Our objectives culminate in the proposal of a valuable approximation for training continuous diffusion models within a discrete space, accompanied by enhancements to the generative model for categorical values. Beyond the good results obtained for the problem at hand, we believe that the proposed work holds promise for enhancing the adaptability of generative diffusion models across diverse data types beyond the realm of imagery.

A statistical parametric approach to singing voice synthesis based on hidden Markov Models (HMMs) has been growing in popularity over the last few years. The spectrum, excitation, vibrato, and duration of singing voices in this approach are simultaneously modeled with context-dependent HMMs and waveforms are generated from the HMMs themselves. HMM-based singing voice synthesis systems are heavily based on the training data in performance because these systems are “corpus-based.” Therefore, HMMs corresponding to contextual factors that hardly ever appear in the training data cannot be well-trained. Pitch should especially be correctly covered since generated F 0 trajectories have a great impact on the subjective quality of synthesized singing voices. We applied the method of “speaker adaptive training” (SAT) to “pitch adaptive training,” which is discussed in this paper. This technique made it possible to normalize pitch based on musical notes in the training process. The experimental results demonstrated that the proposed technique could alleviate the data sparseness problem.

We study the problem of synthetic generation of samples of environmental features for autonomous vehicle navigation. These features are described by a spatiotemporally varying scalar field that we refer to as a threat field. The threat field is known to have some underlying dynamics subject to process noise. Some "real-world" data of observations of various threat fields are also available. The assumption is that the volume of "real-world" data is relatively small. The objective is to synthesize samples that are statistically similar to the data. The proposed solution is a generative artificial intelligence model that we refer to as a split variational recurrent neural network (S-VRNN). The S-VRNN merges the capabilities of a variational autoencoder, which is a widely used generative model, and a recurrent neural network, which is used to learn temporal dependencies in data. The main innovation in this work is that we split the latent space of the S-VRNN into two subspaces. The latent variables in one subspace are learned using the real-world data, whereas those in the other subspace are learned using the data as well as the known underlying system dynamics. Through numerical experiments we demonstrate that the proposed S-VRNN can synthesize data that are statistically similar to the training data even in the case of very small volume of real-world training data.

Artificial intelligence (AI) has the ability to predict rheological properties and constituent composition of 3D-printed materials with appropriately trained models. However, these models are not currently available for use. In this work, we trained deep learning (DL) models to (1) predict the rheological properties, such as the storage (G’) and loss (G”) moduli, of 3D-printed polyacrylamide (PAA) substrates, and (2) predict the composition of materials and associated 3D printing parameters for a desired pair of G’ and G”. We employed a multilayer perceptron (MLP) and successfully predicted G’ and G” from seven gel constituent parameters in a multivariate regression process. We used a grid-search algorithm along with 10-fold cross validation to tune the hyperparameters of the MLP, and found the R2 value to be 0.89. Next, we adopted two generative DL models named variational autoencoder (VAE) and conditional variational autoencoder (CVAE) to learn data patterns and generate constituent compositions. With these generative models, we produced synthetic data with the same statistical distribution as the real data of actual hydrogel fabrication, which was then validated using Student’s t-test and an autoencoder (AE) anomaly detector. We found that none of the seven generated gel constituents were significantly different from the real data. Our trained DL models were successful in mapping the input–output relationship for the 3D-printed hydrogel substrates, which can predict multiple variables from a handful of input variables and vice versa.

The digitization of histology slides has revolutionized pathology, providing massive datasets for cancer diagnosis and research. Contrastive self-supervised and vision-language models have been shown to effectively mine large pathology datasets to learn discriminative representations. On the other hand, generative models, capable of synthesizing realistic and diverse images, present a compelling solution to address unique problems in pathology that involve synthesizing images; overcoming annotated data scarcity, enabling privacy-preserving data sharing, and performing inherently generative tasks, such as virtual staining. We introduce PixCell, the first diffusion-based generative foundation model for histopathology. We train PixCell on PanCan-30M, a vast, diverse dataset derived from 69,184 H&E-stained whole slide images covering various cancer types. We employ a progressive training strategy and a self-supervision-based conditioning that allows us to scale up training without any annotated data. PixCell generates diverse and high-quality images across multiple cancer types, which we find can be used in place of real data to train a self-supervised discriminative model. Synthetic images shared between institutions are subject to fewer regulatory barriers than would be the case with real clinical images. Furthermore, we showcase the ability to precisely control image generation using a small set of annotated images, which can be used for both data augmentation and educational purposes. Testing on a cell segmentation task, a mask-guided PixCell enables targeted data augmentation, improving downstream performance. Finally, we demonstrate PixCell’s ability to use H&E structural staining to infer results from molecular marker studies; we use this capability to infer IHC staining from H&E images. Our trained models are publicly released to accelerate research in computational pathology.

Exploiting class label in generative score spaces

In this extended abstract, we present the use of Hidden Markov Models (HMMs) in order to synthesize walk sequences with a given style using a small amount of training data from the target style. As a first step, a general model of walk is built. Starting from that model, an adaptive training enables to adapt our model to any particular style using only a small amount of training data. This technique, which was originally developed for speaker adaptation in speech synthesis [Zen et al. 2007], enables to reduce the main problem of machine learning techniques which is the large amount of data needed to train each new model, and to adapt models to the exaggerated style variations of our database that were far from an average walk.

Froth flotation is a popular ground ore separation method with a wide range of applications in the mining industry due to its versatility and simplicity. In this method, a separation with a certain grade and recovery is achieved thanks to the difference in the surface hydrophobicity of the different insoluble compounds that are present in an industrial ground ore. Under a process intensification scope, the study of froth flotation columns is of remarkable interest due to their higher gas phase surface-to-volume ratios and longer bubble residence times that result in higher grades and recoveries while reducing the space requirements in a plant layout and the associated capital and operating costs. However, despite the progress made in flotation physics throughout the years, its complex nature has forced many controloriented modelling efforts to rely on empirical or mixed approaches with limited generalizability for a broad range of operating conditions. In contrast, while the use of high spatial resolution methods has improved our comprehension of flotation physics at both microscopic and macroscopic levels, the computational cost constitutes a drawback when developing tools for monitoring and control purposes. In view of this, a rework of a fundamental model originally developed in our group to describe a single batch flotation cell into a spatially-finite framework is proposed. Such rework features the use of discrete bubble size classes and population balance equations to track the evolution of gas holdup in the pulp and froth zones, and combines this discretized scheme with first-order attachment and detachment mechanisms to describe bubble-particle interactions as a function of time and height. Training data gathered in-situ from experiments using a conductivity probe inserted in the pulp zone, a machine vision-based measurement system (VisioFroth™) mounted on top of the froth zone, and experimental protocols to measure the cumulative cumulative solids recovery were used to train the model parameters in the pulp, interface and froth zones. Once trained, the predictive capability of the model was qualitatively assessed by carrying out parametric studies using two independent operating variables: the air inlet flow rate and the stirring rate. Subsequently, quantitative assessments were conducted by contrasting model predictions with testing data, both under operating conditions different to those of the training data. These procedures were conducted using experimental data gathered under a hybrid operation mode, where agitation was provided by a mechanical impeller inside the JKTech cell located at the base of the column flotation device. The parametric fitting revealed that the bubble-particle attachment phenomena were considered the driving force in the modelling of solids recovery, with a reasonable match between model trends and training experimental data (SSE for cumulative solids recovery of 690.4, 289.6 and 117.9 in the experimental datasets HY 900RPM - 5LMIN, HY 900RPM 11LMIN 200PPM and HY 1500RPM 11LMIN, respectively). The parametric studies and the Normalized Forward Sensitivity Index analysis revealed that the solids recovery predictions made by the trained model had a higher sensitivity to variations in the air inlet flow rate, which was evidenced in the predictive studies with opposing performances in the prediction of the testing experimental data (SSE for cumulative solids recovery of 102.8 and 3909.5 in the experimental datasets HY 900RPM 11LMIN and HY 1500RPM 5LMIN, respectively) despite the acceptable performance of the trained model in the prediction of the bubble size distribution dynamics in both zones. The proposed strategies to improve the predictive capability of the model included the retraining of the model under higher training and testing data volumes, the use of alternative data split ratios and the use of estimation methods to refine the accuracy of the predictions in real time.