Variational Bayesian Subgroup Adaptive Sparse Component Extraction for Diagnostic Imaging System

Accurate subcategorization of tumour types through gene-expression profiling requires analytical techniques that estimate the number of categories or clusters rigorously and reliably. Parametric mixture modelling provides a natural setting to address this problem. We compare a criterion for model selection that is derived from a variational Bayesian framework with a popular alternative based on the Bayesian information criterion. Using simulated data, we show that the variational Bayesian method is more accurate in finding the true number of clusters in situations that are relevant to current and future microarray studies. We also compare the two criteria using freely available tumour microarray datasets and show that the variational Bayesian method is more sensitive to capturing biologically relevant structure.

We propose an application of variational Bayesian inference to the problem of blind estimation of single-input multiple-output (SIMO), finite-impulse-response (FIR) channels. The blind channel estimation problem has been extensively studied well over a decade. Nevertheless, there are some types of channel that cannot be successfully estimated by existing blind methods. One example of such a channel type is frequency-selective, exponentially-weighted Rayleigh fading channel. Our blind estimation approach to it is to incorporate prior knowledge on channel weighting in the form of channel prior distribution, then obtain the approximation to the posterior distribution of the channel using the variational Bayesian framework. Despite the error due to the mean field approximation involved in the variational framework, the derived algorithm has been verified by simulation to provide much improvements over the existing ones in a statistical sense.

The planar compound eye imaging system uses multiple sub-apertures to image the scene. Due to the constraint of the imaging sub-aperture size and spatial sampling rate of the image sensor, the image quality of each sub-aperture is low. How to fuse multiple sub-aperture images for a high-resolution image is an urgent problem. Multi-image super-resolution theory uses multiple images with complementary information to reconstruct high spatial resolution image. However, existing theories usually adopt the oversimplified motion model which is not suitable for planar compound eye imaging. If the existing multi-image super-resolution theory is directly applied to the resolution enhancement of the planar compound eye, the inaccurate motion estimation will reduce the performance of image resolution enhancement. In order to solve these problems, the motion model of the multi-image super-resolution is improved in the variational Bayesian framework, and the derived joint estimation algorithm is used to enhance the resolution of the planar compound eye. The correctness and effectiveness of the proposed method is verified by the simulation data experiments and the real compound eye data experiments.

Variational Bayesian inference based robust multiple measurement sparse signal recovery

The highly accurate state estimation of nonlinear systems is needed in many different application fields. In many existing nonlinear filters, the uncompensated nonlinear approximation error and widely used numerical sampling methods both seriously affect the estimation accuracy and numerical stability. In this paper, based on the variational Bayesian framework, a novel iterative nonlinear filter consisting of forward modeling and inverse estimation is proposed. In the forward modeling stage, a linear Gaussian regression process with distributed variational parameters (VPs) is designed to fit the whole nonlinear measurement likelihood. The distributed VPs consider both first and second moments information, so as to compensate for approximation error from the whole likelihood perspective, instead of only the nonlinear function approximation, which can further improve the accuracy of Bayesian posterior update. In the linear Gaussian regression process, the original nonlinear measurement function (NMF) is just viewed as a prior and no longer directly participates in the modeling of the measurement likelihood. Hence, in the inverse estimation stage, when maximizing the evidence lower bound to update the posteriors of the state and VPs, we can obtain closed‐form solutions without numerical sampling methods to approximate the integral of NMF required in many existing methods, which can guarantee numerical stability. Moreover, to avoid the random setting of hyperparameters and to block the propagation of the accumulated error, we propose two rules, estimation and lower bound consistencies, to conduct the hyperparameters' initialization at the beginning of iteration at each sampling time and also derive an optimization method to update hyperparameters during iteration. Finally, the performance of our proposed method is demonstrated in the nonlinear orbital estimation problems.

In this paper we propose a new method for EEG source localization. Solving the above problem usually requires choosing an appropriate regularization term. Our method is based on the bayesian approach, hence, the regularization term is closely connected to the prior distribution. The proposed prior distribution has sparse properties favoring focal EEG sources. In order to obtain an efficient algorithm we use the Variational Bayesian framework which provides us a tractable iterative algorithm of closed form equations. We compare the proposed method with well-known approaches of EEG source localization and the results, using synthetic and real EEG data, have shown that our method presents state-of-the-art performance, especially in cases where we expect few activated brain regions.

A variational Bayesian based robust cubature Kalman filter under dynamic model mismatch and outliers interference

Time-sync comments (TSCs) reveal a new way of extracting the online video tags. However, such TSCs have lots of noises due to users’ diverse comments, introducing great challenges for accurate and fast video tag extractions. In this article, we propose an unsupervised video tag extraction algorithm named Semantic Weight-Inverse Document Frequency (SW-IDF). Specifically, we first generate corresponding semantic association graph (SAG) using semantic similarities and timestamps of the TSCs. Second, we propose two graph cluster algorithms, i.e., dialogue-based algorithm and topic center-based algorithm, to deal with the videos with different density of comments. Third, we design a graph iteration algorithm to assign the weight to each comment based on the degrees of the clustered subgraphs, which can differentiate the meaningful comments from the noises. Finally, we gain the weight of each word by combining Semantic Weight (SW) and Inverse Document Frequency (IDF). In this way, the video tags are extracted automatically in an unsupervised way. Extensive experiments have shown that SW-IDF (dialogue-based algorithm) achieves 0.4210 F1-score and 0.4932 MAP (Mean Average Precision) in high-density comments, 0.4267 F1-score and 0.3623 MAP in low-density comments; while SW-IDF (topic center-based algorithm) achieves 0.4444 F1-score and 0.5122 MAP in high-density comments, 0.4207 F1-score and 0.3522 MAP in low-density comments. It has a better performance than the state-of-the-art unsupervised algorithms in both F1-score and MAP.

Different from popular neural networks using quasiconvex activations, non-monotonic networks activated by periodic nonlinearities have emerged as a more competitive paradigm, offering revolutionary benefits: 1) compactly characterizing high-frequency patterns; 2) precisely representing high-order derivatives. Nevertheless, they are also well-known for being hard to train, due to easily over-fitting dissonant noise and only allowing for tiny architectures (shallower than 5 layers). The fundamental bottleneck is that the periodicity leads to many poor and dense local minima in solution space. The direction and norm of gradient oscillate continually during error backpropagation. Thus non-monotonic networks are prematurely stuck in these local minima, and leave out effective error feedback. To alleviate the optimization dilemma, in this paper, we propose a non-trivial soft transfer approach. It smooths their solution space close to that of monotonic ones in the beginning, and then improve their representational properties by transferring the solutions from the neural space of monotonic neurons to the Fourier space of non-monotonic neurons as the training continues. The soft transfer consists of two core components: 1) a rectified concrete gate is constructed to characterize the state of each neuron; 2) a variational Bayesian learning framework is proposed to dynamically balance the empirical risk and the intensity of transfer. We provide comprehensive empirical evidence showing that the soft transfer not only reduces the risk of non-monotonic networks on over-fitting noise, but also helps them scale to much deeper architectures (more than 100 layers) achieving the new state-of-the-art performance.

BackgroundThe preoperative prediction of spread through air spaces (STAS) in patients with early-stage lung adenocarcinoma (LUAD) is crucial for selecting the appropriate surgical approach and improving patient outcomes. Previous research has confirmed that there is a significant correlation between consolidation-to-tumor ratio (CTR) and STAS. This study aimed to develop a Bayesian deep learning (DL) model based on the CTR prior to predict STAS in patients with stage IA LUAD.MethodsThis large-scale diagnostic study included patients with solitary primary invasive LUAD who underwent complete resection between November 2017 and October 2023. Enrolled patients were randomly assigned to training, validation, and test cohorts in a 7:2:1 ratio. Using a variational Bayesian inference framework, we developed a DL model based on the CTR prior (STAS-DLPrior CTR). The performance of STAS-DLPrior CTR was compared with another DL model without the CTR prior (STAS-DLNon-prior CTR) using the receiver operating characteristic (ROC) curve, calibration curve, decision curve analysis (DCA), and clinical impact curve (CIC).ResultsA total of 1,374 patients were included, with 961 in the training cohort, 275 in the validation cohort, and 138 in the test cohort. The results showed that CTR in the STAS-positive group was significantly higher than that in the STAS-negative group [0.63 (interquartile range, 0.36, 0.98) vs. 0.35 (interquartile range, 0.19, 0.60), P<0.001]. Compared to STAS-DLNon-prior CTR, the area under the ROC curve (AUC) tends to be higher for STAS-DLPrior CTR (0.831 vs. 0.731, P=0.06) in the validation cohort, and STAS-DLPrior CTR demonstrated a significantly higher AUC (0.858 vs. 0.637, P=0.008) in the test cohort. Additionally, the calibration curve suggested better calibration for STAS-DLPrior CTR. DCA and CIC also indicated that STAS-DLPrior CTR conferred higher clinical net benefit.ConclusionsThe proposed model based on the CTR prior offers significant advantages in predicting STAS in patients with stage IA LUAD, and incorporating doctors’ knowledge as priors can effectively guide the development of DL models.

The great success of deep learning (DL) has inspired researchers to develop more accurate and efficient symbol detectors for multi-input multi-output (MIMO) systems. Existing DL-based MIMO detectors, however, suffer several drawbacks. To address these issues, in this paper, we develop a model-driven DL detector based on variational Bayesian inference. Specifically, the proposed unrolled DL architecture is inspired by an inverse-free variational Bayesian learning framework which circumvents matrix inversion via maximizing a relaxed evidence lower bound. Two networks are respectively developed for independent and identically distributed (i.i.d.) Gaussian channels and arbitrarily correlated channels. The proposed networks, referred to as VBINet, have only a few learnable parameters and thus can be efficiently trained with a moderate amount of training samples. The proposed VBINet-based detectors can work in both offline and online training modes. An important advantage of our proposed networks over state-of-the-art MIMO detection networks such as OAMPNet and MMNet is that the VBINet can automatically learn the noise variance from data, thus yielding a significant performance improvement over the OAMPNet and MMNet in the presence of noise variance uncertainty. Simulation results show that the proposed VBINet-based detectors achieve competitive performance for both i.i.d. Gaussian and realistic 3GPP MIMO channels.

Electromagnetic imaging of inhomogeneous objects is a nonlinear inverse scattering problem which can be effectively handled by means of linear inversion methods or nonlinear optimization methods. In addition to being nonlinear, inverse scattering problems are also ill-posed which can be overcome through a regularization scheme by taking into account some kind of a priori information. However, it is not straightforward to incorporate such inhomogeneity prior information into conventional inversion methods. In this work, we propose an innovative method to solve the problem of imaging inhomogeneous objects with piecewise homogeneity. Under Born approximation (BA), this problem is cast into a linear inversion one which is then addressed by means of a hierarchical Bayesian inversion method. To account for the piecewise homogeneity of the image, a Gaussian mixture model (GMM) is developed to enforce the feature. Specifically, a hidden label variable which is associated with each pixel is introduced for the class of materials and the contrast of each pixel is modeled as a mixture of independent Gaussian priors. Then, the proposed hierarchical Bayesian inversion method incorporates the GMM to the variational Bayesian inference framework to approximate the posterior distribution of all the unknowns (the unknown contrast, hidden label variable, and other model parameters). In addition, the damped generalized approximate message passing (GAMP) is utilized to circumvent the matrix inversion, making the overall process very computationally efficient. Extensive numerical examples will be shown to demonstrate the effectiveness of the proposed method.

In this paper, we present a computer-aided follow-up (CAF) scheme to support physicians to track interval changes of pulmonary nodules on three dimensional (3D) CT images and to decide the treatment strategies without making any under or over treatment. Our scheme involves analyzing CT histograms to evaluate the volumetric distribution of CT values within pulmonary nodules. A variational Bayesian mixture modeling framework translates the image-derived features into an image-based risk score for predicting the patient recurrence-free survival. Through applying our scheme to follow-up 3D CT images of pulmonary nodules, we demonstrate the potential usefulness of the CAF scheme which can provide the trajectories that can characterize time interval changes of pulmonary nodules.

The Web brings an open-ended set of semantic relations. Discovering the significant types is very challenging. Unsupervised algorithms have been developed to extract relations from a corpus without knowing the relation types in advance, but most rely on tagging arguments of predefined types. One recently reported system is able to jointly extract relations and their argument semantic classes, taking a set of relation instances extracted by an open IE (Information Extraction) algorithm as input. However, it cannot handle polysemy of relation phrases and fails to group many similar (“synonymous”) relation instances because of the sparseness of features. In this paper, the authors present a novel unsupervised algorithm that provides a more general treatment of the polysemy and synonymy problems. The algorithm incorporates various knowledge sources which they will show to be very effective for unsupervised relation extraction. Moreover, it explicitly disambiguates polysemous relation phrases and groups synonymous ones. While maintaining approximately the same precision, the algorithm achieves significant improvement on recall compared to the previous method. It is also very efficient. Experiments on a real-world dataset show that it can handle 14.7 million relation instances and extract a very large set of relations from the Web.

Asset integrity management of ageing oil and gas assets is an ongoing challenge. This paper uses unsupervised algorithms (i.e. clustering technique) to identify carbon steel piping with increased probability of failure due to various internal corrosion mechanisms. The application used over 20 variables including wellhead planktonic bacterial counts, Fe2+ levels, oil and water production rates, historical Non-Destructive Testing (NDT) records, remaining life of downstream equipment, previous remediation data and geographical location data. An unsupervised machine learning clustering algorithm was written grounded in mathematical techniques of Principal Component Analysis (PCA) and k-means clustering. The probabilistic algorithm identified implicit patterns, which were then used to identify critical and non-critical piping clusters. Outputs from the clustering model were used to prioritise field measurements, and while these are ongoing there appears to be a good agreement with model predictions. The paper further discusses the measures that have a higher impact on the classification accuracy of the algorithm.