Stochastic Expectation Maximization Algorithm for Long-Memory Fast-Fading Channels

Bayesian model updating provides a rigorous framework to account for uncertainty induced by lack of knowledge about engineering systems in their respective mathematical models through updates of the joint probability density function (PDF), the so-called posterior PDF, of the unknown model parameters. The Markov chain Monte Carlo (MCMC) methods are currently the most popular approaches for generating samples from the posterior PDF. However, these methods often found wanting when sampling from difficult distributions (e.g., high-dimensional PDFs, PDFs with flat manifolds, multimodal PDFs, and very peaked PDFs). This paper introduces a new multi-level sampling approach for Bayesian model updating, called Sequential Gauss-Newton algorithm, which is inspired by the Transitional Markov chain Monte Carlo (TMCMC) algorithm. The Sequential Gauss-Newton algorithm improves two aspects of TMCMC to make an efficient and effective MCMC algorithm for drawing samples from difficult posterior PDFs. First, the statistical efficiency of the algorithm is enhanced by use of the systematic resampling scheme. Second, a new MCMC algorithm, called Gauss-Newton MCMC algorithm, is proposed which is essentially an M-H algorithm with a Gaussian proposal PDF tailored to the posterior PDF using the gradient and Hessian information of the negative log posterior. The effectiveness of the proposed algorithm for solving the Bayesian model updating problem is illustrated using three examples with irregularly shaped posterior PDFs.

When a Markov Chain Monte Carlo (MCMC) method is applied to solve signal-processing problems, it is commonly implemented using Gibbs sampler. The implementation of Gibbs sampler requires the availability of full conditional probability density functions (pdfs) of all the parameters of interest of a problem. For some problems, however, the full conditional pdfs of all the parameters of interest are not readily available. In such cases, Metropolis-Hastings method can be incorporated within a Gibbs sampler to draw samples from the parameters whose full conditional pdf cannot be analytically determined. This paper demonstrates the application of such an algorithm, known as Metropolis-Hastings-within Gibbs, by considering the problem of joint data detection and channel estimation of a single-hop relay-based communication system. By formulating the signal model of the transmission process in alternative ways, we develop two algorithms for the problem. Moreover, simulation results of the two algorithms are provided to illustrate their effectiveness.

Maximum a posteriori (MAP) based iterative data detection and pilot symbol assisted channel estimation of the single parity check product coded multiple antenna wireless communication system is discussed. In iterative data-detection and channel-estimation algorithms, the channel estimator and the data detector recursively exchange the information order to improve the system performance. The work in this paper uses the single-parity-check code (SPC) as component code, parallel concatenated, of product code (PC). The SPC-PC codes are high-rate codes when reasonable block lengths are used. They are simple in both encoding and decoding, and are shown to produce good performance while decoded iteratively. By taking the output of the SPC-PC encoder into two defined-interleavers and transmitting through multiple transmit antennas, will provide space and time diversity in the system. This scheme would be BLAST-like system. Pilot-symbol aided channel estimation is used to measure the channel impulse response. Following the iteration of the decoding process, the channel impulse response will be updated by employing the extrinsic information output of the decoder. The convergence of this channel impulse response update will investigate.

An asynchronous multiuser communication system is considered where K users share an additive white Gaussian channel. Binary phase-shift keying and direct sequence spread spectrum signaling is employed for data modulation. In this work, we propose an algorithm for near-far resistant joint parameter estimation and data detection for multiuser systems: the successive interference cancellation scheme employs a stochastic approximation (SA) method for estimating delays, the expectation-maximization (EM) algorithm for obtaining carrier phase and amplitude estimates, and a multistage algorithm for data detection. Using an SA-scheme, the complexity of the receiver structure can be considerably reduced. Monte Carlo simulations illustrate the capability of the system to obtain parameter estimates even in the presence of strong interferers such that the near-far effect can be overcame.

In MIMO systems, where multiple antennas are used at both transmitter and receiver to achieve high spectral efficiency, channel impulse responses are often assumed to be constant over a block or packet. This assumption of block stationarity on channels is valid for most fixed wireless scenarios. However, for communications in a high mobility environment, the assumption will result in considerable performance degradation. In this paper, we focus on channel estimation for a MIMO system with OFDM transmission technique. In our system, pilots are placed on N/sub P/ subcarriers for a novel channel estimation at the receiver with Kalman filters. With the channels estimated by a Kalman filter, we apply the QRD-M algorithm for suboptimal data detection with reasonable computational cost. The outputs of data detection are fed back to another Kalman filter for an improved channel estimation. By alternatively and iteratively using these two Kalman filters, a better channel estimation can be obtained. A simulation result of the bit error rate shows the improvement of 10 dB in SNR with four iterations over noniteration.

This paper proposes a new iterative algorithm for OFDM joint data detection and phase noise (PHN) cancellation based on minimum mean square prediction error. We particularly highlight the problem of "overfitting" such that the iterative approach may converge to a trivial solution. Although it is essential for this joint approach, the overfitting problem was relatively less studied in existing algorithms. In this paper, specifically, we apply a hard decision procedure at every iterative step to overcome the overfitting. Moreover, compared with existing algorithms, a more accurate Pade approximation is used to represent the phase noise, and finally a more robust and compact fast process based on givens rotation is proposed to reduce the complexity to a practical level. Numerical simulations are also given to verify the proposed algorithm.

In MIMO systems, channel estimation is important to distinguish transmit signals from multiple transmit antennas. The space-alternating generalized expectation-maximization (SAGE) algorithm is known to be good for channel estimation and data detection. However, the SAGE algorithm has not been applied to MIMO systems. In this paper, we propose a SAGE algorithm for the channel estimation and data detection in MIMO systems. In addition, we propose a simplified SAGE algorithm for the channel estimation and data detection by tracking the channel variation in MIMO systems. In the simplified SAGE algorithm, we divide a transmit frame into some subblocks and apply the SAGE algorithm to each subblock, and we use the channel estimates in the previous subblock as the initial channel estimates in the current subblock. According to the division of the transmit frame, the computational complexity is decreased. In addition, the simplified SAGE algorithm can track the channel variation by using the channel estimates transferred between the subblocks.

In this paper, we propose a novel receiver design for orthogonal frequency division multiplexing (OFDM) transmission over doubly (time- and frequency-) spread channels, where the receiver employs a turbo message passing (TMP) algorithm for joint channel estimation and data detection. We show that the proposed TMP-based receiver is able to make more efficient use of the prior information about the channel and the data for inter-carrier interference (ICI) cancellation than the existing iterative receivers. The proposed TMP algorithm can be further combined with a soft-input soft-output (SISO) decoder for joint channel estimation, data detection and decoding. Numerical results are presented to demonstrate the performance superiority of our proposed TMP-based OFDM receiver.

When the number of base station (BS) antennas is considerably larger than the number of user terminals (UTs), a simple linear minimum mean-square-error (LMMSE) data detection algorithm is near-optimal for uplink massive multiuser multiple-input and multiple-output (MIMO) systems. However, the LMMSE detector suffers significant performance loss when the number of UTs is comparable to the number of BS antennas. To alleviate the performance loss caused by the decrease in the BS-antenna-to-UT ratio, known data detection algorithm, namely “TASER,” can yield bit error rate (BER) performance close to that of the optimal detector in symmetric massive multiuser MIMO systems, where the BS-antenna-to-UT ratio is one. However, this TASER algorithm can only handle binary and quadratic phase-shift keying (PSK) modulations. Moreover, the computational complexity of the TASER algorithm is still remarkably high. Thus, a novel data detection algorithm is proposed in this study to reduce the computational complexity of the detection algorithm while achieving acceptable BER performance. The proposed algorithm is developed from the retraction-based Riemannian manifold optimization (RMO) framework for symmetric massive multiuser MIMO systems. Unlike the TASER-based detector, the proposed RMO-based detector can be applied for general $M$ -ary PSK modulations. Simulation results show that the proposed detector significantly outperforms the LMMSE detector in terms of BER performance. In addition, the required complexity of RMO is substantially lower than that of TASER. These findings indicate that the proposed RMO-based detector exhibits a highly desirable tradeoff between BER performance and complexity.

Recent progress using geometry in the design of efficient Markov chain Monte Carlo (MCMC) algorithms have shown the effectiveness of the Fisher Riemannian structure. Furthermore, the theory of the underlying geometry of spaces of statistical models has made an important breakthrough by extending the classical theory on exponential families to their closures, the so-called extended exponential families. This paper looks at the underlying geometry of the Fisher information, in particular its limiting behaviour near boundaries, which illuminates the excellent behaviour of the corresponding geometric MCMC algorithms. Further, the paper shows how Fisher geodesics in extended exponential families smoothly attach the boundaries of extended exponential families to their relative interior. We conjecture that this behaviour could be exploited for trans-dimensional MCMC algorithms.

We propose a joint channel estimation and data detection (JED) algorithm for densely-populated cell-free massive multiuser (MU) multiple-input multiple-output (MIMO) systems, which reduces the channel training overhead caused by the presence of hundreds of simultaneously transmitting user equipments (UEs). Our algorithm iteratively solves a relaxed version of a maximum a-posteriori JED problem and simultaneously exploits the sparsity of cell-free massive MU-MIMO channels as well as the boundedness of QAM constellations. In order to improve the performance and convergence of the algorithm, we propose methods that permute the access point and UE indices to form so-called virtual cells, which leads to better initial solutions. We assess the performance of our algorithm in terms of root-mean-squared-symbol error, bit error rate, and mutual information, and we demonstrate that JED significantly reduces the pilot overhead compared to orthogonal training, which enables reliable communication with short packets to a large number of UEs.

Bayesian model updating has received considerable attention and has been extensively used in structural damage detection. It provides a rigorous statistical framework for realizing structural system identification and characterizing uncertainties associated with modeling and measurements. The Markov Chain Monte Carlo (MCMC) is a promising tool for inferring the posterior distribution of model parameters to avoid the intractable evaluation of multi-dimensional integration. However, the efficacy of most MCMC techniques suffers from the curse of parameter dimension, which restricts the application of Bayesian model updating to the damage detection of large-scale systems. In addition, there are several MCMC techniques that require users to properly choose application-specific models, based on the understanding of algorithm mechanisms and limitations. As seen in the literature, there is a lack of comprehensive work that investigates the performances of various MCMC algorithms in their application of structural damage detection. In this study, the Differential Evolutionary Adaptive Metropolis (DREAM), a multi-chain MCMC, is explored and adapted to Bayesian model updating. This paper illustrates how DREAM is used for model updating with many uncertainty parameters (i.e., 40 parameters). Furthermore, the study provides a tutorial to users who may be less experienced with Bayesian model updating and MCMC. Two advanced single-chain MCMC algorithms, namely, the Delayed Rejection Adaptive Metropolis (DRAM) and Transitional Markov Chain Monte Carlo (TMCMC), and DREAM are elaborately introduced to allow practitioners to understand better the concepts and practical implementations. Their performances in model updating and damage detection are compared through three different engineering applications with increased complexity, e.g., a forty-story shear building, a two-span continuous steel beam, and a large-scale steel pedestrian bridge.

Clustering is a fundamental problem in statistics and machine learning, whose solution is commonly computed by the Expectation-Maximization (EM) method, which finds a locally optimal solution for an objective function called log-likelihood. Since the surface of the log-likelihood function is non convex, a stochastic search with Markov Chain Monte Carlo (MCMC) methods can help escaping locally optimal solutions. In this article, we tackle two fundamental conflicting goals: Finding higher quality solutions and achieving faster convergence. With that motivation in mind, we introduce an efficient algorithm that combines elements of the EM and MCMC methods to find clustering solutions that are qualitatively better than those found by the standard EM method. Moreover, our hybrid algorithm allows tuning model parameters and understanding the uncertainty in their estimation. The main issue with MCMC methods is that they generally require a very large number of iterations to explore the posterior of each model parameter. Convergence is accelerated by several algorithmic improvements which include sufficient statistics, simplified model parameter priors, fixing covariance matrices and iterative sampling from small blocks of the data set. A brief experimental evaluation shows promising results.

Sampling across tree space is one of the major challenges in Bayesian phylogenetic inference using Markov chain Monte Carlo (MCMC) algorithms. Standard MCMC tree moves consider small random perturbations of the topology, and select from candidate trees at random or based on the distance between the old and new topologies. MCMC algorithms using such moves tend to get trapped in tree space, making them slow in finding the globally most probable trees (known as “convergence”) and in estimating the correct proportions of the different types of them (known as “mixing”). Here, we introduce a new class of moves, which propose trees based on their parsimony scores. The proposal distribution derived from the parsimony scores is a quickly computable albeit rough approximation of the conditional posterior distribution over candidate trees. We demonstrate with simulations that parsimony-guided moves correctly sample the uniform distribution of topologies from the prior. We then evaluate their performance against standard moves using six challenging empirical data sets, for which we were able to obtain accurate reference estimates of the posterior using long MCMC runs, a mix of topology proposals, and Metropolis coupling. On these data sets, ranging in size from 357 to 934 taxa and from 1740 to 5681 sites, we find that single chains using parsimony-guided moves usually converge an order of magnitude faster than chains using standard moves. They also exhibit better mixing, that is, they cover the most probable trees more quickly. Our results show that tree moves based on quick and dirty estimates of the posterior probability can significantly outperform standard moves. Future research will have to show to what extent the performance of such moves can be improved further by finding better ways of approximating the posterior probability, taking the trade-off between accuracy and speed into account. [Bayesian phylogenetic inference; MCMC; parsimony; tree proposal.]

One of the most common types of models that helps us to understand neuron behavior is based on the Hodgkin–Huxley ion channel formulation (HH model). A major challenge with inferring parameters in HH models is non-uniqueness: many different sets of ion channel parameter values produce similar outputs for the same input stimulus. Such phenomena result in an objective function that exhibits multiple modes (i.e., multiple local minima). This non-uniqueness of local optimality poses challenges for parameter estimation with many algorithmic optimization techniques. HH models additionally have severe non-linearities resulting in further challenges for inferring parameters in an algorithmic fashion. To address these challenges with a tractable method in high-dimensional parameter spaces, we propose using a particular Markov chain Monte Carlo (MCMC) algorithm, which has the advantage of inferring parameters in a Bayesian framework. The Bayesian approach is designed to be suitable for multimodal solutions to inverse problems. We introduce and demonstrate the method using a three-channel HH model. We then focus on the inference of nine parameters in an eight-channel HH model, which we analyze in detail. We explore how the MCMC algorithm can uncover complex relationships between inferred parameters using five injected current levels. The MCMC method provides as a result a nine-dimensional posterior distribution, which we analyze visually with solution maps or landscapes of the possible parameter sets. The visualized solution maps show new complex structures of the multimodal posteriors, and they allow for selection of locally and globally optimal value sets, and they visually expose parameter sensitivities and regions of higher model robustness. We envision these solution maps as enabling experimentalists to improve the design of future experiments, increase scientific productivity and improve on model structure and ideation when the MCMC algorithm is applied to experimental data.