Robust Least-Squares Estimation With a Relative Entropy Constraint

The active vibration control of cable net structures by piezoelectric stacked actuators is the subject of the present work. The focus of this paper lies on the proposal of a nonlinear method to address the vibration problem for such structures in the presence of uncertainties. To this end, a cable net structure with integrated piezoelectric stacked actuators is considered. For the structure, a nominal model is derived through the finite element method, and an uncertain model is constructed on the basis of the nominal model by taking into consideration the uncertainties. Based on the established uncertain model, a nonlinear state feedback control is deduced through the Lyapunov stability theory. This control guarantees that every uncertain system response enters and remains within a certain neighborhood of the zero state after a finite interval of time. To demonstrate the superiority of the proposed nonlinear control, a linear control based on the linear quadratic regulator is presented for comparison. Displacement responses and control voltages of the uncertain system under an impulse excitation are simulated for both the nonlinear and linear control cases. It is observed that the nonlinear method is feasible and robust as it stabilizes the uncertain system to a smaller size of the region of ultimate boundedness. Besides, the nonlinear approach improves the performance of the uncertain system by reducing the vibration to a lower amplitude within a shorter time.

In this dissertation, a novel approach of integrating template matching with statistical modeling is proposed to improve continuous speech recognition. Hidden Markov Modeling (HMMs) has been the dominant approach in statistical speech recognition since it provides a principled way of jointly modeling speech spectral variations and time dynamics. However, HMMs have the shortcoming of assuming the observations being independent within each state, which makes it ineffective in modeling the details of speech temporal evolutions that are important for characterizing nonstationary speech sounds. Template-based methods make comparisons between a test pattern and the templates derived from training data, and therefore they are able to capture speech dynamics and time correlation of speech frames better than HMM based methods. However, template matching requires large memory space and computational time since feature vectors of training data need to be stored in computer memory for access at the recognition stage, which is difficult in large vocabulary continuous speech recognition (LVCSR). Our proposed approach takes advantages of both statistical modeling and template matching, which overcomes the weakness of conventional template-based method and is feasible for LVCSR. We use multiple Gaussian Mixture Model (GMM) indices to represent each frame of speech templates, and define the template unit to be context-dependent phone segments (triphone context). We also use phonetic decision trees borrowed from those commonly used in HMMs to tie triphone templates and predict triphones unseen in training data. Two local distances, log likelihood ratio (LLR) and Kullback-Leibler (KL) divergence, are proposed for dynamic time warping (DTW) based template matching. In order to reduce computational complexity and storage space, we propose methods of minimum distance template selection (MDTS) and maximum log-likelihood template selection (MLTS), and investigate a template compression method on top of template selection to further improve recognition performance. The template based methods were used to rescore lattices generated by baseline HMMs on the tasks of TIMIT continuous phone recognition and teleheath LVCSR and experimental results demonstrated that the proposed approach of integrating template matching with statistical modeling significantly improved recognition performances over the HMM baselines. The template selection methods also provided significant recognition accuracy improvements over the HMM baseline while largely reducing the computation and storage complexities. When all templates or MDTS were used, using the LLR local distance obtained better recognition performance than the KL divergence local distance. For MLTS and template compression, KL divergence local distance provided better performance than the LLR local distance, and the template compression method made further improvements over KL based MLTS. Since the templates were constructed based on the GMM indices extracted from HMM baselines, we also validated the effectiveness of the proposed template methods based on enhanced HMM baselines. Experimental results showed that LLR based all template method was able to consistently improve TIMIT phone recognition accuracies based on four enhanced HMM baselines. Prosodic features such as duration, energy, and pitch can reflect longer span information of speech than conventional single frame vectors but they have commonly been ignored by HMMs. Template based methods provide possibilities to conveniently integrate prosodic features into speech recognition, which has not been well studied in the past. In this dissertation, we investigate combining template based methods with the speech prosodic features of duration, energy and pitch to further improve speech recognition accuracy. The scores of prosodic information were computed by a GMM based method and a non-parametric method, and the prosodic scores were combined with the acoustic scores in triphone template matching. Experimental results obtained on the telehealth task showed that prosodic information had positive effects on vowel sound recognition.

We study the Cauchy problem for the retarded functional differential equations that model the dynamics of some living systems. We find certain conditions ensuring the existence, uniqueness, and nonnegativity of solutions on finite and infinite time intervals. We obtain upper bounds for solutions and prove the continuous dependence of solutions on the initial data on finite time intervals.

Great efforts have been devoted to the intelligent control of autonomous systems. Yet, most of existing methods fail to effectively handle the uncertainty of their environment and models. Uncertain locations of dynamic obstacles pose a major challenge for their optimal control and safety, while their linearization or simplified system models reduce their actual performance. To address them, this paper presents a new model predictive control framework with finite samples and a Gaussian model, resulting in a chance-constrained program. Its nominal model is combined with a Gaussian process. Its residual model uncertainty is learned. The resulting method addresses an efficiently solvable approximate formulation of a stochastic optimal control problem by using approximations for efficient computation. There is no perfect distribution knowledge of a dynamic obstacle's location uncertainty. Only finite samples from sensors or past data are available for moment estimation. We use the uncertainty propagation of a system's state and obstacles' locations to derive a general collision avoidance condition under tight concentration bounds on the error of the estimated moments. Thus, this condition is suitable for different obstacles, e.g., bounding box and ellipsoid obstacles. We provide proved guarantees on the satisfaction of the chance-constraints corresponding to the nominal yet unknown moments. Simulation examples of a vehicle's control are used to show that the proposed method can well realize autonomous control and obstacle avoidance of a vehicle, when it operates in an uncertain environment with moving obstacles. It outperforms the existing moment methods in both performance and computational time.

This article addresses the robust estimation of the output layer linear parameters in a radial basis function network (RBFN). A prominent method used to estimate the output layer parameters in an RBFN with the predetermined hidden layer parameters is the least-squares estimation, which is the maximum-likelihood (ML) solution in the specific case of the Gaussian noise. We highlight the connection between the ML estimation and minimizing the Kullback-Leibler (KL) divergence between the actual noise distribution and the assumed Gaussian noise. Based on this connection, a method is proposed using a variant of a generalized KL divergence, which is known to be more robust to outliers in the pattern recognition and machine-learning problems. The proposed approach produces a surrogate-likelihood function, which is robust in the sense that it is adaptive to a broader class of noise distributions. Several signal processing experiments are conducted using artificially generated and real-world data. It is shown that in all cases, the proposed adaptive learning algorithm outperforms the standard approaches in terms of mean-squared error (MSE). Using the relative increase in the MSE for different noise conditions, we compare the robustness of our proposed algorithm with the existing methods for robust RBFN training and show that our method results in overall improvement in terms of absolute MSE values and consistency.

Coherence and causality measures are often used to analyze the influence of one region on another during analysis of functional brain networks. The analysis methods usually involve a regression problem, where the signal of interest is decomposed into a mixture of regressor and a residual signal. In this paper, we revisit this basic problem and present solutions that provide the minimal-entropy residuals for different types of regression filters, such as causal, instantaneously causal, and noncausal filters. Using optimal prediction theory, we derive several novel frequency-domain expressions for partial coherence, causality, and conditional causality analysis. In particular, our solution provides a more accurate estimation of the frequency-domain causality compared with the classical Geweke causality measure. Using synthetic examples and in vivo resting-state functional magnetic resonance imaging data from the human connectome project, we show that the proposed solution is more accurate at revealing frequency-domain linear dependence among high-dimensional signals.

Supermarket models with different servers have become a key in modelling resource management of stochastic networks, such as computer networks, manufacturing systems, transportation networks, and healthcare systems. However, the different servers always make analysis of such a supermarket model more interesting, difficult and challenging. This paper provides a novel method for analysing the supermarket models with different servers through a multi-dimensional continuous-time Markov reward process. Firstly, some utility functions are constructed for designing the routine selection mechanism according to the queue lengths, the service rates, and the probability of individual preference. Secondly, using the state jump points of the continuous-time Markov reward process, some segmented stochastic integrals of the random reward function are established by means of an event-driven technique. Based on this, the mean of the random reward function in a finite time interval is computed, and the mean of the discounted random reward function in an infinite time interval can also be calculated. Finally, some simulation experiments are given to indicate how the expected queue length of each server depends on some key parameters of this supermarket model.

Probability theory deals with limit theorems, which consider limits (when time tends to infinity) of some functions (observables) on a sample space, or averages of these observables over an infinite time interval. But what is happening in a finite time or over finite time intervals? Such questions, important for virtually all applications, seem to be intractable mathematically (and generally sound unreasonable). For instance, equilibrium statistical mechanics deals with phase transitions (a number of equilibrium probability distributions/states) rather than time evolution, while nonequilibrium statistical mechanics is concerned with convergence of nonequilibrium states to equilibrium ones. Again, such processes occur on infinite time intervals. It turns out, however, that there are natural and reasonable questions about finite time dynamics of random and deterministic chaotic systems, which can be answered and, moreover, rigorously answered. This allows one to make predictions about a finite time evolution of such systems.

Controlled stationary Markov chains with a finite number of states and possible actions and continuous time have been studied by many authors. It is well known that a control maximizing the expected reward in a finite time interval can be obtained from the equation of dynamic programming, and that there exists a stationary control giving maximal (discounted or average) reward in an infinite time interval. In this paper controlled Markov chains whose transition probabilities are periodic in time are considered. It is shown that there exists a periodic control giving maximal expected future reward, and that it can be approximatively calculated by making a discretization in time. An example of a system that can be described by such a process is a service system or an inventory whose input is, e.g., a Poisson process with an intensity that varies seasonally.

This paper considers the estimation methods for dynamic panel data (DPD) models with fixed effects, which suggested in econometric literature, such as least squares (LS) and generalized method of moments (GMM). These methods obtain biased estimators for DPD models. The LS estimator is inconsistent when the time dimension (T) is short regardless of the cross-sectional dimension (N). Although consistent estimates can be obtained by GMM procedures, the inconsistent LS estimator has a relatively low variance and hence can lead to an estimator with lower root mean square error after the bias is removed. Therefore, we discuss in this paper the different methods to correct the bias of LS and GMM estimations. The analytical expressions for the asymptotic biases of the LS and GMM estimators have been presented for large N and finite T. Finally; we display new estimators that presented by Youssef and Abonazel [40] as more efficient estimators than the conventional estimators.

We present sufficient frameworks for the uniform-in-time nonrelativistic limits for the relativistic Cucker–Smale (RCS) model and the relativistic kinetic Cucker–Smale (RKCS) equation. For the RCS model, one can easily show that the difference between the solutions to the RCS model and the CS model can be bounded by a quantity proportional to the exponential of time and inversely proportional to some power of the speed of light via a standard Grönwall-type differential inequality. However, this finite-in-time nonrelativistic limit result cannot be used in a uniform-in-time estimate due to the exponential factor of lifespan of solution as it is. For the uniform-in-time nonrelativistic limit, we split the deviation functional between the relativistic solution and the nonrelativistic solution into two parts (finite-time interval and infinite-time interval). In the finite-time interval, the deviation functional is bounded by a finite-in-time nonrelativistic limit result, and then, after a finite time, we use asymptotic flocking estimates with the same asymptotic momentum-like quantity for the RCS model and the CS model to show that the deviation functional can be made as small as possible. In this manner, we can derive a uniform-in-time nonrelativistic limit for the RCS model. For the RKCS equation, we use a uniform-in-time mean-field limit in a measure theoretic framework and a uniform-in-time nonrelativistic limit result for the RCS model to derive a uniform-in-time nonrelativistic limit for the RKCS equation.

A note on least-squares estimates from likelihood ratios

In statistical inference, the information-theoretic performance limits can often be expressed in terms of a statistical divergence between the underlying statistical models (e.g., in binary hypothesis testing, the error probability is related to the total variation distance between the statistical models). As the data dimension grows, computing the statistics involved in decision-making and the attendant performance limits (divergence measures) face complexity and stability challenges. Dimensionality reduction addresses these challenges at the expense of compromising the performance (the divergence reduces by the data-processing inequality). This paper considers linear dimensionality reduction such that the divergence between the models is maximally preserved. Specifically, this paper focuses on Gaussian models where we investigate discriminant analysis under five f-divergence measures (Kullback–Leibler, symmetrized Kullback–Leibler, Hellinger, total variation, and ). We characterize the optimal design of the linear transformation of the data onto a lower-dimensional subspace for zero-mean Gaussian models and employ numerical algorithms to find the design for general Gaussian models with non-zero means. There are two key observations for zero-mean Gaussian models. First, projections are not necessarily along the largest modes of the covariance matrix of the data, and, in some situations, they can even be along the smallest modes. Secondly, under specific regimes, the optimal design of subspace projection is identical under all the f-divergence measures considered, rendering a degree of universality to the design, independent of the inference problem of interest.

This letter presents a single-channel speech dereverberation approach using a non-causal minimum variance distortionless response (MVDR) filter. The non-causal filter is adopted to utilize the additional information of the desired signal that lies in subsequent frames. Note that the desired signal output has minimal distortion due to the introduction of the MVDR criterion. The proposed system further suppresses the late reverberation by employing a statistical reverberant model. Experimental results demonstrate the superiority of the proposed algorithm to conventional approaches.

New version of Optimal Homotopy Asymptotic Method for the solution of nonlinear boundary value problems in finite and infinite intervals