Blind separation and filtering using state space models

Blind Signal Separation II. Linear, Convolutive Combinations: II. Linear, Convolutive Combinations

We investigate the model for an industrial isothermal HDPE slurry reactor. The model, consisting of several nonlinear equations, can be linearized to give sets of linear time invariant state space model. The effectiveness of the linearized model is verified by the numerical simulations. A simple model predictive control scheme is constructed based on the linear state space model. The value of the melt index is obtained from the values of the manipulated and controlled variables generated from the control scheme. The control performance can be evaluated from the comparison between the computed melt index values and measured melt index values. The control scheme shows good tracking performance in the numerical simulations. We believe that the model developed in the present study can be effectively used to predict process variables as well as to control the melt index.

Independent component analysis (ICA) or blind signal separation (BSS) has become an increasing important research field due to its rapidly growing applications in various areas, such as telecommunication systems, sonar and radar systems, audio and acoustics, image enhancement and biomedical signal processing. First, a novel adaptive ICA (AICA) entropy optimization algorithm for finding pairs of simplified activation functions (SAF) is introduced. Then, the theoretical explanation is described. Finally we discuss the algorithm with a few existing representative methods. Experimental simulation results prove that the algorithm is effective at separating signals.

ERP is commonly obtained by averaging over segmented EEC epochs. In case artifacts are present in the raw EEC measurement, pre-processing is required to prevent the averaged ERP waveform being interfered by artifacts. The simplest pre-processing approach is by rejecting trials in which presence of artifact is detected. Alternatively artifact correction instead of rejection can be performed by blind source separation, so that waste of ERP trials is avoided. In this paper, we propose a second order statistics based blind source separation approach to ERP artifact correction. Comparing with blind separation using independent component analysis, second order statistics based method does not rely on higher order statistics or signal entropy, and therefore leads to more robust separation even if only short epochs are available.

In the last years Independent Component Analysis (ICA) has been applied with success in signal processing and many algorithms have been developed in order to perform ICA. In this paper we review some algorithms, like INFOMAX (Bell and Sejnowski 1995), extended-INFOMAX (Lee, Girolami and Sejniowski 1997), FastICA (OjA, and Hyvarinen 1999), that solve the ICA problem under the assumption of the linear mixture model. We also show an overview of the nonlinear ICA algorithms and we discuss the MISEP (Almeida 2003). In order to test the performances of the reviewed algorithms, we present some applications of ICA in biomedical signal processing. In particular the application of ICA to the electroencephalographic (EEG) and surface electromyographic (sEMG) recordings are shown.

Linear parametric state-space models are a ubiquitous tool for analyzing neural time series data, providing a way to characterize the underlying brain dynamics with much greater statistical efficiency than non-parametric data analysis approaches. However, neural time series data are frequently time-varying, exhibiting rapid changes in dynamics, with transient activity that is often the key feature of interest in the data. Stationary methods can be adapted to time-varying scenarios by employing fixed-duration windows under an assumption of quasi-stationarity. But time-varying dynamics can be explicitly modeled by switching state-space models, i.e., by using a pool of state-space models with different dynamics selected by a probabilistic switching process. Unfortunately, exact solutions for state inference and parameter learning with switching state-space models are intractable. Here we revisit a switching state-space model inference approach first proposed by Ghahramani and Hinton. We provide explicit derivations for solving the inference problem iteratively after applying a variational approximation on the joint posterior of the hidden states and the switching process. We introduce a novel initialization procedure using an efficient leave-one-out strategy to compare among candidate models, which significantly improves performance compared to the existing method that relies on deterministic annealing. We then utilize this state inference solution within a generalized expectation-maximization algorithm to estimate model parameters of the switching process and the linear state-space models with dynamics potentially shared among candidate models. We perform extensive simulations under different settings to benchmark performance against existing switching inference methods and further validate the robustness of our switching inference solution outside the generative switching model class. Finally, we demonstrate the utility of our method for sleep spindle detection in real recordings, showing how switching state-space models can be used to detect and extract transient spindles from human sleep electroencephalograms in an unsupervised manner.

Application of a Linear Gaussian State Space Model and the Kalman Filter in Estimating Expectations of the Natural Logarithmic Monthly Nav Returns of the Open – End Funds of Major Investment Banks.

Deep learning has been used to train neural networks to estimate the Remaining Useful Life (RUL) of a machine given sensor signals from that machine. This has resulted in some accurate state-of-the-art RUL estimation models. Here we present a novel way to construct a generative model to estimate the RUL and quantify its uncertainty. It takes the form of a Linear Gaussian State Space Model (LGSSM) and is trained using the Kalman Filter; hence, uncertainty is quantified using this LGSSM instead of by other popular methods like Monte Carlo Dropout or Deep Ensembles. This means we can train by directly using the marginal log-likelihood loss and don’t require multiple samples to represent the uncertainty. However, this method is limited to using Gaussian distributions to quantify uncertainty. We avoid needing to use non-linear variants of the filter by processing the sensors using a neural network to represent noncausal sensor sequences as a “control variable” in the LGSSM. This noncausal representation is shown to be important for achieving state-of-the-art performance. The model is tested on a turbofan engine and dust filter dataset. The code can be found on GitHub. 1 1 https://github.com/StarMarco/Kalman_DVAE • Deep Neural Networks encode time window of sensor signals. • These are used as control/forcing variables of a Linear Gaussian State Space Model (LGSSM). • The “observables" of the LGSSM are the Remaining Useful Life (RUL) values of the machine. • The Kalman filtering algorithm is used to estimate the marginal log-likelihood of these RUL estimates. • After training the LGSSM can be evaluated over time to estimate the RUL trajectories.

The technique of blind source separation (BSS) has been well studied. In this paper, we apply the BSS technique, particularly based on independent component analysis (ICA), to a meeting situation. The goal is to enhance the spoken utterances and to estimate the location of each speaker by means of multiple microphones. The technique may help us to take the minutes of a meeting.

In this paper, we define and characterize cointegrated solutions of continuous-time linear state-space models driven by Lévy processes. Cointegrated solutions of such models are shown to be representable as a sum of a Lévy process and a stationary solution of a linear state-space model, analogous to the Granger representation for cointegrated VAR models. Moreover, we prove that the class of cointegrated multivariate Lévy-driven autoregressive moving-average (MCARMA) processes, the continuous-time analogues of the classical vector ARMA processes, is equivalent to the class of cointegrated solutions of continuous-time linear state-space models. Necessary conditions for MCARMA processes to be cointegrated are given as well extending the results of Comte [Discrete and continuous time cointegration, J. Econometrics 88 (1999), pp. 207–226] for MCAR processes. The conditions depend only on the autoregressive polynomial if we have a minimal model. Finally, we investigate cointegrated continuous-time linear state-space models observed on a discrete time-grid and calculate their linear innovations. Based on the representation of the linear innovations, we derive an error correction form. The error correction form uses an infinite linear filter in contrast to the finite linear filter for VAR models. Some sufficient identifiable criteria are also given.

Current analytical techniques applied to functional magnetic resonance imaging (fMRI) data require a priori knowledge or specific assumptions about the time courses of processes contributing to the measured signals. Here we describe a new method for analyzing fMRI data based on the independent component analysis (ICA) algorithm of Bell and Sejnowski ([1995]: Neural Comput 7:1129-1159). We decomposed eight fMRI data sets from 4 normal subjects performing Stroop color-naming, the Brown and Peterson work/number task, and control tasks into spatially independent components. Each component consisted of voxel values at fixed three-dimensional locations (a component "map"), and a unique associated time course of activation. Given data from 144 time points collected during a 6-min trial, ICA extracted an equal number of spatially independent components. In all eight trials, ICA derived one and only one component with a time course closely matching the time course of 40-sec alternations between experimental and control tasks. The regions of maximum activity in these consistently task-related components generally overlapped active regions detected by standard correlational analysis, but included frontal regions not detected by correlation. Time courses of other ICA components were transiently task-related, quasiperiodic, or slowly varying. By utilizing higher-order statistics to enforce successively stricter criteria for spatial independence between component maps, both the ICA algorithm and a related fourth-order decomposition technique (Comon [1994]: Signal Processing 36:11-20) were superior to principal component analysis (PCA) in determining the spatial and temporal extent of task-related activation. For each subject, the time courses and active regions of the task-related ICA components were consistent across trials and were robust to the addition of simulated noise. Simulated movement artifact and simulated task-related activations added to actual fMRI data were clearly separated by the algorithm. ICA can be used to distinguish between nontask-related signal components, movements, and other artifacts, as well as consistently or transiently task-related fMRI activations, based on only weak assumptions about their spatial distributions and without a priori assumptions about their time courses. ICA appears to be a highly promising method for the analysis of fMRI data from normal and clinical populations, especially for uncovering unpredictable transient patterns of brain activity associated with performance of psychomotor tasks.

Independent Component Analysis (ICA) is a promising tool for the analysis of functional magnetic resonance imaging (fMRI) time series. In these studies, mostly assumed is a spatially independent component map of fMRI data (spatial ICA). In this paper, we assume that the temporal courses of the signal and noises are independent within a Tiny spatial domain (temporal ICA). Then with fast-ICA algorithm, spatially neighboring fMRI data were blindly separated into several temporal courses and were preassumed to be formed by a signal time course and several noise time courses where the signal has the largest correlation coefficient with the reference signal. The final functional imaging was completed for the signals obtained from each voxel. Simulations showed that compared with the spatial ICA method, the new temporal ICA method is more effective than the spatial ICA in detecting weak signal in a fMRI dataset. As background noise, the simulations include simulated Gaussian noise and fMRI data without stimulation. Finally, vivo fMRI tests showed that the excited areas evoked by a visual stimuli are mainly in the region of the primary visual cortex and that evoked by auditory stimuli are mainly in the region of the primary temporal cortex.

Objective: Magnetoencephalography source imaging (MSI) of interictal epileptiform discharges (IED) is a useful presurgical tool in the evaluation of drug-resistant frontal lobe epilepsy (FLE) patients. Yet, failures in MSI can arise related to artifacts and to interference of background activity. Independent component analysis (ICA) is a popular denoising procedure but its clinical application remains challenging, as the selection of multiple independent components (IC) is controversial, operator dependent, and time consuming. We evaluated whether selecting only one IC of interest based on its similarity with the average IED field improves MSI in FLE.Methods: MSI was performed with the equivalent current dipole (ECD) technique and two distributed magnetic source imaging (dMSI) approaches: minimum norm estimate (MNE) and coherent Maximum Entropy on the Mean (cMEM). MSI accuracy was evaluated under three conditions: (1) ICA of continuous data (Cont_ICA), (2) ICA at the time of IED (IED_ICA), and (3) without ICA (No_ICA). Localization performance was quantitatively measured as actual distance of the source maximum in relation to the focus (Dmin), and spatial dispersion (SD) for dMSI.Results: After ICA, ECD Dmin did not change significantly (p > 0.200). For both dMSI techniques, ICA application worsened the source localization accuracy. We observed a worsening of both MNE Dmin (p < 0.05, consistently) and MNE SD (p < 0.001, consistently) for both ICA approaches. A similar behaviour was observed for cMEM, for which, however, Cont_ICA seemed less detrimental.Conclusion: We demonstrated that a simplified ICA approach selecting one IC of interest in combination with distributed magnetic source imaging can be detrimental. More complex approaches may provide better results but would be rather difficult to apply in real-world clinical setting. In a broader perspective, caution should be taken in applying ICA for source localization of interictal activity. To ensure optimal and useful results, effort should focus on acquiring good quality data, minimizing artifacts, and determining optimal candidacy for MEG, rather than counting on data cleaning techniques.

A geometric algorithm for overcomplete linear ICA

The purpose of this paper is to contrast, in terms of standard criteria for statistical independence, cumulant-based methods of independent component analysis (ICA) with a potentially more robust blind signal separation technique called BLISS. This is able to separate independent non-Gaussian co-channel signals from multisensor data using only the joint probability distributions of instantaneous linear mixtures of those signals. BLISS is also able, without prior array calibrations or training waveforms, to estimate individual steering vectors including unknown mutual coupling and multipath. We point out fundamental reasons for the difficulty of comparing the performance of different ICA algorithms on finite duration practical data. We also propose a novel method for applying real-valued ICA to complex-valued data. By separately estimating in-phase and quadrature un-mixing parameters, we avoid the difficulty of selecting a subset of real and complex-valued cumulants. To justify our approach, we extend the definition of independence to the complex-valued case.