Linear Additive Markov Processes

The Markov process amplitude (MPA) EEG model effectively representing spontaneous brain activity of the EEG was introduced. The relationship between the electrical mechanism for EEG generation and the proposed model was also investigated. The MPA EEG model was described by the sinusoidal waves with the randomly fluctuating amplitude of the first-order Markov process. The parameters of the MPA EEG model were determined optimally based on the real EEG records. The results of model outputs in the frequency domain demonstrated an excellent fit with the power spectrum of the corresponding EEG. The simulated model signal in the time domain also showed good agreement with the EEG time series. The satisfactory results from the MPA EEG model suggest its possible applicability in clinical practice. Furthermore, from the high goodness of fit, the authors think that the neurons oscillate at fixed frequencies and are modulated by synaptic interactions in accordance with the first-order Markov process.

In this paper, we develop a new "robust mixing" framework for reasoning about adversarially modified Markov Chains (AMMC). Let $\mathbb{P}$ be the transition matrix of an irreducible Markov Chain with stationary distribution $\pi$. An adversary announces a sequence of stochastic matrices $\{\mathbb{A}_t\}_{t > 0}$ satisfying $\pi\mathbb{A}_t = \pi$. An AMMC process involves an application of $\mathbb{P}$ followed by $\mathbb{A}_t$ at time $t$. The robust mixing time of an ergodic Markov Chain $\mathbb{P}$ is the supremum over all adversarial strategies of the mixing time of the corresponding AMMC process. Applications include estimating the mixing times for certain non-Markovian processes and for reversible liftings of Markov Chains. Non-Markovian card shuffling processes: The random-to-cyclic transposition process is a non-Markovian card shuffling process, which at time $t$, exchanges the card at position $L_t := t {\pmod n}$ with a random card. Mossel, Peres and Sinclair (2004) showed a lower bound of $(0.0345+o(1))n\log n$ for the mixing time of the random-to-cyclic transposition process. They also considered a generalization of this process where the choice of $L_t$ is adversarial, and proved an upper bound of $C n\log n + O(n)$ (with $C \approx 4\times 10^5$) on the mixing time. We reduce the constant to $1$ by showing that the random-to-top transposition chain (a Markov Chain) has robust mixing time $\leq n\log n + O(n)$ when the adversarial strategies are limited to holomorphic strategies, i.e. those strategies which preserve the symmetry of the underlying Markov Chain. We also show a $O(n\log^2 n)$ bound on the robust mixing time of the lazy random-to-top transposition chain when the adversary is not limited to holomorphic strategies. Reversible liftings: Chen, Lovasz and Pak showed that for a reversible ergodic Markov Chain $\mathbb{P}$, any reversible lifting $\mathbb{Q}$ of $\mathbb{P}$ must satisfy $\mathcal{T}(\mathbb{P}) \leq \mathcal{T}(\mathbb{Q})\log (1/\pi_*)$ where $\pi_*$ is the minimum stationary probability. Looking at a specific adversarial strategy allows us to show that $\mathcal{T}(\mathbb{Q}) \geq r(\mathbb{P})$ where $r(\mathbb{P})$ is the relaxation time of $\mathbb{P}$. This gives an alternate proof of the reversible lifting result and helps identify cases where reversible liftings cannot improve the mixing time by more than a constant factor.

An expanded form of the third-order Markov undulation model is developed which includes undulation of the geoid, vertical deflections, gravity anomaly, and the anomalous gravity gradients. First, the complete set of autocorrelation and cross-correlation functions, as specified by the autocorrelation of undulation, is obtained. Then, a time-domain approach is used to synthesize the corresponding linear system driven by white noise that represents the anomalous gravity field behavior along a straight horizontal track. The resulting nine-state Markov model provides a shaping filter for applying covariance analysis and optimal estimation techniques to inertial navigation and gravity gradiometry systems analysis. ONTINUED mechanical refinement has brought inertial navigation technology to the stage where unmodeled variations in the Earth's gravity field have become significant and sometimes dominant error sources.1 At the same time, inertial navigation system analysts have been facing the task of developing error models to evaluate the impact of gravity uncertainties on navigation system performance. Ad- ditionally, the development of gravity gradiometers has made it necessary to model gravity-gradient signals to specify sensor requirements and analyze gradiometer-aided inertial navigation or gradiometry survey performance. To apply the powerful tools of linear system theory and Kalman filtering, it has become common to consider gravity uncertainties random variables, and to model them as Markov processes. Following the first treatment of vertical deflections as a first-order Markov process,2 a succession of higher order Markov models was proposed culminating with the third- order Markov undulation model3'4 which is fully self- consistent; that is, the auto- and cross-correlation functions of undulation, vertical deflections, and gravity anomaly obey and are derived according to the mathematical relationships of physical geodesy. The motivation for using Markov models for gravity un- certainties is that they can be cast in the form of linear dif- ferential equations driven by white noise. This allows use of the model as a shaping filter in covariance analysis of linear dynamic systems, Kalman filters, and optimal smoothing algorithms. The expanded third-order Markov undulation model described in this paper is derived from the original version, but it has three differences. First, it was found that the original model could easily be extended to include all the anomalous gravity gradients by continued differentiation of auto- and cross-correlation functions. Second, the exact cross-correlation function between undulation and gravity anomaly contains modified Bessel functions.4 Modified Bessel functions are solutions to a class of differential equations whose coefficients are functions of the independent variable. To obtain a Markov shaping filter, it is necessary to substitute an approximate function which is the solution to a linear differential equation with constant coefficients. While the chosen exponential form did not approximate the exact Bessel function as well as at first hoped possible, it provides a suitable representation of the un- dulation-gravity anomaly cross-correlation.

Bayesian dynamic probit models for the analysis of longitudinal data

In this paper, we develop a new “robust mixing” framework for reasoning about adversarially modified Markov Chains (AMMC). Let ℙ be the transition matrix of an irreducible Markov Chain with stationary distribution π. An adversary announces a sequence of stochastic matrices $\{{\mathbb{A}}_t\}_{t > 0}$ satisfying $\pi{\mathbb{A}}_t = \pi$. An AMMC process involves an application of ℙ followed by ${\mathbb{A}}_t$ at time t. The robust mixing time of an irreducible Markov Chain ℙ is the supremum over all adversarial strategies of the mixing time of the corresponding AMMC process. Applications include estimating the mixing times for certain non-Markovian processes and for reversible liftings of Markov Chains. Non-Markovian card shuffling processes: The random-to-cyclic transposition process is a non-Markovian card shuffling process, which at time t, exchanges the card at position $t {\pmod n}$ with a random card. Mossel, Peres and Sinclair (2004) showed that the mixing time of this process lies between (0.0345+o(1))nlogn and Cnlogn + O(n) (with C ≈4 ×105). We reduce the constant C to 1 by showing that the random-to-top transposition chain (a Markov Chain) has robust mixing time ≤nlogn + O(n) when the adversarial strategies are limited to those which preserve the symmetry of the underlying Markov Chain. Reversible liftings: Chen, Lovasz and Pak showed that for a reversible ergodic Markov Chain ℙ, any reversible lifting ℚ of ℙ must satisfy ${\mathcal{T}}({\mathbb{P}}) \leq {\mathcal{T}}(\mathbb{Q})\log (1/\pi_*)$ where π* is the minimum stationary probability. Looking at a specific adversarial strategy allows us to show that ${\mathcal{T}}(\mathbb{Q}) \geq r({\mathbb{P}})$ where r(ℙ) is the relaxation time of ℙ. This helps identify cases where reversible liftings cannot improve the mixing time by more than a constant factor.

In the nuclear engineering community, the error propagation of the Monte Carlo fission source distribution through cycles is known to be a linear Markov process when the number of histories per cycle is sufficiently large. In the statistics community, linear Markov processes with linear observation functions are known to have an autoregressive moving average (ARMA) representation of orders p and p — 1. Therefore, one can perform ARMA fitting of the binned Monte Carlo fission source in order to compute physical and statistical quantities relevant to nuclear criticality analysis. In this work, the ARMA fitting of a binary Monte Carlo fission source has been successfully developed as a method to compute the dominance ratio, i.e., the ratio of the second-largest to the largest eigenvalues. The method is free of binning mesh refinement and does not require the alteration of the basic source iteration cycle algorithm. Numerical results are presented for problems with one-group isotropic, two-group linearly anisotropic, and continuous-energy cross sections. Also, a strategy for the analysis of eigenmodes higher than the second-largest eigenvalue is demonstrated numerically.

In the present note we modify a linear positive Markov process of discrete type by using so called multiplicative calculus. In this framework, a convergence property and the error of approximation are established.

Bernoulli and first-order Markov processes are used to approximate the output process of a class of slotted multiuser random-access communication networks. The parameters of the approximating processes are analytically calculated for a network operating under a specific random-access algorithm. The mean time that a packet spends in the central node of the star topology is calculated under the proposed approximations of the output processes of the interconnected networks. The results are compared with simulation results of the actual system. It turns out that the memoryless approximation gives satisfactory results up to a certain per-network-traffic load. Beyond that point, the first-order Markov processes performs better. >

The problem of sampling from the stationary distribution of a Markov chain finds widespread applications in a variety of fields. The time required for a Markov chain to converge to its stationary distribution is known as the classical mixing time. In this article, we deal with analog quantum algorithms for mixing. First, we provide an analog quantum algorithm that, given a Markov chain, allows us to sample from its stationary distribution in a time that scales as the sum of the square root of the classical mixing time and the square root of the classical hitting time. Our algorithm makes use of the framework of interpolated quantum walks and relies on Hamiltonian evolution in conjunction with von Neumann measurements. There also exists a different notion for quantum mixing: the problem of sampling from the limiting distribution of quantum walks, defined in a time-averaged sense. In this scenario, the quantum mixing time is defined as the time required to sample from a distribution that is close to this limiting distribution. Recently, we provided an upper bound on the quantum mixing time for Erd\ifmmode \mbox{\H{o}}\else \H{o}\fi{}s-R\'enyi random graphs [Phys. Rev. Lett. 124, 050501 (2020)]. Here, we also extend and expand upon our findings therein. Namely, we provide an intuitive understanding of the state-of-the-art random matrix theory tools used to derive our results. In particular, for our analysis we require information about macroscopic, mesoscopic, and microscopic statistics of eigenvalues of random matrices which we highlight here. Furthermore, we provide numerical simulations that corroborate our analytical findings and extend this notion of mixing from simple graphs to any ergodic, reversible, Markov chain.

Sequences and time-series often arise in robot tasks, e.g., in activity recognition and imitation learning. In recent years, deep neural networks (DNNs) have emerged as an effective data-driven methodology for processing sequences given sufficient training data and compute resources. However, when data is limited, simpler models such as logic/rule-based methods work surprisingly well, especially when relevant prior knowledge is applied in their construction. However, unlike DNNs, these "structured" models can be difficult to extend, and do not work well with raw unstructured data. In this work, we seek to learn flexible DNNs, yet leverage prior temporal knowledge when available. Our approach is to embed symbolic knowledge expressed as linear temporal logic (LTL) and use these embeddings to guide the training of deep models. Specifically, we construct semantic-based embeddings of automata generated from LTL formula via a Graph Neural Network. Experiments show that these learnt embeddings can lead to improvements on downstream robot tasks such as sequential action recognition and imitation learning.

Coupling and mixing times in a Markov chain

SummaryThis paper examines the long-run equilibrium wealth distribution in the context of a general equilibrium model in which heterogeneous agents are born into dynasties. When agents are born, they are randomly assigned to be either altruists or egoists. An altruistic agent cares about his progeny and so consumes part of his income (wages plus bequests) and bequeaths the rest to his heirs. These bequests determine the capital stock next period. An egoistic agent consumes all his income (wages plus bequests) leaving nothing to his heirs. Assuming that the probability that an altruist (egoist) is born to an altruist or an egoist follows a first-order Markov process, and given initial conditions on the distribution of wealth and bequests, the equilibrium dynamics of the economy can be characterised, and a stationary equilibrium (numerically) solved. This done, two fiscal policy tax/intragenerational transfer regimes are evaluated in terms of individual welfare and the distribution of wealth.

Recent empirical papers have analyzed collusion in the Joint Executive Committee in an attempt to determine which of several theories of cartel behavior is supported by the behavior of this 19th century railroad cartel. Non-parametric tests of whether high and low profit regimes followed a first-order Markov process when one controls for the number of firms support the theory of optimal collusion given by Abreu, Pearce, and Stachetti. Results on whether transition probabilities depend on the number of firms are inconclusive.

An analytical solution is presented for a discrete-time, steady-state Kalman filter with correlated measurement noise. The measurement model is a first-order Markov process, characterized by variance and correlation parameters. The analytical results are used to study the effect of a correlation on steady-state tracking accuracies. >

Recent work on the performance characterization of communication systems operating over correlated fading channels has largely concluded that modeling a mobile radio channel as a first-order Markov process is sufficient. In this paper, a new performance criterion that is directly related to the error probability of a communication system is introduced. This criterion suggests that employing a Markov model of higher order will greatly improve the accuracy of model-based performance characterizations. Next, two design techniques for higher-order Markov models are considered. Numerical results suggest that, with no increase in complexity, the use of a second-order Markov model significantly improves the accuracy of characterizing the performance of systems operating over fading channels.