Inferring behavioural states from tracking data with hidden Markov models - a validation study using GPS video-camera collars.

Advances in biologging techniques and the availability of high‐resolution fisheries data have improved our ability to understand the interactions between seabirds and fisheries and to evaluate mortality risk due to bycatch. However, it remains unclear whether movement patterns and behaviour differ between birds foraging naturally or scavenging behind vessels and whether this could be diagnostic of fisheries interactions. We deployed novel loggers that record the GPS position of birds at sea and scan the surroundings to detect radar transmissions from vessels and immersion (activity) loggers on wandering albatrossesDiomedea exulansfrom South Georgia. We matched these data to remotely sensed fishing vessel positions and used a combination of hidden Markov and random forest models to investigate whether it was possible to detect a characteristic signature from the seabird tracking and activity data that would indicate fine‐scale vessel overlap and interactions. Including immersion data in our hidden Markov models allowed two distinct foraging behaviours to be identified, both indicative of Area Restricted Search (ARS) but with or without landing behaviour (likely prey capture attempts) that would not be detectable with location data alone. Birds approached vessels during all behavioural states, and there was no clear pattern associated with this type of scavenging behaviour. The random forest models had very low sensitivity, partly because foraging events at vessels occurred very rarely, and did not contain any diagnostic movement or activity pattern that was distinct from natural behaviours away from vessels. Thus, we were unable to predict accurately whether foraging bouts occurred in the vicinity of a fishing vessel, or naturally, based on behaviour alone. Our method provides a coherent and generalizable framework to segment trips using auxiliary biologging (immersion) data and to refine the classification of foraging strategies of seabirds. These results nevertheless underline the value of using radar detectors that detect vessel proximity or remotely sensed vessel locations for a better understanding of seabird–fishery interactions.

The most important challenge of the spectrum sensing in cognitive radio (CR) is to find a way to share the licensed spectrum without interfering with the licensed user transmission. Predicting the near future of the licensed or primary user (PU) channel state can solve this problem. Many studies have investigated the primary user channel state prediction in recent literature, in this study we introduce new approach for PU channel state prediction in time domain based on hidden Markov model (HMM) and Markov switching model (MSM). In our new approach we use a time series to capture the primary user channel state detection sequence (PU channel “idle” or “occupied”). Then, we fed this time series as an observation sequence into HMM and MSM algorithm to predict the switching time between the two states, “idle and occupied” before it happens so that the secondary user (SU) can adjust its transmission strategies accordingly. The experimental results show that all HMM and MSM perform very well for PU channel state prediction. It has also shown in a simulation comparison between HMM and MSM algorithm that MSM algorithm can preform without need for training process and provide a smoother prediction with low computational complexity than HMM.

Spectrum prediction plays a critical role in cognitive radio networks because it is promising to significantly speed up the sensing process and hence save energy as well as improve resource utilization. However, most existing spectrum prediction models are not able to fully explore the hidden correlation among adjacent observations or appropriately describe the channel behavior. In this paper, we propose a novel prediction approach termed high-order hidden bivariate Markov model (H^2BMM), by leveraging the advantages of both HBMM and high-order. H^2BMM applies two dimensional parameters, i.e., hidden process and underlying process, to more accurately describe the channel behavior. In addition, the current channel state is predicted by observing multiple previous states. Extensive simulations are conducted and results verify that the prediction accuracy is significantly improved using the proposed H^2BMM compared with traditional Hidden Markov Model (HMM) and Hidden Bivariate Markov Model (HBMM).

An approach to user modelling with discrete stochastic processes is presented, which aims at the dynamic individualization of user interfaces on the syntactic layer. The state of the art of syntactic user modelling is surveyed. The mathematical background of simple Markov Chains and the 'classic' Hidden Markov Model is presented. Furthermore, dynamic Bayesian Networks are introduced, which generalize these Markovian Models. A case study of the simulation experiments uses a multimodal user interface for supervisory control of advanced manufacturing cells. A corresponding simulation model is created and exploited to generate interaction cases, which are the empirical basis for the evaluation. Six topologies of dynamic Bayesian Networks are evaluated for 100 interaction cases and 50 replications each: (1) Markov Chain of order 1, (2) Hidden Markov Model, (3) autoregressive Hidden Markov Model, (4) factorial Hidden Markov Model, (5) simple hierarchical Hidden Markov Model, and (6) tree structured Hidden Markov Model. The dependent variable is the prediction accuracy for a single prediction lead. In a first step, a one-way analysis of variance in conjunction with Tukey's post-hoc test demonstrated a significant superiority of the simple hierarchical Hidden Markov Model. In a second step, an additional two-way analysis of variance also indicated a significantly better prediction accuracy of the simple hierarchical Hidden Markov Model compared to the Hidden Markov Model, but the number of interaction cases also had a significant effect. Hence, the modeller has to take both factors--model topology and number of interaction cases--into account when designing syntactic user models with stochastic processes.

Prediction of 24-hour-average PM2.5 concentrations using a hidden Markov model with different emission distributions in Northern California

Survey of Models A Survey of Models for Discrete-Valued Time Series Introduction: The Need for Discrete-Valued Time Series Models Markov Chains Higher-Order Markov Chains The DARMA models of Jacobs and Lewis Models Based on Thinning The Bivariate Geometric Models of Block, Langberg, and Stoffer Markov Regression Models Parameter-Driven Models State-Spaced Models Miscellaneous Models Discussion Hidden Markov Models The Basic Models Introduction Some Theoretical Aspects of Hidden Markov Models in Speech Processing Hidden Markov Time Series Models: Definition and Notation Correlation Properties Evaluation of the Likelihood Function Distributional Properties Parameter Estimation Identification of Outliers Reversibility Discussion Extensions and Modifications Introduction Models Based on a Second-Order Markov Chain Multinomial-Hidden Markov Models Multivariate Models Models with State-Dependent Probabilities Depending on Covariates Models in Which th e Markov Chain Is Homogeneous but Not Assumed Stationary Models in Which the Markov Chain Is Nonhomogeneous Joint Models for the Numbers of Trials and the Numbers of Successes in Those Trials Discussion Applications Introduction The Durations of Successive Eruptions of the Old Faithful Geyser Epileptic Seizure Counts Births at Edendale Hospital Locomotory Behaviour of Locusta migratoria Wind Direction at Koeberg Evapotranspiration Thinly Traded Shares on the Johannesburt Stock Exchange Daily Rainfall at Durban Homicides and Suicides, Cape Town, 1986-1991 Conclusion Appendices A : Proofs of Results Used in the Derivation of the Baum-Welch Algorithm B: Data

The study reported in this paper represents a continuation of our previous work on the application of Hidden Markov models (HMMs) to the translational and rotational invariant classification of SAR targets. The traditional method of making classification decisions using an HMM does not achieve the desired objective of minimizing the number of misclassifications. We present a novel technique that minimizes the probability of misclassification error. This approach, which is an adaptation of an existing Minimum Classification Error strategy, is globally optimal. The proposed method applies basic principles of pattern recognition to reduce the expected misclassification rate by dynamically perturbing the HMM parameters using a constraint on a cross-entropy measure and distance separation between pairs of HMM models. Like the traditional implementation of an HMM, our new formulation can still be implemented using an efficient forward-backward algorithm for estimating the HMM parameters. We tested our classifier on a public mixed- target MSTAR database and compared our approach to the original HMM approach trained by using a maximum likelihood criterion. The results indicate a significant improvement over the original HMM approach. Current scores from our method are about in excess of 90% on testing data sets.

This paper is concerned with filtering of hidden Markov processes (HMP) which possess (or approximately possess) the property of lumpability. This property is a generalization of the property of lumpability of a Markov chain which has been previously addressed by others. In essence, the property of lumpability means that there is a partition of the (atomic) states of the Markov chain into aggregated sets which act in a similar manner as far as the state dynamics and observation statistics are concerned. We prove necessary and sufficient conditions on the HMP for exact lumpability to hold. For a particular class of hidden Markov models (HMM), namely finite output alphabet models, conditions for lumpability of all HMP representable by a specified HMM are given. The corresponding optimal filter algorithms for the aggregated states are then derived. The paper also describes an approach to efficient suboptimal filtering for HMP which are approximately lumpable. By this we mean that the HMM generating the process may be approximated by a lumpable HMM. This approach involves directly finding a lumped HMM which approximates the original HMM well, in a matrix norm sense. An alternative approach for model reduction based on approximating a given HMM by an exactly lumpable HMM is also derived. This method is based on the alternating convex projections algorithm. Some simulation examples are presented which illustrate the performance of the suboptimal filtering algorithms.

Facies identification from basic well logs is essential for optimizing well operation or constructing reservoir model. The sequential nature of well log data makes the Hidden Markov Model (HMM) a suitable unsupervised machine learning approach for clustering. This study investigates the effectiveness of HMM combined with window filtering for unsupervised facies detection and compares its performance with the widely used K-Means Clustering (KMC) method. Synthetic gamma ray (GR), density (DEN), and neutron (NEU) logs, along with geologist-guided facies in a channel depositional environment, were used as input data. HMM identifies facies by modeling hidden geological states based on sequential well log data. Transition probabilities capture facies relationship across depths. A mode-based moving window filter smoothed the initial results by reducing fluctuations, while preserving facies boundaries. Validation was performed by comparing the results with reference facies using accuracy, adjusted rand index (ARI), normalized mutual information (NMI), and visual assessment. Both HMM and KMC clustering revealed thin non-geological facies, primarily at the facies boundaries. In the middle of facies, HMM mitigated thin facies more effectively than KMC. These thin facies arose from fluctuations in the well logs. Visually, HMM provided more geologically realistic results, despite KMC sometimes showed slightly better performance metrics. To address thin artificial facies, a moving window filter was applied. This slightly improved clustering performance metrics but significantly enhanced geological consistency by removing most artificial facies at boundaries or within facies, particularly in the smoothed HMM. Smoothed HMM achieved slightly better or comparable accuracy and NMI to smoothed KMC. However, minor thin facies persisted in both methods, more notably in KMC, due to significant well log fluctuations. The filter effectively smoothed the facies by reducing variations in thin facies across small depth intervals, ultimately enhancing the facies continuity. This improved facies distribution coherency with geological expectation, particularly for the smoothed HMM. HMM with post-processing plays a crucial role in identifying geologically consistent facies while maintaining or slightly improving clustering performance metrics. Unsupervised machine learning that incorporates sequential information is vital for facies clustering from well log data. The proposed workflow which integrates HMM with window filtering, enhances clustering outcomes by producing more geologically consistent result and more interpretable facies. This approach aids geoscientists in reservoir characterization and modeling, enabling more informed decision-making.

Adaptive ROC-based ensembles of HMMs applied to anomaly detection

Up till recently, state-of-the-art, large vocabulary, continuous speech recognition (CSR) had employed hidden Markov modeling (HMM) to model speech sounds. In an attempt to improve over HMM we developed a hybrid system that integrates HMM technology with neural networks. We present the concept of a Segmental Neural Net (SNN) for phonetic modeling in CSR. By taking into account all the frames of a phonetic segment simultaneously, the SNN overcomes the well-known conditional-independence limitation of HMMs. We have developed a novel hybrid SNN/HMM system that combines the advantages of SNNs and HMMs using a multiple hypothesis (or N-best) paradigm. In this system, we generate likely phonetic segmentations from the HMM N-best list of word sequences, which are scored by the SNN. The HMM and SNN scores are then combined to optimize performance. In several speaker-independent, 1000-word CSR tests, the error rate for the hybrid system dropped 20% from that of a state-of-the-art HMM system alone.

Hidden Markov model (HMM) is a powerful mathematical tool for prediction and recognition. Many computer software products implement HMM and hide its complexity, which assist scientists to use HMM for applied researches. However comprehending HMM in order to take advantages of its strong points requires a lot of efforts. This report is a tutorial on HMM with full of mathematical proofs and example, which help researchers to understand it by the fastest way from theory to practice. The report focuses on three common problems of HMM such as evaluation problem, uncovering problem, and learning problem, in which learning problem with support of optimization theory is the main subject.

Long‐term bio‐logging has the potential to reveal how movements, and hence life‐history trade‐offs, vary over a lifetime. Reproductive tactics in particular may vary as individuals' trade‐off current investment versus lifetime fitness. Male African savanna elephants (Loxodona africana) provide a telling example of balancing body growth with reproductive fitness due to the combination of indeterminate growth and strongly delineated periods of sexual activity (musth), which results in reproductive tactics that alter with age.Our study aims to quantify the extent to which male elephants alter their movement patterns, and hence energetic allocation, in relation to (a) reproductive state and (b) age, and (c) to determine whether musth periods can be detected directly from GPS tracking data.We used a combination of GPS tracking data and visual observations of 25 male elephants ranging in age from 20 to 52 years to examine the influence of reproductive state and age on movement. We then used a three‐state hidden Markov model (HMM) to detect musth behaviour in a subset of sequential tracking data.Our results demonstrate that male elephants increased their daily mean speed and range size with age and in musth. Furthermore, non‐musth speed decreased with age, presumably reflecting a shift towards energy acquisition during non‐musth. Thus, despite similar speeds and marginally larger ranges between reproductive states at age 20, by age 50, males were travelling 2.0 times faster in a 3.5 times larger area in musth relative to non‐musth. The distinctiveness of musth periods over age 35 meant the three‐state HMM could automatically detect musth movement with high sensitivity and specificity, but could not for the younger age class.We show that male elephants increased their energetic allocation into reproduction with age as the probability of reproductive success increases. Given that older male elephants tend to be both the target of legal trophy hunting and illegal poaching, man‐made interference could drive fundamental changes in elephant reproductive tactics. Bio‐logging, as our study reveals, has the potential both to quantify mature elephant reproductive tactics remotely and to be used to institute proactive management strategies around the reproductive behaviour of this charismatic keystone species.

This paper describes an automatic text‐ and speaker‐independent language identification method based on hidden Markov models (HMMs) for acoustic features. The hidden Markov modeling is used to represent the phonotactics for each language. Each language has its proper phonotactics.The HMM topology here is a fully structured (ergodic) model wherein any state could transit to all states. Two kinds of HMMs are used: the discrete HMM (DHMM) with the codebook and the continuous density HMM (CHMM). The HMM was trained using both the Baum‐Welch (forward‐backward) algorithm and the Viterbi algorithm. The latter was used for emphasizing the state transition probability.For comparison, experiments also were conducted on the identification using a mixtured Gaussian distribution model with one state. This single‐state Gaussian distribution model gave the same performance as the HMM trained with the Baum‐Welch algorithm. This is because the transition between states which reflects the characteristics of each language does not affect the likelihood scores very much. This problem was addressed by emphasizing the transition probabilities and using the Viterbi algorithm, which resulted in an improvement in the recognition rates. The trigram for optimal state sequence is introduced. Combining it with the HMM produced the best results.

Sleep quantity affects an individual's personal health. The gold standard of measuring sleep and diagnosing sleep disorders is Polysomnography (PSG). Although PSG is accurate, it is expensive and it lacks portability. A number of wearable devices with embedded sensors have emerged in the recent past as an alternative to PSG for regular sleep monitoring directly by the user. These devices are intrusive and cause discomfort besides being expensive. In this work, we present an algorithm to detect sleep using a smartphone with the help of its inbuilt accelerometer sensor. We present three different approaches to classify raw acceleration data into two states - Sleep and Wake. In the first approach, we take an equation from Kushida's algorithm to process accelerometer data. Henceforth, we call it Kushida's equation. While the second is based on statistical functions, the third is based on Hidden Markov Model (HMM) training. Although all the three approaches are suitable for a phone's resources, each approach demands different amount of resources. While Kushida's equation-based approach demands the least, the HMM training-based approach demands the maximum. We collected data from mobile phone's accelerometer for four subjects for twelve days each. We compare accuracy of sleep detection using each of the three approaches with that of Zeo sensor, which is based on Electroencephalogram (EEG) sensor to detect sleep. EEG is an important modality in PSG. We find that HMM training-based approach is as much as 84% accurate. It is 15% more accurate as compared to Kushida's equation-based approach and 10% more accurate as compared to statistical method-based approach. In order to concisely represent the sleep quality of people, we model their sleep data using HMM. We present an analysis to find out a tradeoff between the amount of training data and the accuracy provided in the modeling of sleep. We find that six days of sleep data is sufficient for accurate modeling. We compare accuracy of our HMM training-based algorithm with a representative third party app SleepTime available from Google Play Store for Android. We find that the detection done using HMM approach is closer to that done by Zeo by 13% as compared to the third party Android application SleepTime. We show that our HMM training-based approach is efficient as it takes less than ten seconds to get executed on Moto G Android phone.