Permutation Entropy for Signal Analysis

The present thesis explores some approaches to classify time series without prior statistical information using the concept of permutation entropy. Motivated by the results from a previous published and relevant work that set similarity relationships between EEG time series, a reproduction of the proposed approach was performed giving negative results. The failure to reproduce those results led to the conclusion that the approach of building statistics from permutation patterns have to be complemented with another metric in order to be used for classification purposes. The concept of Total Variation Distance (TVD) was then used to develop three algorithms to classify time series in a non-parametric way. At first, the developed algorithms were tested using EEG time series. Even though the results using the developed algorithms were better than previous results, they were not as satisfactory as desired. However, the inherent complexity of brain measurements led to switch to self-generated data to test the algorithms. Using time series coming from different sets of filtered versions of Gaussian white noise the classification was performed. For comparison purposes a parametric classification approach using the Maximum Likelihood Estimation was also used. Results showed that when each set of data came from the same filtering equation the classification using the developed algorithms was optimal reaching 100% success rate in many cases, being as good as the ML approach. On the other hand, when each set of data came from a mixture of different filter equations that generate the time series (reflecting the complex situations we faced when processing EEG data) , results were fairly successful with variations with respect to the ML approach, which was outperformed in some cases but also not surpassed in others. The results obtained pointed the permutation entropy analysis to be an approach in the right direction to efficiently classify time series, however more research needs to be done to adjust the correct metric to get better results.

The complexity of a system recorded in time series data can be measured statistically using permutation entropy (PE). The state of a system (e.g. regular, chaotic, random, etc.) that underlies the appearance of variations in time series can be determined with PE. Since volcanoes are considered as the complex dynamical system controlled by interactions of many processes. Permutation entropy can be applied to study the system mechanism of volcano. We utilized PE to study system mechanism of Kelud volcano in 2007 dome-forming eruption period, from 3 (KWH; KLD; UMBK) seismic stations with different distances from the crater lake. Then, we want to compare the results. The result of study shows that the PE pattern for each station is different. The unique PE pattern that can be used as an eruption precursor is only shown at KWH and KLD stations. This pattern began to appear 2.7 days before the eruption on 3 November 2007. Data from UMBK station doesn’t show unique PE pattern. The factors such as sensor distance from magmatic activity center, size, and type of eruption probably influenced the final PE result. Using PE as the addition to volcano monitoring can maximize efforts in mitigation activities.

Permutation entropy and statistical analysis of the historical evolution of the Mexican stock market index

The complexity of time series can be quantitatively measured using Permutation Entropy (PE). PE has recently been introduced as a potential tool in eruption forecasting by applying it to the seismic time series. Examples of successful applications are the eruptions at Strokkur Geyser, Iceland, the 1996 eruption of Gjálp, Iceland, and the eruptions of Shinmoedake volcano, Japan, in 2011, 2017, and 2018. While PE is able to show temporal changes prior to an eruption, these features are not always prominent. To improve this method, we calculate PE not only for the amplitudes of the seismic signals but also for the seismic instantaneous phases, called Phase Permutation Entropy (PPE). To understand the difference between PE and PPE, we performed synthetic tests by creating several synthetic waveforms using different numbers of sin wave superposition. We used more wave superposition with different frequencies to create complex waves containing broader frequency spectrum, while less superposition is used to create simpler waves containing narrower frequency spectrum. PE and PPE values are both low for simple waves and high for complex waves, but their absolute values differ, which might contain valuable information. The gap, dP = PE-PPE, is found to be smaller for complex waves compared to the more simple waves.  We then calculated PE and PPE for seismic data recorded from January 2014 to December 2015, which covered the eruption period of Holuhraun in Iceland. During the time of quiescence, both PE and PPE exhibit a long period variation which seems to be seasonal. Calculating dP weakens the long period noise and generates a more stable baseline.  We observe that the temporal variation of dP started to decrease below the baseline after 24 May 2014, indicating that the ground motion got more complex. An abrupt drop of dP to its lowest level was observed on 16 August 2014, when the dyke started to propagate from Bardarbunga to Holuhraun. While dP increases after the onset of eruption on 29 and 31August 2014, there is no prominent feature between the dyke propagation and the onset of the eruptions.  During the eruption period, dP stays lower than the background dP, indicating a higher ground motion complexity compared to the quiescence time. After January 2015, the gradual increase of dP back to the baseline level is clearly observed, showing the method’s potential to foresee the end of the eruption.

Computational characteristics of interictal EEG as objective markers of epileptic spasms

Reliable detection of incipient faults in rotating machinery under noisy environments and varying speeds is essential for ensuring operational safety and noise reduction. This study explores the potential of ordinal pattern (OP)-based methods for acoustic and vibration signal analysis in fault diagnosis, leveraging their ability to capture underlying modulation structures often masked by background noise. Two types of OP-based methods are investigated: 1. Statistical approaches, including Permutation Entropy (PE) and its amplitude-sensitive extensions-Dispersion Entropy, Bubble Entropy, Amplitude-Aware PE, and improved PE. 2. Time-domain decomposition methods, specifically Ordinal Pattern Mode Decomposition (OPMD), utilize OP-based filters to extract dominant oscillatory and narrowband components iteratively. Theoretical analysis confirms that OPMD outperforms Empirical Mode Decomposition (EMD) in preserving modulation patterns. The study was conducted in a noisy industrial environment. Acoustic signals were recorded, while vibration signals were simultaneously acquired for comparative analysis. The results on five bearing conditions (healthy and faulty) demonstrate that both PE-based approaches and OPMD applied to acoustic signals effectively detect faults. However, high noise levels can distort ordinal pattern probability distributions, potentially impacting detection performance. Despite this, the proposed methods outperform classical signal processing techniques and offer valuable insights for noise-aware diagnostics in industrial settings.

Permutation entropy and statistical complexity are measures for complex time series. The Bandt–Pompe methodology evaluates probability distribution using permutation. The method is robust and effective to quantify information of time series data. Statistical complexity is the product of Jensen–Shannon divergence and permutation entropy. These physical parameters are introduced to analyse time series of emission and magnetic fluctuations in low-aspect-ratio reversed-field pinch (RFP) plasma. The observed time-series data aggregates in a region of the plane, the so-called C–H plane, determined by entropy versus complexity. The C–H plane is a representation space used for distinguishing periodic, chaos, stochastic and noisy processes of time series data. The characteristics of the emissions and magnetic fluctuation change under different RFP-plasma conditions. The statistical complexities of soft x-ray emissions and magnetic fluctuations depend on the relationships between reversal and pinch parameters.

Heart rate variability (HRV) contains important information about the modulation of the cardiovascular system. Various methods of nonlinear dynamics (e.g., estimating Lyapunov exponents) and complexity measures (e.g., correlation dimension or entropies) have been applied to HRV analysis. Permutation entropy, which was proposed recently, has been widely used in many fields due to its conceptual and computational simplicity. It maps a time series onto a symbolic sequence of permutation ranks. The original permutation entropy assumes the time series under study has a continuous distribution, thus equal values are rare and can be ignored by ranking them according to their order of emergence, or broken by adding small random perturbations to ensure every symbol in a sequence is different. However, when the observed time series is digitized with lower resolution leading to a greater number of equal values, or the equalities represent certain characteristic sequential patterns of the system, it may not be rational to simply ignore or break them. In the present paper, a modified permutation entropy is proposed that, by mapping the equal value onto the same symbol (rank), allows for a more accurate characterization of system states. The application of the modified permutation entropy to the analysis of HRV is investigated using clinically collected data. Results show that modified permutation entropy can greatly improve the ability to distinguish the HRV signals under different physiological and pathological conditions. It can characterize the complexity of HRV more effectively than the original permutation entropy.

Overcoming the visual barrier and developing "see-through vision" has been one of mankind's long-standing dreams. Unlike visible light, Radio Frequency (RF) signals penetrate opaque obstructions and reflect highly off humans. This paper establishes a deep-learning model that can be trained to reconstruct continuous video of a 15-point human skeleton even through visual occlusion. The training process adopts a student/teacher learning procedure inspired by the Feynman learning technique, in which video frames and RF data are first collected simultaneously using a co-located setup containing an optical camera and an RF antenna array transceiver. Next, the video frames are processed with a computer-vision-based gait analysis "teacher" module to generate ground-truth human skeletons for each frame. Then, the same type of skeleton is predicted from corresponding RF data using a "student" deep-learning model consisting of a Residual Convolutional Neural Network (CNN), Region Proposal Network (RPN), and Recurrent Neural Network with Long-Short Term Memory (LSTM) that 1) extracts spatial features from RF images, 2) detects all people present in a scene, and 3) aggregates information over many time-steps, respectively. The model is shown to both accurately and completely predict the pose of humans behind visual obstruction solely using RF signals. Primary academic contributions include the novel many-to-many imaging methodology, unique integration of RPN and LSTM networks, and original training pipeline.

Permutation entropy and detrend fluctuation analysis for the natural complexity of cardiac heart interbeat signals

Variational mode decomposition is a new signal decomposition method, which can process non-linear and non-stationary signals. It can overcome the problems of mode mixing and compensate for the shortcomings in empirical mode decomposition. Permutation entropy is a method which can detect the randomness and kinetic mutation behavior of a time series. It can be considered for use in fault diagnosis. The complexity of wind power generation systems means that the randomness and kinetic mutation behavior of their vibration signals are displayed at different scales. Multi-scale permutation entropy analysis is therefore needed for such vibration signals. This research investigated a method based on variational mode decomposition and permutation entropy for the fault diagnosis of a wind turbine roller bearing. Variational mode decomposition was adopted to decompose the bearing vibration signal into its constituent components. The components containing key fault information were selected for the extraction of their permutation entropy. This entropy was used as a bearing fault characteristic value. The nearest neighbor algorithm was employed as a classifier to identify faults in a roller bearing. The experimental data showed that the proposed method can be applied to wind turbine roller bearing fault diagnosis.

Several approaches have been proposed to extend the permutation entropy (PE) by incorporating amplitude information, which is typically discarded in the original PE formulation. In this paper, we introduce a new PE variant that explicitly accounts for amplitude information using the well-established spectral properties of circulant determinants. This circulant determinant-based PE (CDPE) effectively captures the theoretical PE of sinusoids within the normalized frequency range <tex>$[0.01,0.2]$</tex>, outperforming existing amplitude-dependent PE (ADPE) methods. Since this range aligns with the spectral characteristics of surface electromyography (sEMG) signals <tex>$(5-500 \text{Hz})$</tex> sampled at 2048 Hz, the second key contribution of our work is investigating the sensitivity of CDPE and existing ADPE measures to excitation levels. To this end, these ADPE methods were applied to simulated physiological high-density sEMG (HDsEMG). HDsEMG represent a spatially distributed acquisition of sEMG using a dense array of electrodes, enabling a refined analysis of muscle electrical activity. Our findings provide a deeper understanding of the applicability and robustness of ADPE-based methods for HDsEMG signal analysis.

Detecting nonlinearity in short and noisy time series using the permutation entropy

Comparing different approaches to compute Permutation Entropy with coarse time series

An original multivariate multi-scale methodology for assessing the complexity of physiological signals is proposed. The technique is able to incorporate the simultaneous analysis of multi-channel data as a unique block within a multi-scale framework. The basic complexity measure is done by using Permutation Entropy, a methodology for time series processing based on ordinal analysis. Permutation Entropy is conceptually simple, structurally robust to noise and artifacts, computationally very fast, which is relevant for designing portable diagnostics. Since time series derived from biological systems show structures on multiple spatial-temporal scales, the proposed technique can be useful for other types of biomedical signal analysis. In this work, the possibility of distinguish among the brain states related to Alzheimer’s disease patients and Mild Cognitive Impaired subjects from normal healthy elderly is checked on a real, although quite limited, experimental database.