Evaluating Temporal Correlations in Time Series Using Permutation Entropy, Ordinal Probabilities and Machine Learning

In this paper, we introduce a deep spiking delayed feedback reservoir (DFR) model to combine DFR with spiking neuros: DFRs are a new type of recurrent neural networks (RNNs) that are able to capture the temporal correlations in time series while spiking neurons are energy-efficient and biologically plausible neurons models. The introduced deep spiking DFR model is energy-efficient and has the capability of analyzing time series signals. The corresponding field programmable gate arrays (FPGA)-based hardware implementation of such deep spiking DFR model is introduced and the underlying energy-efficiency and recourse utilization are evaluated. Various spike encoding schemes are explored and the optimal spike encoding scheme to analyze the time series has been identified. To be specific, we evaluate the performance of the introduced model using the spectrum occupancy time series data in MIMO-OFDM based cognitive radio (CR) in dynamic spectrum sharing (DSS) networks. In a MIMO-OFDM DSS system, available spectrum is very scarce and efficient utilization of spectrum is very essential. To improve the spectrum efficiency, the first step is to identify the frequency bands that are not utilized by the existing users so that a secondary user (SU) can use them for transmission. Due to the channel correlation as well as users' activities, there is a significant temporal correlation in the spectrum occupancy behavior of the frequency bands in different time slots. The introduced deep spiking DFR model is used to capture the temporal correlation of the spectrum occupancy time series and predict the idle/busy subcarriers in future time slots for potential spectrum access. Evaluation results suggest that our introduced model achieves higher area under curve (AUC) in the receiver operating characteristic (ROC) curve compared with the traditional energy detection-based strategies and the learning-based support vector machines (SVMs).

Colored noise in GRACE total water storage time series: Its impact on trend significance in the Türkiye region and major world river basins

In 2002, in a seminal article, Bandt and Pompe proposed a new methodology for the analysis of complex time series, now known as Ordinal Analysis. The ordinal methodology is based on the computation of symbols (known as ordinal patters) which are defined in terms of the temporal ordering of data points in a time series, and whose probabilities are known as ordinal probabilities. With the ordinal probabilities the Shannon entropy can be calculated, which is the permutation entropy. Since it was proposed, the ordinal method has found applications in fields as diverse as biomedicine and climatology. However, some properties of ordinal probabilities are still not fully understood, and how to combine the ordinal approach of feature extraction with machine learning techniques for model identification, time series classification or forecasting, remains a challenge. The objective of this perspective article is to present some recent advances and to discuss some open problems.

The statistical properties of the autocorrelation function from a time series composed of independently and identically distributed stochastic variables has been studied. Analytical expressions for the autocorrelation function's variance have been derived. It has been found that two common ways of calculating the autocorrelation, moving-average and Fourier transform, exhibit different uncertainty characteristics. For periodic time series, the Fourier transform method is preferred because it gives smaller uncertainties that are uniform through all time lags. Based on these analytical results, a statistically robust method has been proposed to test the existence of correlations in a time series. The statistical test is verified by computer simulations and an application to single-molecule fluorescence spectroscopy is discussed.

Comparing different approaches to compute Permutation Entropy with coarse time series

Shannon Entropy is the preeminent tool for measuring the level of uncertainty (and conversely, information content) in a random variable. In the field of communications, entropy can be used to express the information content of given signals (represented as time series) by considering random variables which sample from specified subsequences. In this paper, we will discuss how an entropy variant, the \textit{permutation entropy} can be used to study and classify radio frequency signals in a noisy environment. The permutation entropy is the entropy of the random variable which samples occurrences of permutation patterns from time series given a fixed window length, making it a function of the distribution of permutation patterns. Since the permutation entropy is a function of the relative order of data, it is (global) amplitude agnostic and thus allows for comparison between signals at different scales. This article is intended to describe a permutation patterns approach to a data driven problem in radio frequency communications research, and includes a primer on all non-permutation pattern specific background. An empirical analysis of the methods herein on radio frequency data is included. No prior knowledge of signals analysis is assumed, and permutation pattern specific notation will be included. This article serves as a self-contained introduction to the relationship between permutation patterns, entropy, and signals analysis for studying radio frequency signals and includes results on a classification task.

Time series analysis of literary creativity: A potential paradigm

There is a growing interest in methods for detecting and interpreting changes in experimental time-evolution data. Based on measured time series, the quantitative characterization of dynamical phase transitions at bifurcation points of the underlying chaotic systems is a notoriously difficult task. Building on prior theoretical studies that focus on the discontinuities at <a:math xmlns:a="http://www.w3.org/1998/Math/MathML"><a:mrow><a:mi>q</a:mi><a:mo>=</a:mo><a:mn>1</a:mn></a:mrow></a:math> in the order-<b:math xmlns:b="http://www.w3.org/1998/Math/MathML"><b:mi>q</b:mi></b:math> dynamical Rényi entropy, we measure the derivative of its spectrum given by the variance of the logarithmic probability of trajectories. To overcome the difficulties inherent in trajectory-space descriptions of finite-length time series, we derive—within the general framework of Markov processes—a computationally efficient closed-form expression for this measure, while maintaining control over the trade-off between statistical and truncation errors. We investigate the properties of the measure and assess the efficacy of the associated method in estimating characteristic quantities of well-known dynamical systems. Furthermore, we examine its scope and limitations, including its potential utility in predicting dynamical phase transitions from time series and in characterizing the complexity of the underlying behavior. Grounded in the thermodynamic formalism of chaotic systems, the developed framework bridges dynamical systems theory, network theory, and time-series analysis, extending classical concepts such as bit-number variance or varentropy in phase-space statistics, permutation entropy in time-series analysis, and path-length statistics in network science.

The present thesis explores some approaches to classify time series without prior statistical information using the concept of permutation entropy. Motivated by the results from a previous published and relevant work that set similarity relationships between EEG time series, a reproduction of the proposed approach was performed giving negative results. The failure to reproduce those results led to the conclusion that the approach of building statistics from permutation patterns have to be complemented with another metric in order to be used for classification purposes. The concept of Total Variation Distance (TVD) was then used to develop three algorithms to classify time series in a non-parametric way. At first, the developed algorithms were tested using EEG time series. Even though the results using the developed algorithms were better than previous results, they were not as satisfactory as desired. However, the inherent complexity of brain measurements led to switch to self-generated data to test the algorithms. Using time series coming from different sets of filtered versions of Gaussian white noise the classification was performed. For comparison purposes a parametric classification approach using the Maximum Likelihood Estimation was also used. Results showed that when each set of data came from the same filtering equation the classification using the developed algorithms was optimal reaching 100% success rate in many cases, being as good as the ML approach. On the other hand, when each set of data came from a mixture of different filter equations that generate the time series (reflecting the complex situations we faced when processing EEG data) , results were fairly successful with variations with respect to the ML approach, which was outperformed in some cases but also not surpassed in others. The results obtained pointed the permutation entropy analysis to be an approach in the right direction to efficiently classify time series, however more research needs to be done to adjust the correct metric to get better results.

The article will present the methodology for holistic time series analysis, based on time series transformation model REFII (REFII is an acronym for Raise-Equal-Fall and the model version is II or 2) Patel et al. (Mining motifs in massive time series databases, 2002) [1], Perng and Parker (SQL/LPP: a time series extension of SQL based on limited patience patterns, 1999) [2], Popivanov and Miller (Similarity search over time series data using wavelets, 2002) [3]. The main purpose of REFII model is to automate time series analysis through a unique transformation model of time series. The advantage of this approach to a time series analysis is the linkage of different methods for time series analysis linking traditional data mining tools in time series, and constructing new algorithms for analyzing time series. REFII model is not a closed system, which means that there is a finite set of methods. This is primarily a model used for transformation of values of time series, which prepares data used by different sets of methods based on the same model of transformation in the domain of problem space. REFII model gives a new approach in time series analysis based on a unique model of transformation, which is a base for all kind of time series analyses. In combination with elements of other methods, such as self-organizing maps or frequent-pattern trees, REFII models can make new hybrid methods for efficient time temporal data mining. Similar principle of hybridization could be used as a tool for time series temporal pattern recognition. The article describes real case study illustrating practical application of described methodology.

BackgroundParameters derived from an acceleration signal, such as the time accumulated in sedentary behaviour or moderate to vigorous physical activity (MVPA), may not be sufficient to describe physical activity (PA) which is a complex behaviour. Incorporating more advanced wearable-specific indicators of PA behaviour (WIPAB) may be useful when characterising PA profiles and investigating associations with health. We investigated the associations of novel objective measures of PA behaviour with glycated haemoglobin (HbA1c) and insulin sensitivity (Quicki index).MethodsThis observational study included 1026 adults (55% women) aged 18-79y who were recruited from the general population in Luxembourg. Participants provided ≥ 4 valid days of triaxial accelerometry data which was used to derive WIPAB variables related to the activity intensity, accumulation pattern and the temporal correlation and regularity of the acceleration time series.ResultsAdjusted general linear models showed that more time spent in MVPA and a higher average acceleration were both associated with a higher insulin sensitivity. More time accumulated in sedentary behaviour was associated with lower insulin sensitivity. With regard to WIPAB variables, parameters that were indicative of higher PA intensity, including a shallower intensity gradient and higher average accelerations registered during the most active 8 h and 15 min of the day, were associated with higher insulin sensitivity. Results for the power law exponent alpha, and the proportion of daily time accumulated in sedentary bouts > 60 min, indicated that activity which was characterised by long sedentary bouts was associated with lower insulin sensitivity. A greater proportion of time spent in MVPA bouts > 10 min was associated with higher insulin sensitivity. A higher scaling exponent alpha at small time scales (< 90 min), which shows greater correlation in the acceleration time series over short durations, was associated with higher insulin sensitivity. When measured over the entirety of the time series, metrics that reflected a more complex, irregular and unpredictable activity profile, such as the sample entropy, were associated with lower HbA1c levels and higher insulin sensitivity.ConclusionOur investigation of novel WIPAB variables shows that parameters related to activity intensity, accumulation pattern, temporal correlation and regularity are associated with insulin sensitivity in an adult general population.

Since the discovery of slow slip events, many methods have been successfully applied to model obvious transient events in geodetic time series, such as the widely used network strain filter. Independent seismological observations of tremors or low‐frequency earthquakes and repeating earthquakes provide evidence of low‐amplitude slow deformation but do not always coincide with clear occurrences of transient signals in geodetic time series. Here we aim to extract the signal corresponding to slow slips hidden in the noise of GPS time series, without using information from independent data sets. We first build a library of synthetic slow slip event templates by assembling a source function with Green's functions for a discretized fault. We then correlate the templates with postprocessed GPS time series. Once the events have been detected in time, we estimate their duration T and magnitude Mw by modeling a weighted stack of GPS time series. An analysis of synthetic time series shows that this method is able to resolve the correct timing, location, T, and Mw of events larger than Mw 6 in the context of the Mexico subduction zone. Applied on a real data set of 29 GPS time series in the Guerrero area from 2005 to 2014, this technique allows us to detect 28 transient events from Mw 6.3 to 7.2 with durations that range from 3 to 39 days. These events have a dominant recurrence time of 40 days and are mainly located at the downdip edges of the Mw&gt;7.5 slow slip events.

Based on the mathematical arguments formulated within the multifractal detrended fluctuation analysis (MFDFA) approach it is shown that, in the uncorrelated time series from the Gaussian basin of attraction, the effects resembling multifractality asymptotically disappear for positive moments when the length of time series increases. A hint is given that this applies to the negative moments as well and extends to the Lévy stable regime of fluctuations. The related effects are also illustrated and confirmed by numerical simulations. This documents that the genuine multifractality in time series may only result from the long-range temporal correlations, and the fatter distribution tails of fluctuations may broaden the width of the singularity spectrum only when such correlations are present. The frequently asked question of what makes multifractality in time series-temporal correlations or broad distribution tails-is thus ill posed. In the absence of correlations only the bifractal or monofractal cases are possible. The former corresponds to the Lévy stable regime of fluctuations while the latter to the ones belonging to the Gaussian basin of attraction in the sense of the central limit theorem.

The Indian commodity market is characterized by high volatility. When considering the agro-based commodity market, the prices may sometimes vary on a daily basis and regional basis. For the purpose of our research, we have restricted our region of study to the Indian national capital New Delhi. This paper aims to find out whether commodity markets follow a pattern with respect to prices, and if they do, then whether this could be determined by using basic fractal theory and determination of Hurst exponent. We have followed a suitable algorithm to find the Hurst exponent using statistical methods, specifically linear regression and time series analysis, wherein time is the independent variable and price of the commodity considered is dependent. The reason why time series analysis is chosen is because of the tendency of a time series to regress strongly to its mean. A statistical measure chosen to classify time series is the Hurst exponent. Initially, we have focused on onion prices for the years 2013 to 2017. The data set has been derived from the official website of the Consumer Affairs Department of the Government of India. The daily retail prices for Delhi for the month of June were observed and analyzed. We eventually aim to investigate if the market for onions has a long-term memory and will it be suitable to extend this conclusion to all other agro-based commodities. Our study has been motivated by the Fractal Market Hypothesis (FMH) that analyses the daily randomness of the market. We seek to find out whether the commodity market follows such a pattern provided that external factors remain constant. By external factors, we mean the variations that occur in the market with time, which include the demand, inflation, global price change, changes in the economy, etc. Keeping this in mind, we have attempted a time series analysis, using the monofractal analysis, at the end of which we would be estimating the Hurst exponent. The determination of Hurst exponent will help us to classify the time series as persistent or anti-persistent, i.e., how strong is the tendency of the time series to revert to its long-term mean value. Further, the multifractal analysis has been used to detect small as well as large fluctuations within the time series taken into consideration. This result would thus lead us to understand if prices in the commodity market could be remotely predicted, and what is the strength of the time series to return to its long-term mean value. Hence, this fractal analysis can be used to determine the characteristics of the prices in an agro-based economy.

Data were collected from 281 Taiwan continuous Global Positioning System (cGPS) Array sites from 2007-2013 and processed with GAMIT/GLOBK software. Power spectral density stacking from cGPS position time series in Taiwan found the spectral index as -0.72, -0.77, and -0.57 for the E, N, U components, respectively. This indicates the cGPS data errors can be described as a combination of white noise and flicker noise. The common-mode errors are removed by stacking data from 50 cGPS sites with data periods greater than 5 years. By removing the common-mode errors the GPS data precision is further improved to 2.3, 1.9, and 6.9 mm in the E, N, U components, respectively. After strict data quality control, time series analysis and noise analysis, we derive a new Taiwan velocity field using cGPS data from 2007-2013. The general pattern of the newly derived 2007-2013 velocity field is quite similar to that from previous studies, but the station density is much larger and spatial coverage better. About 80 mm yr^(-1) plate convergence rate is observed, half of the rate is accommodated on the fold and thrust belt of western Taiwan and another half is taken up in the Longitudinal Valley and Coastal Range in eastern Taiwan. The velocities in western Taiwan generally show a fan-shaped pattern, consistent with the maximum compression tectonic stress direction. In northern Taiwan the velocity vectors reveal clockwise rotation, indicating the on-going extensional deformation related to the back-arc extension of the Okinawa Trough. In southern Taiwan, the horizontal velocity increases from about 40 mm yr^(-1) in the Chia-Nan area to 55 mm yr^(-1) in the Kao-Ping area with a counterclockwise rotation.