NODSTAC: Novel Outlier Detection Technique Based on Spatial, Temporal and Attribute Correlations on IoT Bigdata

Spatiotemporal analysis has been recognized as one of the most promising techniques to improve the accuracy of photovoltaic (PV) generation forecasts. In recent years, PV generation data of a number of PV systems distributed in a geographical locale have become increasingly available. This paper conducts a thorough investigation of the spatial-temporal correlation amongst PV generation data of distributed PV systems. PV generation data of different PV systems located at different sites may exhibit similar time varying patterns. To quantify such spatial correlation, a suitable spatial similarity metric is chosen and its applicability is examined. To evaluate the temporal correlations amongst the PV generation data collected from distributed PV systems, a shape-based distance metric is proposed. A data-driven inference model, built on a Bayesian network, is developed for a very short-term PV generation forecast (less than 30 min). The model utilizes historic PV generation and weather data, and incorporates the abovementioned spatial similarity and temporal correlation to support the PV output forecast. The experiment results show that the proposed method achieves a promising performance compared to a number of baseline methods.

Stochastic methods have been typically used for the design and operations of hydraulic infrastructure. They allow decision makers to evaluate existing or new infrastructure under different possible scenarios, giving them the flexibility and tools needed in decision making. In this paper, we present a novel stochastic streamflow simulation approach able to replicate both temporal and spatial dependencies from the original data in a multi‐site basin context. The proposed model is a multi‐site extension of the modified Fractional Gaussian Noise (mFGN) model which is well‐known to be efficient to maintain periodic correlation for several time lags, but presents shortcomings in preserving the spatial correlation. Our method, called Weighted‐mFGN (WmFGN), incorporates spatial dependency into streamflows simulated with mFGN by relying on the Cholesky decomposition of the spatial correlation matrix of the historical streamflow records. As the order in which the decomposition steps are performed (temporal then spatial, or vice‐versa) affects the performance in terms of preserving the temporal and spatial correlation, our method searches for an optimal convex combination of the resulting correlation matrices. The result is a Pareto‐curve that indicates the optimal weights of the convex combination depending on the importance given by the user to spatial and temporal correlations. The model is applied to a number of river basins in Chile, where the results show that the WmFGN approach maintains the qualities of the single‐site mFGN, while significantly improving spatial correlation.

Spatial-temporal imputation remains a challenging problem in transportation, environment and healthcare, where the missing value is filled based on spatial, temporal, and cross correlations. Previous research mainly focused on feature-level correlation integration and comprehension with the hand-crafted enhancement strategy. Meanwhile, the recently prevalent large language models (LLMs) provide token-level understanding for language linguistics, and whether they could be applied for spatial-temporal correlation enhancement is under exploration. To this end, we proposed an LLM-native framework STOMA to fully utilize the intrinsic relevance. We designed semantic enhancing methods by converting the complex correlations, e.g. spatial correlation in network, temporal correlation with periodicity and cross correlation from human behavior, into the embedded tokens. Specifically, we reform dynamic time warping as an asymmetric correlation constructor for complex dynamics. We adapt the proposed backbone along with the spatial-temporal fine-tuning technique, and the empirical results demonstrate the effectiveness of our methods over recent LLM-inspired methods evaluating on real-world datasets.

Data collection plays an important role in wireless sensor networks. Recovery of spatio-temporal data from incomplete sensing data is vital to the network lifetime. Many works have utilized the spatial and temporal correlations to achieve satisfactory data recovery results. However, these methods introduce large computational overhead at the fusion center. In this paper, we develop an ADMM-Net framework for correlated spatio-temporal data recovery. Both the spatial correlation and temporal correlation of sensing data are considered in a convex optimization problem, which is solved by the alternating direction method of multipliers (ADMM) algorithm. We then unfold the ADMM algorithm into a fixed-length neural network that reduces the iterations dramatically and does not require additional location information of nodes. Experimental results on a realworld dataset demonstrate that the proposed method can achieve faster convergence speed than the baseline ADMM algorithm with slight accuracy loss.

A Technique of Filtering Dirty Data Based on Temporal- Spatial Correlation in Wireless Sensor Network

Data in the Internet of Things (IoT) domain may be missing due to connectivity errors, environmental extremes, sensor malfunctions, and human errors. Despite the many approaches for imputing missing values, the most significant difficulty in terms of imputation precision or compute complexity for larger missing sub-sequences in uni-variate series is still being explored. This work introduced IMD-MP (Imputation of Missing Data using Matrix Profile), a new technique that improves imputation accuracy for big data analysis in IoT applications based on spatial-temporal correlations using a novel distance metric Matrix Profile Distance (MPD). Our method preserves spatial correlation by grouping the sensors present in the network (using grouping algorithm-GA) to impute the missing data of the failed sensor node. After grouping, similar sensor nodes to the failed sensor node are identified using the Node Similarity Algorithm (NSF). From its similar sensor data, a certain number of sub-sequences that are most similar to the one preceding the failed node’s missing values are gathered. These sub-sequences heights are optimized to ensure temporal correlation in the imputed data. To find the optimal imputation sequence, the current research uses MPD and similarity scores. Numerical findings using sensor data from real-time environmental mon-itoring and Intel data sets demonstrate the algorithm’s effectiveness compared to other benchmarks.

Chapter 3 - Uncertainty modeling

The realistic channel environment can be characterized by a geometry-based stochastic channel (GBSC) model composed of multiple clusters, each of which consists of many rays. For the GBSC model, the power angle of arrival (AoA) spectrum of a cluster depends on the power angle of departure-Doppler (AoD-Doppler) spectrum. Hence, this model is non-Kronecker structured. In over-the-air (OTA) testing, the prefaded signal synthesis (PFS) method is valid for emulating the GBSC to measure devices under test (DUTs). However, the emulated channel under the conventional PFS method is a Kronecker case due to the independently and identically distributed (i.i.d.) fading sequences between probes when synthesizing clusters. It will lead to inaccurate link characteristics for emulation. Therefore, a non-Kronecker structured PFS (NKSPFS) method is proposed. Specifically, the rays mapped to probes are individually weighted to ensure that fading sequences between probes are independent but not identically distributed. To determine the ray weights without degrading marginal correlations for emulation, the constraints are introduced on the deviation between the target spatial–temporal correlation and the emulated spatial–temporal correlation according to the emulated spatial correlation and temporal correlation. Simulation results show that the proposed NKSPFS method outperforms the conventional PFS method in OTA testing of multi-antenna devices.

This paper is conceived as an extension in the analysis of the periodic stator-rotor interaction in a low-speed axial fan with inlet guide vanes. Here, the present work focuses on the deterministic fluctuations that occur in the axial gap between the blade rows. In particular, we present experimental data on the phase averaged velocity in the stator frame of reference. Detailed measurements of the flow field were obtained using hot-wire anemometry in a traverse sector that covered the whole span of the stage for a complete stator pitch. The blade-to-blade velocity gradient in the rotor passage is observed as an unsteady flow in the stator frame of reference due to the relative motion of the blades. As a consequence, this periodic fluctuation is superimposed on the steady vane-to-vane velocity distribution, resulting in a nonuniform unsteadiness with additional phase-dependent wake-blockage interaction. This phase-dependent interaction is determined by the difference between the overall deterministic fluctuation and the rotating spatial blade-to-blade distribution, when the latter is accurately transformed to the stator reference frame. The results revealed that high unsteadiness in the tip region is mainly derived from the radial increase of the blade-to-blade nonuniformities in the rotor, whereas the wake-blockage interaction exhibits a spanwise uniform distribution. Hence, the hub-to-tip torsion of the blades is responsible for setting off a major spatial distortion between the rows. Complementarily, we observed that a reduction in the rows’ spacing or an increase in the blade loadings leads to a higher wake-blockage interaction, modifying the impingement of the incoming stator wakes. In addition, the deterministic stresses were calculated by time averaging the phase-dependent velocity correlations and compared to the turbulent stresses. The deterministic stresses were also segregated in temporal, spatial, and spatial-temporal correlations in order to analyze the dominant mechanisms involved in their generation. At this point, the deterministic kinetic energy levels were found to be generally lower than turbulent kinetic energy levels, with both temporal and spatial correlations being the most significant terms of the tensor.

This paper develops a simple, robust framework for the problem of finding the route with the least expected travel time from any node to any given destination in a stochastic and time-dependent network. Spatial and temporal link travel time correlations are both considered in the proposed solution, which is based on a dynamic programming approach. In particular, the spatial correlation is represented by a Markovian property of the link states, in which each link is assumed to experience congested or uncongested conditions. The temporal correlation is manifested through the time-dependent expected link travel time given the condition of the link traversed. The framework enables the use of a route guidance system, in which at any decision node within a network, a decision can be made on the basis of current traffic information about which node to take next to achieve the shortest expected travel time to the destination. Numerical examples are presented to illustrate the computational steps involved in the framework to make route choices and to demonstrate the effectiveness of the proposed solution.

Melt pool dynamics reflect the formulation of microstructural defects in parts during Laser-Based Additive Manufacturing (LBAM). The thermal images of the melt pool collected during the LBAM process provide unique opportunities for modeling and monitoring its evolution. The recognized anomalies are evidence of part defects that are to be eliminated for higher product quality. A unique concern in analyzing thermal images is spatial-temporal correlations – the heat transfer within the melt pool causes spatial correlations among pixels in an image, and the evolution of the melt pool causes temporal correlations across images. The objective of this study is to develop a LBAM modeling-monitoring framework that incorporates spatial-temporal effects in characterizing and monitoring melt pool behavior. Spatial-Temporal Conditional Autoregressive (STCAR) models are explored. STCAR-AR is identified as the best candidate among the numerous STCAR variants. A novel two-level control chart is constructed on top of the STCAR-AR model to monitor the melt pool dynamics. A hierarchical structure underlies the two-level control chart in the sense that global anomalies recognized in Level II can be traced in Level I for further inspection. A comparison with other recently developed in-situ monitoring approaches shows that the proposed framework achieves the best detection power and false positive rate.

Wireless sensor networks have limited power for processing data, storage, and communication. Due to power shortages and anonymous attacks, sensor nodes may produce faulty or anomaly data which affects the accuracy of the entire system. Effective anomaly detection is essential to make an accurate prediction of the result. Moreover, clustering-based anomaly detection reduces energy consumption by avoiding individual sensory data reporting to the base station. The proposed methodology consists of two phases: Correlated graph clustering, and anomaly detection using a Fuzzy model. In the first phase, the spatial correlation of the sensor readings is used to generate a graph, partitioned into clusters. The intra-cluster and inter-cluster temporal correlations are analyzed to refine the optimized cluster structure. Finally, a fuzzy Mamdani model is used to classify the clusters as either normal or anomalous based on their membership values. The proposed approach leverages both spatial and temporal correlation between sensor measurements to form optimized clusters that are more effective for anomaly detection. The Experiments performed on a real-world dataset of WSNs indicate the efficacy of the proposed methodology, which shows significant improvement over traditional anomaly detection methods the electronic file of your paper will be formatted further for final publication.

Accurate origin-destination traffic prediction (ODTP) is a persistent problem in network management. Due to the structural nature of networks, the spatial correlations are critical for effective perdition. In this paper, the ODTP problem is built onto the graph domain by mapping OD traffic into graph-structured data involving the topology information. Moreover, due to the flow characteristics of OD traffic, the complex spatial-temporal (ST) correlations should not be limited at one-hop neighbors or consecutive time steps. To benefit from this observation, we propose a novel graph learning based method, called multi-scale spatial-temporal graph neural network (MSTNN). In MSTNN, spatial and temporal extractors are designed to capture multi-scale spatial and temporal correlations. Specifically, the spatial extractor employs the graph attention mechanism to capture the time-varying spatial correlations of nodes and their multi-hop neighbors. In the temporal extractor, we design gated dilated convolution layers with different dilation factors, each of which represents a different granularity of time, to exploit the multi-scale temporal correlations in both adjacent and non-adjacent time steps. After cascading the spatial and temporal extractors, the multi-scale ST correlations are weighted fused. Simulations on two real-world datasets show that MSTNN outperforms existing approaches that work well in various prediction tasks.

Taxis are part of the important components of urban transportation, and the demand for taxis is essential to build an efficient transportation system in smart cities. Accurate taxi demand forecasts can guide vehicle scheduling, improve vehicle utilization, ease traffic congestion, and improve passenger ride experience. Aiming at the complex spatiotemporal and spatial dependence of taxi demand, how to accurately predict taxi demand is a current research hotspot. This paper proposes a new taxi usage demand forecasting model, namely CGRU model, which uses spectral domain graph convolutional networks(ChebNet) to encode the topology of taxi usage requirements to obtain topological correlations, while modelling spatial correlations with reference to usage requirements of other regions of functional similarity in that region, using gated recurrent units(GRU) model the temporal correlation, combine the spatial correlation with the temporal correlation, and complete the analysis of the temporal and spatial correlation of the taxi demand. The proposed model is assessed on the NYCTAXI_DYNA open-source dataset, and the results show that the CGRU model outperforms the baseline model on evaluation metrics such as MAE and RMSE.

Assessing the impact of uncertainties in ice-sheet models is a major and challenging issue that needs to be faced by the ice-sheet community to provide more robust and reliable model-based projections of ice-sheet mass balance. In recent years, uncertainty quantification (UQ) has been increasingly used to characterize and explore uncertainty in ice-sheet models and improve the robustness of their projections. A typical UQ analysis involves first the (probabilistic) characterization of the sources of uncertainty followed by the propagation and sensitivity analysis of these sources of uncertainty. Previous studies concerned with UQ in ice-sheet models have generally focused on the last two steps but paid relatively little attention to the preliminary and critical step of the characterization of uncertainty. Sources of uncertainty in ice-sheet models, like uncertainties in ice-sheet geometry or surface mass balance, typically vary in space and potentially in time. For that reason, they are more adequately described as spatio(-temporal) random fields, which account naturally for spatial (and temporal) correlation. As a means of improving the characterization of the sources of uncertainties in ice-sheet models, we propose in this paper to represent them as Gaussian random fields with Matérn covariance function. The class of Matérn covariance functions provides a flexible model able to capture statistical dependence between locations with different degrees of spatial correlation or smoothness properties. Samples from a Gaussian random field with Matérn covariance function can be generated efficiently by solving a certain stochastic partial differential equation. Discretization of this stochastic partial differential equation by the finite element method results in a sparse approximation known as a Gaussian Markov random field. We solve this equation efficiently using the finite element method within the Ice-sheet and Sea-level System Model (ISSM). In addition, spatio-temporal samples can be generated by combining an autoregressive temporal model and the Matérn field. The implementation is tested on a set of synthetic experiments to verify that it captures well the desired spatial and temporal correlations. Finally, we demonstrate the interest of this sampling capability in an illustration concerned with assessing the impact of various sources of uncertainties on the Pine Island Glacier, West Antarctica. We find that both larger spatial and temporal correlations lengths will likely result in increased uncertainty in the projections.