A Method for Cleaning Power Grid Operation Data Based on Spatiotemporal Correlation Constraints

This paper presents a new approach for detection and identification of ultiple bad data --- estimation-identification approach which has been developed in order to improve the performance of static-state estimators of power systems. It describes the important function of detection and identification of bad data which is of crucial importance in designing a static-state estimator as a basic element in power system real-time control; it also discusses tthe basic principle, method of solution and programing technique of so called "Estimation-Identificaition Approach" which is based on the better utilization of the resources of residues and the analysis of the property of its sensitivity matrix; and some test results of simulation on a model system and a practical searching system of doubtful data are given. Test results of identifying bad data is good, it is usually successful especially in the case of the occurence of multi-bad data. At the end of this paper comments concerning the advantages of this approach and its recommendation as a prospective practical approach in real-time applications are given.

Bad-data detection in pre-estimation can help to improve state estimation [Teeuwsen & Erlich, 2006]. Since transient and abnormal conditions may occur in a power system, measurements may be polluted by bad data to cause estimated errors. Hence, it is necessary to accurately detect the bad data. To ensure the reliability of the measured data, a practical state estimator should have an ability to detect and identify the bad data as well as to eliminate their effects on the estimation [Zhang & Lo, 1991]. Generally, bad-date detection is important to guarantee the reliability of the measured data. If one or more errors occur in power system measurements, the states of the estimated system may be biased and the safety of power supply may be potentially dangerous. To avoid this situation, several bad-data detection and identification schemes have been presented. For example, WLS (weighted least squares) was proposed in 1989 [EI-Keib et al., 1989]. The weighted sum of squares of the measurement residuals was chosen as the objective function to be minimized. But, WLS-based state estimators were only developed by using a linearized measurement function [Huang & Lin, 2003] with complicated computations. Then, linear programming (LP) was proposed to improve the identification method [Peterson & Girgis, 1988]. However, the LP estimator may fail to reject the bad data and it can be attributed to the existence of leverage in the power system model [Ali Abur, 1990]. Thus, Ali Abur had proposed hypothesis testing identification (HTI) to extend the case of the LP estimator. Nevertheless, it had caused computational burdens with taking the special properties of the LP estimation equations into account. Huang [Huang & Lin, 2003] proposed a changeable weighting matrix to identify the bad data but it only can apply for static state estimations. Nevertheless, Zhang had proposed recursive measurement error estimation identification (RMEEI) for bad-data identification [Zhang et al., 1992]. State variables, residuals and their parameters can be updated after removing a measurement from the suspected data set to the remaining data set by using a set of linear recursive equations. With splitting the raw measurements into some parts, a set of residual equations used by the traditional methods can only apply to linear systems and it may result the operation of calculation burden and complexity because each part consists of some measurements [Zhang et al., 1992]. Source: Kalman Filter, Book edited by: Vedran Kordic, ISBN 978-953-307-094-0, pp. 390, May 2010, INTECH, Croatia, downloaded from SCIYO.COM

Phasor measurement units (PMUs) are power system devices placed at various locations in electrical power network that provide the measurement of phasors of voltages and currents from the respective meters or instruments placed at locations. Data provided by the PMUs are used for system analysis and also help to analyze the sequence of events that may have contributed to the failure of the power system. The presence of bad data in PMUs is increasing in today’s power system due to the system complexity and associated issues. This will lead to inaccurate measurement of voltage magnitude and phase angle. Hence, it is important to detect the presence of bad data and identify the PMU which contains the error. In order to identify the bad data, state estimation is carried out to obtain the true state variables by the application of Load Flow Analysis (LFA). The application of Engineering Optimization is called into this work for improving the detection of bad data and its correction. Firefly optimization and particle swarm optimization techniques are developed for the above problem, and later results are compared with respect to the accuracy of results. The system is modeled in MATLAB platform.

Based on the analysis of the improved GSA (gap statistical algorithm) data mining technology, it is applied to the identification of bad data in the power system. Aiming at the shortcomings of this method, a new data preprocessing method and clustering method are proposed. The improved GSA method is a technique to determine the most appropriate number of clusters. The original measurement data is tested by the improved RBF neural network, and the obtained results are clustered by the improved clustering method, and then the optimal number of clusters is judged by using the elbow criterion. Compared to the original algorithm, it can determine the number of clusters faster, greatly reduce the running time, and successfully identify the bad data in the power system.

The quadratic cost function J(xˆ) and the normalized residuals rN are used conventionally for identifying the presence and location of bad measurements in power system state estimation. These are "post estimation" tests and therefore require the complete re-estimation of system states whenever bad data is identified. This paper presents a pre-estimation filter for detection and identification of gross measurement errors. The basic function of this filter is to compare the measured value of a system variable with its predicted value obtained using an autoregressive (AR) model. If the difference exceeds a pre-determined threshold, the measured value is discarded in favor of the predicted value. Each measurement is processed by an AR filter before being used in the state estimation. The performance of the AR filter is tested against that of the J(xˆ) and rN tests and the results are reported in this paper. The principal advantage of the AR filtering scheme is its speed in bad data identification. Furthermore, it can be used to complement other bad data processing methods.

Due to the insufficient measurements in the distribution system state estimation (DSSE), full observability and redundant measurements are difficult to achieve without using the pseudo measurements. The matrix completion state estimation (MCSE) combines the matrix completion and power system model to estimate voltage by exploring the low-rank characteristics of the matrix. This paper proposes a robust matrix completion state estimation (RMCSE) to estimate the voltage in a distribution system under a low-observability condition. Tradition state estimation weighted least squares (WLS) method requires full observability to calculate the states and needs redundant measurements to proceed a bad data detection. The proposed method improves the robustness of the MCSE to bad data by minimizing the rank of the matrix and measurements residual with different weights. It can estimate the system state in a low-observability system and has robust estimates without the bad data detection process in the face of multiple bad data. The method is numerically evaluated on the IEEE 33-node radial distribution system. The estimation performance and robustness of RMCSE are compared with the WLS with the largest normalized residual bad data identification (WLS-LNR), and the MCSE.

ABSTRACTThis article proposes an improved square root cubature Kalman filter algorithm for dynamic state estimation in cyber‐physical wind energy systems (CPWESs). The integration of wind power generation equipment into the power grid may amplify fluctuations in system measurement data. Bad data can easily be disguised as fluctuation data, invading CPWESs and damaging their accuracy in state estimation. It is difficult to distinguish between measurement data with significant fluctuations and bad data in wind energy access networks using traditional methods. The proposed algorithm can effectively handle the identification and correction of bad data while maintaining high accuracy in CPWESs with strong data fluctuation. First, it aims to identify the mutation data of the new information sequence. By calculating the innovation covariance based on the innovation sequence and comparing the measured variance with the total variance, it constructs a dynamic high‐dimensional observation noise scaling factor to adjust the measurement noise matrix to obtain the first layer of suspicious data set. Then, based on the measurement sequence, it calculates the correlation of the data as the second identifier to distinguish between fluctuation and bad data, so that the bad data set is found and corrected. Simulation examples on the IEEE 57‐bus test system illustrate the effectiveness of the proposed method.

The quality of measurement data directly determines the accuracy of the state estimation. In practice, erroneous measurement may exist due to various physical factors, and they are known as bad data. Moreover, the measured data may also be attacked by hackers. The existence of bad data and cyber attacks could cause the state estimation unable to estimate the actual system states. Therefore, it is of great significance to detect and differentiate bad data and cyber attacks in power system state estimation. To this end, a support vector machine was proposed in this paper. It was validated using IEEE 118 bus and IEEE 30 bus systems. Numerical results prove the effectiveness of the proposed method in identifying bad data and false data injection data.

The reliability of state estimation is of vital importance for secure operation of a power system. The key features of any state estimator are its accuracy, robustness against bad data and the associated computational burden. In order to ensure full system observability and maximise accuracy, a state estimation algorithm needs to take into account all available measurements. A significant number of hybrid estimators that can process both the conventional measurements that have been used in transmission systems for decades, as well as the synchrophasor measurements, can be found in literature. However, these algorithms are predominantly non‐linear, thus requiring iterative solving procedures. In this study, a novel linear algorithm is proposed for state estimation including bad data detection of power systems that are monitored both by conventional and synchrophasor measurements. The proposed estimator is based on the linear weighted least square framework. As a result, the computational burden is rendered extremely low since there is no need for an iterative solving procedure, and standardised post‐processing tools for bad data detection can be used. To validate the accuracy and robustness of the proposed algorithm, an extensive number of test cases of different sizes are solved and the results are presented and discussed.

A Technique of Filtering Dirty Data Based on Temporal- Spatial Correlation in Wireless Sensor Network

State estimation is critical for the normal operation of power system. However, due to the aging of electric infrastructures, the power grid become more vulnerable to cyber-attack. Therefore, in this paper, scenario-based two-stage sparse cyber-attack models for smart grid with complete and incomplete network information are proposed. In previous traditional state estimator, bad data detector (BDD) is used to prevent power systems from the smearing effect of the unsteady power system caused by the bad data. However, the researches have pointed out that BDD can't recognize the abnormal state which launch by the hackers who know the topologies of the power system. Then, in order to detect false data effectively, we proposed a new detect mechanism based on extreme learning machine (ELM). In the proposed defense mechanism, we combine numbers of ELM whit certain rules, which objective is to improve the precision of the detection. Finally, the effectiveness and validation of the proposed method is verified on standard IEEE14-bus AIEEE57-bus system.

The state vector of a power system varies with time owing to the dynamic nature of system loads. Therefore, it is necessary to establish a dynamic model for the time evolution of the state vector. The dynamic state estimation approach consists of predicting the state vector based on past estimations, followed by a filtering process performed when a new set of measurements is available. This paper presents a new algorithm for forecasting and filtering the state vector, using exponential smoothing and least-squares estimation techniques. The proposed algorithm is compared with another one based on standard Kalman filtering theory. Numerical results showing the performance for both dynamic estimators under different operational conditions are presented and discussed. Detection and identification of multiple bad data are also included. The new dynamic estimator exploiting state forecasting is extremely useful to real-time monitoring of power systems.

In this paper, we consider the problem of state estimation through observations possibly corrupted with both bad data and additive observation noises. A mixed $\ell_1$ and $\ell_2$ convex programming is used to separate both sparse bad data and additive noises from the observations. Through using the almost Euclidean property for a linear subspace, we derive a new performance bound for the state estimation error under sparse bad data and additive observation noises. Our main contribution is to provide sharp bounds on the almost Euclidean property of a linear subspace, using the "escape-through-a-mesh" theorem from geometric functional analysis. We also propose and numerically evaluate an iterative convex programming approach to performing bad data detections in nonlinear electrical power networks problems.

Identification and Elimination of Bad Data and Line Errors from Power System State Estimators

With the development of smart grids and the continuous penetration of new energy power generation, the scale of the power grid has increased, and the structure and operation methods have become increasingly complex. In order to more accurately grasp the operating status of the power grid, it is necessary to estimate the operating status of the power system. In recent years, with the development of artificial intelligence and data mining technology, neural networks have also been widely used in the field of state estimation. However, there are few researches on artificial intelligence algorithms in active distribution network state estimation, and the small amount of data in the bad data detection link that needs to be carried out before state estimation also limits the application of many artificial intelligence algorithms that require a large number of training samples. A small sample learning method applied to the detection of bad data for power system state estimation is proposed, which successfully solves the problem of the lack of available training data in building a neural network model for bad data detection, and solves the traditional method calculation speed brought by the massive increase of grid data. The problem of serious decline, and the accuracy of the traditional state estimation method is improved by more than 47%.