Design and Application of Distributed Real-time Database System for Massive Data

Real-time database systems combined with distributed architecture possess the capability to satisfy the timing constraints and preserve the data consistency while storing and processing the massive real-time data in the smart grid. In this thesis, we first review the current situation of the research on distributed real-time database. Then according to the characteristics of the electricity data in the smart grid, we design and implement a distributed real-time database. At present, this new distributed real-time database has been already applied into many business systems in State Grid Corporation of China. It presents good stability and accuracy, and provides a reliable real-time data support to the construction of smart grid in China.

Real-time database management system plays an important role in power grid system. With the increasing development of smart grid, the problem of data integrity security in real-time database has attracted more and more attention. Research on real-time database recovery Method of Smart Grid system is based on IEC61970 standard using RFID algorithm. After the data integrity is destroyed, it takes only a rollback malicious transaction, without having to roll back all transactions, the transaction conflict caused by the transaction revocation can be avoided, that can improve the survivability of real-time databases, protect the integrity of the real-time database of smart grids, and improve power grid dispatching system. Therefore, real-time database recovery Method plays an important role in power grid dispatching system.

Real-time database is the combination of real-time systems and database technology with the qualities of high degree of real-time and high data throughput, etc, and has been widely used in the field of automatic control. Historical data is sampled from real-time database system, and preserved as historical data in the database. Users are able to access historical data at any time from the database. Historical data helps achieving the improvements of processes, maintaining the performance of equipments and fault diagnosis. Generally it is closely integrated with trend analysis, report generation and printing services, and plays an increasingly important role in real-time database system. Smart Grid is a power system constituted by a large number of automatic transmission and distribution systems, achieving all power grid operation in a coordinated, effective and reliable way, with a self-healing capability; rapid response to power market and enterprise business needs; equipped with intelligent communication architecture, real-time, secure and flexible information flow, to provide users with reliable and economical electricity service. This paper discusses the characteristics of real-time historical database and smart grid, exploring inner connection between the real-time historical database and smart grid in self-healing, smart scheduling two-way interaction with customers, etc, and analyzes possible ways of real-time historical database being used in Smart Grid.

In this paper we present a concurrency control algorithm that allows co-existence of soft real-time, relational database transactions, and hard real-time database pointer transactions in real-time database management systems. The algorithm uses traditional pessimistic concurrency-control (i.e. locking) for soft transactions and versioning for hard transactions to allow them to execute regardless of any database lock. We provide formal proof that the algorithm is deadlock free and formally verify that transactions have atomic semantics. We also present an evaluation that demonstrates significant benefits for both soft and hard transactions when our algorithm is used. The proposed algorithm is suited for resource-constrained safety critical, real-time systems that have a mix of hard real-time control applications and soft real-time management, maintenance, or user-interface applications.

With the development of new technologies for smart grid construction, the smart grid is gradually developing. Smart grid construction is an important field of next generation information technology, and its value and significance have been widely recognized. Smart grid construction not only provides development opportunities, but also brings great challenges. Security and privacy have become serious problems in the development of smart grid. Based on this, this paper discusses the system structure of the block chain, and puts forward the security diagnosis scheme of the smart grid according to the existing problems in the smart grid. Based on the theoretical basis of smart grid big data technology, this paper combines the grid data encryption process with the privacy protection process under the premise of protecting privacy and security, and develops the latest network security protocol. Then use block chain distributed storage to reduce the connection protection process in privacy, and use multi-level encryption. The experimental results show that the technology can improve the rate of sensitive data hiding and has good anti-cracking ability, which provides a theoretical reference for the construction of enterprise power grid.

As a key part of the power system, the substations undertake important work. With the development and construction of smart grids, the status data, image monitoring data, and environmental meteorological data of power systems are gradually being integrated and shared on a unified platform. Traditional models based on theoretical analysis are difficult to deal with multi-dimensional, massive data set information. Under this background, starting from the inherent law of the data itself, the use of machine learning methods combined with the infrared image data of the equipment can achieve intelligent identification and early warning of substation equipment failures. This paper first uses the convolutional neural network to identify the substation equipment in the picture, and then combines the infrared image to perform image registration. Finally, the deep belief network is used to determine whether the device is in wrong condition. The overall substation equipment fault identification is tested on real data, and the experimental results show that the proposed method has high accuracy.

A Real-Time DataBase (RTDB) system can be viewed as an amalgamation of a conventional DataBase Management System (DBMS) and a real-time system. Like a DBMS, it has to process transactions and guarantee basic correctness criteria. Furthermore, it has to operate in real-time, satisfying timing constraints imposed on transaction commitments and on the temporal validity of data. A RTDB system may exist as a stand-alone system or as an embedded component in a larger multidatabase system. Although various issues related to the concurrency control and transaction scheduling with timing constraints have been studied earlier, the publication of a 1988 special issue of ACM SIGMOD Record on Real-Time DataBases [22] presented the first collection of papers with structured treatments and extensive discussions on RTDB, marking the first significant milestone of the RTDB research area. Today, almost nine years later, we are pleased to present in this book more recent advances in RTDB systems research.

With the implemeGEPI-173ntation of unattended and intelligent substation, the operation and maintenance personnel are far away from the substation. The cooling system lacks real-time monitoring, fault prediction, remote control and other functions. The number of transformer non-stop caused by cooler fault is further increased, which directly threatens the power grid security. In order to solve the above problems, a new intelligent air-cooling control method is proposed. Based on the large-scale signal acquisition method, a new intelligent air-cooling system design scheme is proposed. The characteristics, hardware system design, software system design, main program design, oil temperature control design, comprehensive switching design, data sharing center design message flow design and other key links of the new intelligent transformer cooler control system are introduced in detail. The results show that the scheme can better meet the needs of unattended substation and realize the intelligent operation of substation. It can further improve the security capacity of transformer production and play an important role in the construction and development of smart grid.

An extended transaction model, called the Adaptive Transaction Model, is proposed for Real-Time Database Management System (RTDBMS) applications where partial and/or approximate results are utilized. The Adaptive Transaction Model is a nested transaction model with optional and required subtransactions. Adaptive Transactions (ATs) have time constraints to support real-time database applications. Optional subtransactions can be eliminated during execution if time (or the scheduling policy) does not permit for completion, thereby reporting partial results. A risk-based transaction load control mechanism determines each transaction’s probability of completion within its deadline as well as the overall system load.

Smart grid planning and construction is a very complex engineering project that requires intelligent technology to protect systems such as grid lines and substations to control grid stability and power quality. The smart grid GIS resource allocation technology has designed a new system architecture for the needs of building smart grids. The article expounds the key technologies and typical applications of smart grid GIS resource allocation technology, and describes its broad application prospects in smart grid. As a public service platform for graphical display applications, GIS platform constructs a spatial geographic information data center, which is intelligent. Applications such as grid integration, data mining, and intensive resource utilization provide a massive data storage, management, and analysis environment. The grid GIS platform provides geospatial information sharing and business application integration for smart grid power generation, transmission, substation, power distribution, power consumption, dispatching, etc., providing planning methods and visual analysis services for the construction of smart grid systems.

Real-time databases are increasingly being used as an integral part of many computer systems. During normal operation, transactions in real-time databases must be executed in such a way that transaction timing and data time validity constraints can be met. Real-time databases must also prepare for possible failures and provide fault tolerance capability. Principles for fault tolerance in real-time databases must take timing requirements into consideration and are distinct from those for conventional databases. We discuss these issues in this paper and describe a logging and recovery technique that is time-cognizant and is suitable for an important class of real-time database applications. The technique minimizes normal runtime overhead caused by logging and has a predictable impact on transaction timing constraints. Upon a failure, the system can recover critical data to a consistent and temporally valid state within predictable time bounds. The system can then resume its major functioning, while non-critical data is being recovered in the background. As a result, the recovery time is bounded and shortened. Our performance evaluation via simulation shows that logging overhead has a small effect on missing transaction deadlines while adding recovery capability. Experiments also show that recovery using our approach is 3 to 6 times faster than traditional recovery.

With the application of real-time databases and the intrusion of malicious transactions, it has become increasingly important to model the ability of real-time database intrusion tolerance and effectively evaluate its survivability. Based on the features of transaction and data for real-time database system, an intrusion tolerant architecture has been proposed for real-time database system. Considering factors such as intrusion detection latency and a variety of parameters for real-time, Semi-Markov evaluation model for survival assessment is established. Based on this model, relevant quantitative criteria are made to define the important indicators of survivability, such as integrity and availability, so as to validate intrusion detection capability and the survivability of real-time database. The three important factors of false alarm, detection rate and the intensity of attack are analyzed in detail by the TPC-C benchmark. Experiments show that the model can accurately predict the behavior of real-time database. The real-time database following the model can still provide essential services when facing attacks and the basic survival characteristics will not be seriously affected.

The construction of smart grid has inevitably become the direction of the development of the power grid, and the use of data mining technologies to improve the level of data analysis is an important part of the construction of smart grid. In power grid scheduling, as the basis for the dispatcher to record the operation status of the power grid, the scheduling log usually uses short text to record the current state of the power grid, accident handling and abnormal alarm. However, it is lack of further analysis of the log. Aiming at the automatic classification of power grid dispatching log faults, we firstly use the custom power word and word segmentation tool to preprocess the text. After obtaining the TF-IDF feature of the text and creating the Word2Vec feature model, we compare three kinds of text classification algorithms, the nearest neighbor algorithm, Naive Bayes and support vector machine respectively. It is found that the support vector machine method is very effective in classification with a correct rate of 93% in the log short text.

Increasing complexity of real-time systems, and demands for enabling their configurability and tailorability are strong motivations for applying new software engineering principles, such as aspect-oriented and component-based development. In this paper we introduce a novel concept of aspectual component-based real-time system development. The concept is based on a design method that assumes decomposition of real-time systems into components and aspects, and provides a real-time component model that supports the notion of time and temporal constraints, space and resource management constraints, and composability. We anticipate that the successful applications of the proposed concept should have a positive impact on real-time system development in enabling efficient configuration of real-time systems, improved reusability and flexibility of real-time software, and modularization of crosscutting concerns. We provide arguments for this assumption by presenting an application of the proposed concept on the design and development of a configurable embedded real-time database, called COMET. Furthermore, using the COMET system as an example, we introduce a novel way of handling concurrency in a real-time database system, where concurrency is modeled as an aspect crosscutting the system.

Many real-time database systems are contained in environments that exhibit restricted access to information, such as government agencies, hospitals and military institutions, where mandatory access control for security is required. In addition to such security constraints, real-time database systems have real-time integrity constraints. These real-time constraints require deadlines to be met and data to be temporally consistent. Conventional multi-level secure database models are inadequate for time-critical applications and conventional real-time database models do not support security constraints. We propose a new concurrency control algorithm for secure real-time databases. We implement the algorithm and study the performance using a real-time database system simulation model. Results show that the algorithm performs fairly well in terms of security and timeliness compared to the non-secure algorithm. We argue and show that achieving more security does not necessarily mean a great deal of sacrifice in maintaining real-time constraints.