A Review of Multi-source Heterogeneous Twin Data Processing Methods in Traffic Scenes

The study of the prediction of stock market volatility is of great significance to rationally control financial market risks and increase excessive investment returns and has received extensive attention from academic and commercial circles. However, as a dynamic and complex system, the stock market is affected by multiple factors and has a comprehensive capability to include complex financial data. Given that the explanatory variables of influencing factors are diverse, heterogeneous and complex, the existing intelligent algorithms have great limitations for the analysis and processing of multi-source heterogeneous data in the stock market. Therefore, this study adopts the edge weight and information transmission mechanism suitable for subgraph data to complete node screening, the gate recurrent unit (GRU) and long short-term memory (LSTM) to aggregate subgraph nodes. The compiled data contain the metapaths of three types of index data, and the introduction of the association relationship attention dimension effectively mines the implicit meanings of multi-source heterogeneous data. The metapath attention mechanism is combined with a graph neural network to complete the classification of multi-source heterogeneous graph data, by which the prediction of stock market volatility is realized. The results show that the above method is feasible for the fusion of heterogeneous stock market data and the mining of implicit semantic information of association relations. The accuracy of the proposed method for the prediction of stock market volatility in this study is 16.64% higher than that of the dimensional reduction index and 14.48% higher than that of other methods for the fusion and prediction of heterogeneous data using the same model.

With the rapid development of smart grid technology, a large amount of multi-source heterogeneous data has been generated in the power system, and its effective utilization is crucial for the optimization operation, demand prediction, and anomaly detection of the power system. However, the fusion processing of multi-source heterogeneous data faces many challenges, such as inconsistent data format, granularity, and quality, and direct fusion can easily lead to information redundancy and contradictions. A multi-source heterogeneous data fusion technology based on big data mining has been proposed to address the above issues. This method combines the advantages of convolutional neural networks and gated recurrent units to automatically extract features from image and sequence data and handle long-term dependency issues in time series data. Meanwhile, the K-means clustering algorithm is used to preprocess the data and train a specialized ConvGRU model. The results showed that in short-term load forecasting and abnormal electricity consumption behaviour detection tasks, the accuracy of this method reached 96.3% and 98.7%, respectively, with AUC values of 0.994 and 0.996. Compared to models that use only CNN or GRU, the performance is significantly improved. This method effectively solves the problem of integrating and processing multi-source heterogeneous power data, improves the accuracy and efficiency of power system data analysis, and provides strong support for the optimized operation of smart grids.

With the growing importance of the fuzzy spatiotemporal data in information application, there is an increasing need for researching on the integration method of multi-source heterogeneous fuzzy spatiotemporal data. In this paper, we first propose a fuzzy spatiotemporal RDF graph model based on RDF (Resource Description Framework) that proposed by the World Wide Web Consortium (W3C) to represent data in triples (subject, predicate, object). Secondly, we analyze and classify the related heterogeneous problems of multi-source heterogeneous fuzzy spatiotemporal data, and use the fuzzy spatiotemporal RDF graph model to define the corresponding rules to solve these heterogeneous problems. In addition, based on the characteristics of RDF triples, we analyze the heterogeneous problem of multi-source heterogeneous fuzzy spatiotemporal data integration in RDF triples, and provide the integration methods FRDFG in this paper. Finally, we report our experiments results to validate our approach and show its significant superiority.

Aiming at the low cleaning rate of the traditional multi-source heterogeneous power grid big data cleaning model, a multi-source heterogeneous power grid big data cleaning model based on machine learning classification algorithm is designed. By capturing high-quality multi-source heterogeneous power grid big data, weight labeling of data source importance measurement, data attributes and tuples, and constructing Tan network based on the idea of machine learning classification algorithm, the data probability value is finally used to complete the classification and cleaning of inaccurate data. Experiments show that the model based on machine learning classification algorithm can effectively improve the imprecise data cleaning rate compared with the traditional model to solve multi-source heterogeneous imprecise data cleaning.

In order to fully utilize the features of multi-source heterogeneous data and effectively improve the accuracy and efficiency of fault diagnosis of rotating machinery, a multi-source heterogeneous data diagnosis method based on parameter collaborative optimization multi-scale convolutional autoencoder (MSCAE) is proposed. Firstly, multi-scale information learning is integrated into the convolutional autoencoder (CAE) to consider the temporal and spatial feature information of the diagnostic object simultaneously. To improve the training and diagnostic efficiency of MSCAE, a quantum particle swarm optimization (QPSO) module is used to perform hyperparameter optimization on it using chaos initialization and dynamic weight strategy (DWS). Besides, the sparse attention mechanism is introduced into the MSCAE model to improve the recognition rate of key fault features hidden in the original heterogeneous signals. Finally, the confusion matrix and visualization techniques are used to achieve fault classification. The experimental results demonstrate that after 100 experiments, the proposed method has an average diagnostic accuracy of 98.5 % and strong robustness to noise, providing a new method for rotating machinery fault diagnosis based on multi-source heterogeneous data.

The complexity and diversity of equipments presented in the development of modern industrial technology brings the wide application of deep learning in fault diagnosis. However, the multi-source heterogeneous data collected from different sensors applied in industrial production cannot be used by traditional neural networks directly, which registers a difficulty to fault diagnosis based on deep learning. In addition, with only one of the data used, traditional deep learning fault diagnosis methods ignores the correlation between heterogeneous data, which may result in the loss of useful information and the accuracy of fault diagnosis. To solve the above problems, a fault diagnosis framework based on deep learning for multi-source heterogeneous data fusion is proposed. In the feature learning stage, stack self-encoder (SAE) and convolution neural network (CNN) are used to extract deep features from one-dimensional vibration data and two-dimensional image data respectively, and then feature fusion fault diagnosis is carried out. It solves the problem that traditional neural network can't make full use of multi-source heterogeneous data, and improves the accuracy of fault diagnosis. What's more, the bearing data of Case Western Reserve University are used to verify the validity of the diagnosis method proposed in this paper.

With continuous innovation of the “Internet+” and development of related industries, huge amounts of data need to be integrated on the basis of spatial data analysis and processing. Therefore, efficient integration of multi-source heterogeneous spatial data and data interoperability requirements has become more and more urgent. The design of a multisource heterogeneous vector spatial data integration scheme on WebGIS has been proposed by GeoJSON as the universal transformation format, based on the analysis of the existing multi-source heterogeneous spatial data conversion method. Finally, a prototype of the online prototype system was developed with. Net platform to verify the feasibility and practicability of the scheme. The scheme has the characteristics of high efficiency, easy to expand and low cost, and provides a new way to realize the data integration of “Internet+”.

This paper proposes an optimization algorithm for constructing education knowledge graphs based on multi-source heterogeneous data fusion (MEGO). The algorithm first cleans and extracts features from multi-source education data, and uses deep learning models to mine deep features of different types of data, then dynamically adjusts the weights of each data source through the attention mechanism to achieve efficient fusion of multi-source data, finally, the graph structure is optimized with the help of graph convolutional networks to remove noise and redundant information. The experiment selects <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{1 0 0, 0 0 0}$</tex> multi-source heterogeneous education data from 10 universities in a certain region within 5 years, and compares them with the traditional TransE algorithm and the KG-Basic algorithm based on a single data source. The results show that the accuracy of the MEGO algorithm is 92.3%, which is 8.7% higher than the TransE algorithm and 12.5% higher than the KG -Basic algorithm, the recall rate is 89.6%, which is 7.2% and 10.8% higher than the two comparison algorithms respectively, the F1 value is 90.9%, which is 7.9% and 11.6% higher respectively, the graph construction time is only 12.5 minutes, which is 35.2% shorter than TransE and 42.8% shorter than KG-Basic. In summary, the MEGO algorithm can effectively improve the quality and construction efficiency of the education knowledge graph, and provide solid technical support for multi-dimensional and precise education quality evaluation.

This paper proposes an energy efficiency optimisation method combined with big data analysis to address the energy efficiency management challenges in intelligent manufacturing workshops. The research constructs a multi-source heterogeneous energy consumption data collection and processing architecture, develops energy consumption pattern recognition technology based on wavelet transform and deep learning, establishes equipment energy efficiency assessment and parameter optimisation mechanism, realises production and energy co-scheduling, and constructs a digital twin-driven energy efficiency closed-loop optimisation system. Through the implementation and application in automotive parts, electronic products and petrochemical enterprises, the method is verified to be of significant value in improving energy efficiency, reducing production costs and promoting green manufacturing, providing technical support for the green transformation of the manufacturing industry.

To deal with the bottle-neck of enterprise distributed multi-source heterogeneous geographic spatial data's interoperating, the four-tier integration framework of multi-source heterogeneous spatial information based on web services is designed. Through the integration of web services technology, the geographical information service criterion of OGC and the scalable vector graphics model (SVG) of W3C, the integrated device of spatial data translating services is constructed, which can realize the heterogeneous spatial data's interoperating and showing by spanning the system's heterogeneous and providing a kind of transparent loosely coupling relation between the users and the servers, service modules can be called through job scheduling services, lastly multi-source heterogeneous geographic spatial information can be integrated and remotely computed. It is primarily designed and applied in the State Earthquake Computing Application System (SECAS) based on the above framework and .NET platform, it realities multi-source heterogeneous geographic spatial data's issuing and remote computing.

With the rapid development of information technology, large amounts of multi-source data are constantly being generated in medical field. The automatic visualization system based on them has gained a lot of attention, since the intuitive data presentation can help even non-professional users effectively get the information hidden behind the separate data obtained from different scenarios and make better decisions. In this paper, based on the Data Lake architecture, we improve the performance of an existing novel data visualization recommendation system and resolve three challenges about the processing of multi-source and heterogeneous data. First, we build the framework based on Data Lake to store multi-source and heterogeneous data. Second, we optimize the data manipulation module in the visualization system based on the distributed processing power of Data Lake to get potentially interesting visualization candidates in a short time. Third, we efficiently run exploratory queries on large datasets based on the calculation capability of Data Lake to meet the actual needs of users. According to the experiment results, our system demonstrates a remarkable acceleration effect on the task of automatic visualization of big multi-source medical data.

Wind load identification of lattice towers using multi-source heterogeneous monitoring data

In medical domain, huge amounts of data are generated at all times. These data are usually difficult to access, with poor data quality and many data islands. Besides, with a wide range of sources and complex structure, these data contain essential information and are difficult to manage. However, few existing data management frameworks based on Data Lake excel in solving the persistence and the analysis efficiency for medical multi-source heterogeneous data. In this paper, we propose an efficient Multi-source Heterogeneous Data Lake Platform (MHDP) to realize the efficient medical data management. Firstly, we propose an efficient and unified method based on Data Lake to store data of different types and different sources persistently. Secondly, based on the unified data store, an efficient multi-source heterogeneous data fusion is implemented to effectively manage data. Finally, an efficient data query strategy is carried out to assist doctors in medical decision-making. In-depth analysis on applications shows that MHDP delivers better performance for data management in medical domain.

With the rapid development of domestic power grid technology, it is difficult for traditional power grids to solve the low-voltage distribution end multi-source data comprehensive processing efficiency and poor cross-data fusion effect. At present, with the vigorous development of edge computing, cloud containers and 5G technologies, new methods are provided for the processing and integration of multi-source data in low-voltage distribution networks. This paper analyzes edge computing and cloud container technologies to design a reasonable framework for low-voltage power distribution data processing to help the construction of smart grids.

Digital oil and gas field is an overly complex integrated information system, and with the continuous expansion of business scale and needs, oil companies will constantly raise more new and higher requirements for digital transformation. In the previous system construction, we adopted multi-phase, multi-vendor, multi-technology and multi-method, resulting in the problem of data silos and fragmentation. The result of the data management problems is that decisions are often made using incomplete information. Even when the desired data is accessible, requirements for gathering and formatting it may limit the amount of analysis performed before a timely decision must be made. Therefore, through the use of advanced computer technologies such as big data, cloud computing and IOT (internet of things), it has become our current goal to build an integrated data integration platform and provide unified data services to improve the company's bottom line. As part of the digital oilfield, offshore drilling operations is one of the potential areas where data processing and advanced analytics technology can be used to increase revenue, lower costs, and reduce risks. Building a data mining and analytics engine that uses multiple drilling data is a difficult challenge. The workflow of data processing and the timeliness of the analysis are major considerations for developing a data service solution. Most of the current analytical engines require more than one tool to have a complete system. Therefore, adopting an integrated system that combines all required tools will significantly help an organization to address the above challenges in a timely manner. This paper serves to provide a technical overview of the offshore drilling data service system currently developed and deployed. The data service system consists of four subsystems. They are the static data management system including structured data (job report) and unstructured data (design documentation and research report), the real-time data management system, the third-party software data management system integrating major industry software databases, and the cloud-based data visual application system providing dynamic analysis results to achieve timely optimization of the operations. Through a unified logical data model, it can realize the quick access to the third-party software data and application support; These subsystems are fully integrated and interact with each other to function as microservices, providing a one-stop solution for real-time drilling optimization and monitoring. This data service system has become a powerful decision support tool for the drilling operations team. The learned lessons and gained experiences from the system services presented here provide valuable guidance for future demands E&amp;P and the industrial revolution.