Research and Application of Short-term Power Load Based on Large Data Analysis

Big data is one of the important development direction of modern information technology, realizing the sharing and analysis of large data will bring immeasurable economic value, but also has a tremendous role in promoting the social. In the age of big data, unified the data representation, large data processing, query, analysis and visualization are the key problem to be solved urgently. In order to provide a standardized framework construction of the large data service platform, this paper designed a large data service oriented architecture user experience. Secondly, in the aspect of data model, in order to achieve high data service for non structured data, the design of the non structured data model based on subject behavior. In large data service model, algebraic model large data services and their composition was established by using process algebra. In large data service applications, detailed retrieval, process analysis and visualization services, and by improving the retrieval accuracy and efficiency of the service in two aspects of measures to achieve the high data service optimization.

The purpose of the workis to explore the current problems and prospects of mining solution, big web data in real time, as well as the possibility of practical implementation of Web Mining technology for big web data on a practical example.Materials and methods.The study included a review of bibliographic sources on big data mining. We used Web Mining technology for associative analysis of large web data, as well as computer modeling of the practical task of transaction analysis using a general-purpose scripting language (PHP).Results.During the work, the specifics of the Data Mining technology were described, and a modern approach to the analysis of large web data –Web Mining was analyzed. A brief classification of tasks solved using Web Mining technology is given. The problem of data mining of large web data in a general-purpose scripting language (PHP) has been solved: the lack of libraries for data mining, the difficult normalization of data to the form necessary for associative analysis, interaction with the database management system. Also, an example showing an approach to the mining of large web data was implemented. Based on the understanding of Web Mining technology and the described difficulties of analyzing web data in the PHP language, methods for effectively solving the practical problem of analyzing web data based on transactions committed in a dynamic web application have been proposed. A module for associative analysis of customer transactions in the programming language PHP was developed. The module includes an intelligent data processing class. The structural scheme of the module and system architecture were developed. The constructed module allows us to solve the main part of the problem of associative analysis of large web data using Web Mining technology in order to solve the problem of identifying patterns in a large array of web data. Associative analysis of web data is much faster because of the combination of a general-purpose scripting language and an object-oriented approach.Conclusion.According to the results of the study, it can be argued that the current state of the technology for the analysis of large web data allows efficiently process data objects, identify patterns, obtain hidden data and receive complete statistical data in real time. The results can be used both for the purpose of the initial research of technologies for analyzing large web data, and as an addition to the content management system for the intelligent analysis of web data. The usage of the technology of associative analysis and the created universal handler class makes the created module flexible, while the possibility of manual integration makes this module universal. With manual integration, the database management system is not important. Algorithm methods work with selected data. This factor greatly simplifies the further development of program code.

The purpose of the work is to study the current problems and prospects of the solution for processing big data received or stored in the Internet (web data), as well as the possibility of practical realization of Data Mining technology for big web data on practical example. Materials and methods. The study included a review of bibliographic sources on big data analysis problems.Data Mining technology was used to analyze large web data, as well as computer modeling of a practical problem using the C # programming language and creating a DDL database structure for accumulating web data.Results. In the course of the work, the specifics of big data were described, the main characteristics of big data were highlighted, and modern approaches to processing big data were analyzed. A brief description of the horizontal-scalable architecture and the BI-solution architecture for big data processing is given. The problems of processing large web data are formulated: limiting the speed of access to data, providing access via network protocols through general-purpose networks.An example showing the approach to processing large web data was also implemented. Based on the idea of big data, the described complexities of web data processing and the methods of Data Mining, techniques were proposed for effectively solving the practical problem of processing and searching patterns in a large data array.The following classes have been developed in the C # programming language:Class of receiving web data via the Internet; Data conversion class;Intelligent data processing class;Created DDL script that creates a structure for the accumulation of web data.A single UML class diagram has been developed.The constructed system of data and classes allows to solve the main part of the problems of processing large web data and perform intelligent processing using Data Mining technology in order to solve the problem posed of identifying certain records in a large array. The combination of object-oriented approach, neural networks and BI-analysis to filter data will speed up the process of data processing and obtaining the result of the studyConclusion. According to the results of the study, it can be argued that the current state of technology for analyzing large web data allows you to efficiently process data objects, identify patterns, get hidden data and get full-fledged statistical data.The obtained results can be used both for the purpose of the initial study of big data processing technologies, and as a basis for developing an already real application for analyzing web data. The use of neural networks and the created universal classes-handlers makes the created architecture flexible and self-learning, and the class declarations and the base DDL structure will greatly simplify the development of program code.

Big data analysis technology can be used to transform massive data of oil production engineering into knowledge that can be used to guide oilfield production. Due to the lack of suitable large data analysis tools, a large data analysis software system was developed to simplify the workload of data analysis. This paper illustrates the research and application of large energy consumption data analysis through the development of big data analysis methods, process improvement and software platform, then achieves the goal of reducing costs and improving efficiency.

Big Data is driving radical changes in traditional data analysis platforms. To perform any kind of analysis on such voluminous and complex data, scaling up the hardware platforms becomes impending. With the entire buzz surrounding Big Data; it is being collected at an unprecedented scale. Big Data has potential to revolutionize much more than just research. Loading large data-sets is often a challenge. Another shift of this Big Data processing is the move towards cloud computing. As many communities begin to rely on cloud based data management, large shared data goes up extensively. Analysis of such large data on distributed processing system or cloud is a bit difficult task to handle. The aim of this chapter is to provide a better understanding of the design challenges of cloud computing and analytics of big data on it. The challenge is related to how a large extent of data is being harnessed, and the opportunity is related to how effectively it is used for analyzing the information from it.

As the big data Era, the amount of data in education shows explosive growth. Big data analysis system architecture intends to provide a reference for large educational data analysis. Through literature analysis and network investigation, This paper reviewed the existing mature system of large educational data analysis system. From two aspects of generality and difference to compare, this paper proposed research framework from several aspects, including analysis of the common thinking, the generality of open source thought, analysis of the common areas, technology framework, the core technology and special functions. This paper compares 3 mainstream data analysis system, summarizes the enlightenment for the analysis of large data, puts forward a kind of intelligent analysis system of education big data, and sums up the development prospect of big data analysis system.

Various big-data analytics tools and techniques have been developed for handling massive amounts of data in the healthcare sector. However, scheduling is a significant problem to be solved in smart healthcare applications to provide better quality healthcare services and improve the efficiency of related processes when considering large medical files. For this purpose, a new hybrid model called Gaussian Relevance Vector MapReduce-based Annealed Glowworm Optimization Scheduling (GRVM-AGS) was designed to improve the balancing of large medical data files between different physicians with higher scheduling efficiency and minimal time. First, a GRVM model was developed for the predictive analysis of input medical data. This model reduces the storage complexity of large medical data analysis by means of eliminating unwanted patient information and predicts the disease class with help of a Gaussian kernel function. Afterwards, GRVM performs AGS to schedule the efficient workloads among multiple datacenters based on the luciferin value in the smart healthcare environment with reduced scheduling time. Through computational experiments, we demonstrate that GRVM-AGS increases the scheduling efficiency and reduces the scheduling time of large medical data analysis compared to state-of-the-art approaches.

Almost all consumer and firm transactions are achieved using computers and as a result gives rise to increasingly large amounts of data available for analysts. The gold standard in Economic data manipulation techniques matured during a period of limited data access, and the new Large Data Analysis (LDA) paradigm we all face may quickly obfuscate most tools used by Economists. When coupled with an increased availability of numerous unstructured, multi-modal data sets, the impending 'data tsunami' could have serious detrimental effects for Economic forecasting, analysis, and research in general. Given this reality we propose a decision-aid framework for<i> <strong>Augmented-LDA (A-LDA) </strong></i>– a synergistic approach to LDA which combines traditional supervised, rule-based Machine Learning (ML) strategies to iteratively uncover hidden sources in large data, the artificial neural network (ANN) Unsupervised Learning (USL) at the minimum Helmholtz free energy for isothermal dynamic equilibrium strategies, and the Economic intuitions required to handle problems encountered when interpreting large amounts of Financial or Economic data. To make the ANN USL framework applicable to economics we define the temperature, entropy, and energy concepts in Economics from non-equilibrium molecular thermodynamics of Boltzmann viewpoint, as well as defining an information geometry, on which the ANN can operate using USL to reduce information saturation. An exemplar of such a system representation is given for firm industry equilibrium. We demonstrate the traditional ML methodology in the economics context and leverage firm financial data to explore a frontier concept known as behavioral heterogeneity. Behavioral heterogeneity on the firm level can be imagined as a firm's interactions with different types of Economic entities over time. These interactions could impose varying degrees of institutional constraints on a firm's business behavior. We specifically look at behavioral heterogeneity for firms that are operating with the label of 'Going-Concern' and firms labeled according to institutional influence they may be experiencing, such as constraints on firm hiring/spending while in a Bankruptcy or a Merger procedure. Uncovering invariant features, or <i><strong>behavioral data metrics </strong></i>from observable firm data in an economy can greatly benefit the FED, World Bank, etc. We find that the ML/LDA communities can benefit from Economic intuitions just as much as Economists can benefit from generic data exploration tools. The future of successful Economic data understanding, modeling, simulation, and visualization can be amplified by new A-LDA models and approaches for new and analogous models of Economic system dynamics. The potential benefits of improved economic data analysis and real time decision aid tools are numerous for researchers, analysts, and federal agencies who all deal with increasingly large amounts of complex data to support their decision making.

Abstract. The shortage of human demand change in business enterprise provides an opportunity for employment concept change, which makes the employment concept sample data capacity greatly increased. In order to realize the analysis of large sample data, this paper does curve fitting on the employment large sample data, using Bias quantile coupling and intelligent decoupling process to calculate coupling balance degree of supply and demand data, finally uses MATLAB software to program the algorithm. Through the simulation iterative calculation, the fitted curve of talent supply and demand data in business enterprise is obtained, finally this paper uses the Bias quantile balance optimization algorithm to obtain the fitting the curve of supply and demand in good condition, which provides a new computer method for employment large sample data mining.

This paper presents a theory of large data analysis of vehicle reliability based on vehicle fault data. By collecting fault data of vehicles with different energy types, the theory of vehicle fault statistics and reliability analysis suitable for large data analysis is proposed, and the reliability level of different types of vehicles is evaluated comprehensively by weighted analysis method. Through comparative analysis, this paper reveals the fault change rule and reliability level of new energy vehicles, and provides basic support for improving the reliability of new energy vehicles, reducing maintenance costs, expanding consumers’ purchase choices, and improving the sales volume of new energy vehicles.

The purpose of this paper is to investigate the current situation of community mental health services in Hunan Province. At the same time, this paper can also provide the corresponding basis for community health services. In the era of large data technology so popular, for the survey of mental health, we can use the technology of large data analysis to conduct the questionnaire survey of community mental health services, and conduct random sampling of community residents. The results of this survey include the degree of community residents’ understanding of mental health, the influencing factors of psychological help seeking, the utilization of psychological service resources, the expected charges and the corresponding equipment. According to the results of the questionnaire survey, we can see that the psychological health service needs of residents in Hunan communities are affected by the level of regional economic development, economic and social status, the quality of professional resources and the support attitude of the government.

This study aims to analyze the effect of digital economy, training, and partnerships on the income of micro and small industry in Indonesia. This study uses panel data from 34 provinces in Indonesia in 2017-2018. The data analysis technique used multiple linear regression with panel data approach. The results showed that the digital economy, training and partnerships have a significant positive effect on the micro and small industry in Indonesia. These findings imply 1) the need for socialization about the use of the digital economy in small and micro industry, 2) small and micro industry need to increase partnerships, and 3) the need for a training program from the government to improve the quality of products of small and micro industry in Indonesia.

We have setup a new course on the large scale data processing using clusters. It introduces the concepts and design of distributed systems. Many newly developed ideas such as Google file system and MapReduce programming framework for processing large scale data sets are introduced. Students will gain practical experience with distributed programming technologies via several small labs and one large multi-week final project. Labs and projects will be completed using Hadoop, an open-source implementation of Google's distributed file system and MapReduce programming model. We have taught this class named Mass Data Processing Technology on Large Scale Clusters for two years. This paper will describe the design, perform of the course as well as the experiences and lessons learned.

The application of large data has opened a time of sharing, creation and innovation. Large data change the way people think, and better stimulate new products and design innovations. In the large data age, everyone is the provider and user of data, in the face of this new challenge and new opportunities, in order to improve the efficiency of enterprises, design needs to make corresponding strategic adjustments. Design is a kind of related way to gain benefits, it is a kind of skill to visualize and commercialize concept, and it is a process of transforming frontier thinking into effective strategy. Design is not just a skill, it is more like a methodology for innovative services and design thinking. In this paper, from the large data characteristics, analysis of large data to design inspiration and thinking change. This research design opportunity recognition, through the combination with the large data, provides a forward-looking reference for the design opportunity recognition. The relevant design revelation includes the data as the key reference, the establishment of large data thinking, in-depth excavation design opportunities and data relevance, and then promote the design of innovation and product commercial value.

Diagnosis of secondary progressive multiple sclerosis (SPMS) is based on a history of gradual worsening of neurological symptoms within 6-12 months without exacerbations following an initial relapsing-remitting (RRMS) disease course. In the absence of reliable MRI, immunological and clinical markers, it is hardly possible to achieve objectivity in determining the transition of MS to a progressive stage. This often leads to a long period of diagnostic uncertainty, which prevents timely therapeutic decisions. Physicians expressed an unmet need for a tool that could be used in routine clinical practice to assess the risks of progression to SPMS quickly and reliably, in an easy-to-interpret output for a joint discussion with the patient. From a wide range of disease symptoms and lifestyle factors reflecting the progression to SPMS and obtained by analysis of large clinical data (3294 cases) and a survey of patients and specialists, significant were identified and ranked by categories according to combined expert opinion. A mathematical model was developed and validated, and an electronic version of the MSProDiscuss questionnaire created. Test sensitivity for SPMS diagnosis is 0.82, specificity 0.84, interrater reliability 0.95. Median time for completion of the questionnaire was 2.16 minutes per patient. Physicians of the Russian Federation who used the questionnaire confirmed its high feasibility. Thus, the methodology of the questionnaire development based on the combined opinion of patients and physicians, along with the large clinical data, made it possible to obtain high sensitivity, specificity and reproducibility of the test and takes little time with a clear output. MSProDiscuss may be useful not only in the differential diagnosis of RRMS and SPMS, but also to determine the risks of progression to SPMS, which is of great clinical importance.