• Home
  • Search
  • Distributed Big Data Mining Platform for Smart Grid
  • Cite Icon6
  • https://doi.org/10.1109/bigdata.2018.8622163Copy DOI Icon

Distributed Big Data Mining Platform for Smart Grid

  • Dec 1, 2018
  • Zhixiang Wang +3 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

With the rapid development of information technology and internet, all kinds of industry data exploded causing difficult to analyze and mine useful information from big data. Traditional analysis system has bottlenecks of performance and scalability in big data processing. The research and development of novel and efficient big data analysis and mining platform has become the focus of all organizations. Along with the development of smart grid, power data with characteristics of power industry needs more targeted and efficient data mining analysis. In this paper, aiming at the shortage of existing work, we propose a distributed big data mining platform based on distributed system infrastructure such as Hadoop and Spark. The platform develops and implements a variety of rapid highly parallel mining algorithm by Spark and Tensorflow, including machine learning, statistics and analysis, deep learning and so on. Using the OSGI technology to build low coupling component model, the platform improve reusability of component algorithm, introduces the workflow engine and user-friendly GUI, reduces the complexity of the user operations, support user-defined data mining tasks. For the characteristics of smart grid big data, the platform develops and improves the dozens of algorithm components about data processing and analysis. And designing a scalable algorithms library and the component library greatly improves the scalability of big data mining platform and processing smart grid data. Our platform has already been launched in a state grid Company, satisfying the demand of various smart grid data analysis business.

Similar Papers
  • Conference Article
  • Citations10

Big data processing and analysis platform for condition monitoring of electric power system

  • Aug 01, 2016
  • Yuanjun Guo +5
  • Research Article

Professional Big Data Technology Training Courses in Universities

  • Feb 25, 2019
  • DEStech Transactions on Social Science, Education and Human Science
  • Allam Maalla +2
  • Research Article
  • Citations4

Application Research of Big Data Real-time Processing Technology in Smart Grid

  • Mar 01, 2021
  • IOP Conference Series: Earth and Environmental Science
  • Sheng Zhou +6
  • PDF
  • Research Article
  • Citations4

The problem of analysis of big web data and the use of data mining technology for processing and searching patterns in big web data on a practical example

  • May 14, 2019
  • Open Education
  • K V Mulyukova +1
  • Single Book
  • Citations27

Big Data Management and Processing

  • May 19, 2017
  • Albert Y Zomaya
  • Research Article

МЕТОДЫ ИНТЕЛЛЕКТУАЛЬНОГО АНАЛИЗА И ОБРАБОТКИ БОЛЬШИХ ДАННЫХ

  • Jan 01, 2024
  • EKONOMIKA I UPRAVLENIE: PROBLEMY, RESHENIYA
  • Petimat L Zulkarnayeva +1
  • Research Article
  • Citations32

Big Data Processing and Mining for Next Generation Intelligent Transportation Systems

  • Jul 15, 2013
  • Jurnal Teknologi
  • Jelena Fiosina +1
  • Conference Article
  • Citations2

Research on Dynamically Configurable Data Acquisition System in the Big Data Era

  • Oct 07, 2021
  • Demei Gao
  • Research Article
  • Citations17

Discussion on geological science big data and its applications

  • May 16, 2016
  • Chinese Science Bulletin
  • Chonglong Wu +4
  • Research Article
  • Citations191

Utilizing Cloud Computing to address big geospatial data challenges

  • Nov 02, 2016
  • Computers, Environment and Urban Systems
  • Chaowei Yang +4
  • Research Article
  • Citations10

Cloud computing and big data: Technologies and applications

  • Mar 29, 2017
  • Concurrency and Computation: Practice and Experience
  • Mostapha Zbakh +2
  • Conference Article
  • Citations13

Challenges of Big Data to Big Data Mining with their Processing Framework

  • Nov 01, 2018
  • Kamlesh Kumar Pandey +1
  • Conference Article
  • Citations11

The survey on approaches to efficient clustering and classification analysis of big data

  • Aug 01, 2016
  • Bhagyashri S Gandhi +1
  • Conference Article
  • Citations51

An architecture for big data processing on intelligent transportation systems. An application scenario on highway traffic flows

  • Sep 01, 2016
  • Guilherme Guerreiro +4
  • PDF
  • Research Article
  • Citations2

Research on Graphic-Text Relationship in Film and Television Works Based on Big Data Model

  • Jun 24, 2022
  • Mobile Information Systems
  • Jun Liu +1
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.