• Home
  • Search
  • A Disk Failure Prediction Method Based on Active Semi-supervised Learning
  • Open Access IconOpen Access
  • Cite Icon9
  • https://doi.org/10.1145/3523699Copy DOI Icon

A Disk Failure Prediction Method Based on Active Semi-supervised Learning

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Disk failure has always been a major problem for data centers, leading to data loss. Current disk failure prediction approaches are mostly offline and assume that the disk labels required for training learning models are available and accurate. However, these offline methods are no longer suitable for disk failure prediction tasks in large-scale data centers. Behind this explosive amount of data, most methods do not consider whether it is not easy to get the label values during the training or the obtained label values are not completely accurate. These problems further restrict the development of supervised learning and offline modeling in disk failure prediction. In this article, Active Semi-supervised Learning Disk-failure Prediction ( ASLDP ), a novel disk failure prediction method is proposed, which uses active learning and semi-supervised learning. According to the characteristics of data in the disk lifecycle, ASLDP carries out active learning for those clear labeled samples, which selects valuable samples with the most significant probability uncertainty and eliminates redundancy. For those samples that are unclearly labeled or unlabeled, ASLDP uses semi-supervised learning for pre-labeled by calculating the conditional values of the samples and enhances the generalization ability by active learning. Compared with several state-of-the-art offline and online learning approaches, the results on four realistic datasets from Backblaze and Baidu demonstrate that ASLDP achieves stable failure detection rates of 80–85% with low false alarm rates. In addition, we use a dataset from Alibaba to evaluate the generality of ASLDP . Furthermore, ASLDP can overcome the problem of missing sample labels and data redundancy in large data centers, which are not considered and implemented in all offline learning methods for disk failure prediction to the best of our knowledge. Finally, ASLDP can predict the disk failure 4.9 days in advance with lower overhead and latency.

Similar Papers
  • Conference Article
  • Citations5

ASLDP: An Active Semi-supervised Learning method for Disk Failure Prediction

  • Aug 09, 2021
  • Yang Zhou +2
  • Research Article
  • Citations49

Minority Disk Failure Prediction Based on Transfer Learning in Large Data Centers of Heterogeneous Disk Systems

  • Sep 01, 2020
  • IEEE Transactions on Parallel and Distributed Systems
  • Ji Zhang +7
  • Conference Article
  • Citations19

Transfer Learning based Failure Prediction for Minority Disks in Large Data Centers of Heterogeneous Disk Systems

  • Aug 05, 2019
  • Ji Zhang +7
  • Conference Article
  • Citations3

CSLE: A Cost-sensitive Learning Engine for Disk Failure Prediction in Large Data Centers

  • Mar 14, 2022
  • Xinyan Zhang +3
  • PDF
  • Research Article
  • Citations29

Active semi-supervised learning for biological data classification

  • Aug 19, 2020
  • PLoS ONE
  • Guilherme Camargo +2
  • Research Article
  • Citations43

A novel myocardial infarction localization method using multi-branch DenseNet and spatial matching-based active semi-supervised learning

  • May 25, 2022
  • Information Sciences
  • Ziyang He +7
  • Book Chapter

Semi-supervised Dictionary Active Learning for Pattern Classification

  • Jan 01, 2018
  • Qin Zhong +2
  • Conference Article
  • Citations1

Failure Order: A Missing Piece in Disk Failure Processing of Data Centers

  • Aug 01, 2019
  • Yusheng Yi +4
  • Conference Article
  • Citations5

ActiveMatch: End-To-End Semi-Supervised Active Representation Learning

  • Oct 16, 2022
  • Xinkai Yuan +2
  • Conference Article
  • Citations25

Lifelong Disk Failure Prediction via GAN-Based Anomaly Detection

  • Nov 01, 2019
  • Tianming Jiang +4
  • Research Article
  • Citations13

Cost‐efficiency disk failure prediction via threshold‐moving

  • Apr 01, 2020
  • Concurrency and Computation: Practice and Experience
  • Tianming Jiang +2
  • Conference Article
  • Citations14

Network Intrusion Detection Based on Active Semi-supervised Learning

  • Jun 01, 2021
  • Yong Zhang +4
  • Research Article
  • Citations11

A two-phase hybrid of semi-supervised and active learning approach for sequence labeling

  • Apr 17, 2013
  • Intelligent Data Analysis
  • Hamed Hassanzadeh +1
  • Research Article
  • Citations243

Combining active and semi-supervised learning for spoken language understanding

  • Oct 30, 2004
  • Speech Communication
  • Gokhan Tur +2
  • Research Article
  • Citations10

SPAE: Lifelong disk failure prediction via end-to-end GAN-based anomaly detection with ensemble update

  • Jun 17, 2023
  • Future Generation Computer Systems
  • Yu Liu +9
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.