Automated Root Cause Analysis in Large-Scale IT Infrastructure Using Big Data Analytics

Abstract

Translate Article

Modern IT infrastructures are increasingly complex and prone to frequent failures that demand real-time diagnosis and resolution. This paper presents a hybrid big data analytics framework for automated root cause analysis (RCA), combining deep learning and machine learning techniques with SHAP-based interpretability. Using the LEMMA-RCA dataset, comprising multi-modal system logs and performance metrics, the framework processes heterogeneous data through a Bi-LSTM for log sequences and a CNN for metric patterns, followed by fusion and classification using a Random Forest model. The system achieves high predictive performance with an accuracy of 94.8 %, precision of 93.5 %, recall of 92.7 %, F1-score of 93.1 %, and AUC-ROC of 96.2%. SHAP values reveal that Error Code, CPU Usage, and Memory Usage are the most influential features. The results confirm the framework's effectiveness in both detecting root causes and offering interpretable, actionable insights for large-scale IT infrastructure monitoring.