Deep learning methods and applications for electrical power systems: A comprehensive review

This review paper examines the current state of lung disease diagnosis based on deep learning (DL) methods. Lung diseases, such as Pneumonia, TB, Covid-19, and lung cancer, are significant causes of morbidity and mortality worldwide. Accurate and timely diagnosis of these diseases is essential for effective treatment and improved patient outcomes. DL methods, which utilize artificial neural networks to extract features from medical images automatically, have shown great promise in improving the accuracy and efficiency of lung disease diagnosis. This review discusses the various DL methods that have been developed for lung disease diagnosis, including convolutional neural networks (CNNs), deep neural networks (DNNs), and generative adversarial networks (GANs). The advantages and limitations of each method are discussed, along with the types of medical imaging techniques used, such as X-ray and computed tomography (CT). In addition, the review discusses the most commonly used performance metrics for evaluating the performance of DL for lung disease diagnosis: the area under the curve (AUC), sensitivity, specificity, F1-score, accuracy, precision, and the receiver operator characteristic curve (ROC). Moreover, the challenges and limitations of using DL for lung disease diagnosis, including the limited availability of annotated data, the variability in imaging techniques and disease presentation, and the interpretability and generalizability of DL models, are highlighted in this paper. Furthermore, strategies to overcome these challenges, such as transfer learning, data augmentation, and explainable AI, are also discussed. The review concludes with a call for further research to address the remaining challenges and realize DL's full potential for improving lung disease diagnosis and treatment.

Deep learning methods have become popular among researchers in the field of fault detection. However, their performance depends on the availability of big datasets. To overcome this problem researchers started applying transfer learning to achieve good performance from small available datasets, by leveraging multiple prediction models over similar machines and working conditions. However, the influence of negative transfer limits their application. Negative transfer among prediction models increases when the environment and working conditions are changing continuously. To overcome the effect of negative transfer, we propose a novel deep transfer learning method, coined deep boosted transfer learning, for wind turbine gearbox fault detection that prevents negative transfer and only focuses on relevant information from the source machine. The proposed method is an instance-based deep transfer learning method that updates the weights of the source and the target machine training samples separately. The weights of different source training samples are gradually decreased to reduce the impact on the final model. The proposed method is verified by the Case Western Reserve University bearing and real field wind farm datasets. The results show that the proposed method ignores negative transfer and achieves higher accuracy compared to standard deep learning and deep transfer learning methods.

Excessive withdrawal of groundwater for agricultural irrigation can cause seawater intrusion into coastal aquifers. Such a case will in turn results in deterioration of irrigation water quality. Determination of irrigation water quality with traditional methods is a time-consuming and costly process. However, machine learning algorithms can be useful tools for modeling and estimating groundwater quality used for irrigation water purposes. In this study, TDS, PS, SAR, and Cl parameters of groundwater were estimated with models based on EC and pH variables. For this purpose, prediction performances of two different deep learning methods (convolutional neural network (CNN) and deep neural network (DNN)) and two different classical machine learning (Random Forest (RF) and extreme gradient boosting (XGBoost)) methods were compared. In addition, predictive uncertainty of the models was determined by quantile regression (QR) analysis. Performance criteria and results of uncertainty analysis revealed that CNN (in testing phase, NSE = 0.95 for TDS, NSE = 0.96 for PS, NSE = 0.67 for SAR and NSE = 0.93 for CI) and DNN (in testing phase, NSE = 0.91 for TDS, NSE = 0.91 for PS, NSE = 0.57 for SAR and NSE = 0.94 for Cl) models had quite a close performance in estimation of TDS, PS, SAR, and Cl parameters and higher than the other two classical machine learning methods. As a result, the CNN model can be considered the best performing model in estimating all quality parameters due to the highest NSE and lowest RMSE values. In addition, the Taylor diagram showed that the values estimated using the CNN model had the highest correlation with the measured data. It was determined that the model with the lowest uncertainty based on the PICP statistics was DNN, followed by the CNN model. However, the CNN model has predicted outliers more accurately. Present findings proved that deep learning models could offer efficient tools for predicting irrigation water quality parameters.

Distributed generation (DG) has changed the energy landscape by increasing system reliability and decreasing environmental impacts through its incorporation into electrical power systems. The additional complexity brought forth by DG, however, also calls for the creation of more precise and effective fault finding techniques. This abstract explores current developments in fault localization strategies for DG-integrated electrical power systems using deep learning (DL) methods. Due to the non-linear, dynamic, and ever-changing nature of DG-rich networks, traditional fault finding algorithms generally struggle to correctly locate faults in these networks. Deep learning's potential as a solution may be seen in its capacity to recognise complex patterns and adjust to new information. Specifically, this study looks into how DL models like CNNs and RNNs can be used to improve the precision with which faults can be detected and localised in DG-integrated power systems. The creation of DL-based fault location algorithms that use high-resolution data from different sensors like smart metres and phasor measurement units (PMUs) is a major advancement. These algorithms draw on the temporal and spatial information available in power system data to pinpoint the exact location of malfunctions. Additionally, the study looks into the stability and transferability of DL models using various DG technologies and setups. Pre-trained models are transferred learning approaches to guarantee adaptability and reliability across a wide range of DG applications. The findings show that in DG-rich contexts, fault location algorithms based on DL perform much better than their conventional counterparts. These developments have the potential to strengthen electrical grids, reduce the frequency and duration of outages, and improve the reliability of today's power systems. The incorporation of deep learning into fault location algorithms is an important step towards building a more dependable and robust power system, especially as DG integration continues to grow

Thermal load forecasting in district heating networks using deep learning and advanced feature selection methods

Artificial Intelligence has been widely applied today, and the subsequent privacy leakage problems have also been paid attention to. Attacks such as model inference attacks on deep neural networks can easily extract user information from neural networks. Therefore, it is necessary to protect privacy in deep learning. Differential privacy, as a popular topic in privacy-preserving in recent years, which provides rigorous privacy guarantee, can also be used to preserve privacy in deep learning. Although many articles have proposed different methods to combine differential privacy and deep learning, there are no comprehensive papers to analyze and compare the differences and connections between these technologies. For this purpose, this paper is proposed to compare different differential private methods in deep learning. We comparatively analyze and classify several deep learning models under differential privacy. Meanwhile, we also pay attention to the application of differential privacy in Generative Adversarial Networks (GANs), comparing and analyzing these models. Finally, we summarize the application of differential privacy in deep neural networks.

With the rapid growth of indoor positioning requirements without equipment and the convenience of channel state information acquisition, the research on indoor fingerprint positioning based on channel state information is increasingly valued. In this article, a multi-level fingerprinting approach is proposed, which is composed of two-level methods: the first layer is achieved by deep learning and the second layer is implemented by the optimal subcarriers filtering method. This method using channel state information is termed multi-level fingerprinting with deep learning. Deep neural networks are applied in the deep learning of the first layer of multi-level fingerprinting with deep learning, which includes two phases: an offline training phase and an online localization phase. In the offline training phase, deep neural networks are used to train the optimal weights. In the online localization phase, the top five closest positions to the location position are obtained through forward propagation. The second layer optimizes the results of the first layer through the optimal subcarriers filtering method. Under the accuracy of 0.6 m, the positioning accuracy of two common environments has reached, respectively, 96% and 93.9%. The evaluation results show that the positioning accuracy of this method is better than the method based on received signal strength, and it is better than the support vector machine method, which is also slightly improved compared with the deep learning method.

Power system fault type classification and location prediction is critical in assessing the reliability of the power system, and later restoring it to a stable operating point followed by a fault. State of the art methods include sequence component, impedance measurement from the origin of the fault, and traveling wave based methods for fault type detection and classification problem. It is important to identify and classify the fault as quickly as possible for restoring power system stability to normal operation. Machine Learning and Deep Learning methods allow the analysis of large data of fault voltages and currents by using fast and efficient algorithms. These methods require large amount of data, however, with the recent advances in the field of power system, data acquisition using smart meters and Phasor Measurement Units (PMU), huge amount of system-wide data can be made available to analyze the problem of fault type classification and location prediction. This paper presents a comparative study of Stochastic Gradient Descent (SGD) based Deep Neural Network (DNN) and Machine Learning (ML) applied to power system fault type and location prediction problem. DNN architecture uses 10 hidden layers, each layer having 60 units with hyperbolic tangent as activation function, and a combination of Support Vector Machine (SVM) and Principal Component Analysis (PCA) method are considered. Comparative results in terms of time taken to run the algorithms, and accuracy of the results obtained are presented for a 3 machine 9 bus system. Results indicate that SVM method is an optimal choice for fault classification with high accuracy for location prediction and low computational requirements compared with DNN.

Less than 10% of patients with cancer have detectable pathogenic germline alterations, which may be partially due to incomplete pathogenic variant detection. To evaluate if deep learning approaches identify more germline pathogenic variants in patients with cancer. A cross-sectional study of a standard germline detection method and a deep learning method in 2 convenience cohorts with prostate cancer and melanoma enrolled in the US and Europe between 2010 and 2017. The final date of clinical data collection was December 2017. Germline variant detection using standard or deep learning methods. The primary outcomes included pathogenic variant detection performance in 118 cancer-predisposition genes estimated as sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). The secondary outcomes were pathogenic variant detection performance in 59 genes deemed actionable by the American College of Medical Genetics and Genomics (ACMG) and 5197 clinically relevant mendelian genes. True sensitivity and true specificity could not be calculated due to lack of a criterion reference standard, but were estimated as the proportion of true-positive variants and true-negative variants, respectively, identified by each method in a reference variant set that consisted of all variants judged to be valid from either approach. The prostate cancer cohort included 1072 men (mean [SD] age at diagnosis, 63.7 [7.9] years; 857 [79.9%] with European ancestry) and the melanoma cohort included 1295 patients (mean [SD] age at diagnosis, 59.8 [15.6] years; 488 [37.7%] women; 1060 [81.9%] with European ancestry). The deep learning method identified more patients with pathogenic variants in cancer-predisposition genes than the standard method (prostate cancer: 198 vs 182; melanoma: 93 vs 74); sensitivity (prostate cancer: 94.7% vs 87.1% [difference, 7.6%; 95% CI, 2.2% to 13.1%]; melanoma: 74.4% vs 59.2% [difference, 15.2%; 95% CI, 3.7% to 26.7%]), specificity (prostate cancer: 64.0% vs 36.0% [difference, 28.0%; 95% CI, 1.4% to 54.6%]; melanoma: 63.4% vs 36.6% [difference, 26.8%; 95% CI, 17.6% to 35.9%]), PPV (prostate cancer: 95.7% vs 91.9% [difference, 3.8%; 95% CI, -1.0% to 8.4%]; melanoma: 54.4% vs 35.4% [difference, 19.0%; 95% CI, 9.1% to 28.9%]), and NPV (prostate cancer: 59.3% vs 25.0% [difference, 34.3%; 95% CI, 10.9% to 57.6%]; melanoma: 80.8% vs 60.5% [difference, 20.3%; 95% CI, 10.0% to 30.7%]). For the ACMG genes, the sensitivity of the 2 methods was not significantly different in the prostate cancer cohort (94.9% vs 90.6% [difference, 4.3%; 95% CI, -2.3% to 10.9%]), but the deep learning method had a higher sensitivity in the melanoma cohort (71.6% vs 53.7% [difference, 17.9%; 95% CI, 1.82% to 34.0%]). The deep learning method had higher sensitivity in the mendelian genes (prostate cancer: 99.7% vs 95.1% [difference, 4.6%; 95% CI, 3.0% to 6.3%]; melanoma: 91.7% vs 86.2% [difference, 5.5%; 95% CI, 2.2% to 8.8%]). Among a convenience sample of 2 independent cohorts of patients with prostate cancer and melanoma, germline genetic testing using deep learning, compared with the current standard genetic testing method, was associated with higher sensitivity and specificity for detection of pathogenic variants. Further research is needed to understand the relevance of these findings with regard to clinical outcomes.

Droughts are a major disaster in Africa, threatening livelihoods through their influence on crop yields but also by impacting and weakening ecosystems. Modeling the vegetation state can help anticipate and reduce the impact of droughts by predicting the vegetation response over time. Forecasting the state of vegetation is challenging: it depends on complex interactions between the plants and different environmental drivers, which can result in both instantaneous and time-lagged responses, as well as spatial effects. Furthermore, modeling these interactions at the fine resolution of landscape scale can only rely on remote sensing observations, as in-situ measurements are not global and weather models have a coarse grid. With the increasing availability of remote sensing data, deep learning methods are a promising avenue for these spatiotemporal tasks. Here, we introduce both a dataset and a baseline deep neural network, modeling the vegetation response to climate at landscape scale in Africa.EarthNet2021 [1] introduced leveraging self-supervised learning for satellite imagery forecasting based on coarse-scale weather in Europe. Here, we introduce EarthNet2023 with a more narrow focus on drought impacts in Africa. It contains over 45,000 Spatio-temporal minicubes (each 1.28x1.28km) at representative locations over the whole African continent. Alongside Sentinel-2 reflectance, ERA5 weather, and topography, it also contains Sentinel-1 backscatter, soil properties, and a long-term Normalized Difference Vegetation Index (NDVI) climatology based on Landsat. The latter allows evaluating models on vegetation anomalies, thereby including modeling of drought impacts. EarthNet2023 is intended as an open benchmark challenge, allowing multiple research groups to develop their approaches to drought impact modeling in Africa. As a baseline for EarthNet2023, we train a  Convolutional Long Short-Term Memory (ConvLSTM) deep learning model. Previous work has shown it is suitable for spatiotemporal satellite imagery forecasting [2, 3, 4]. The ConvLSTM baseline captures the seasonal evolution of NDVI over a wide range of vegetation types. General spatial patterns are well-captured as well as a first indication of skill during weather extremes is seen, although the accuracy of the predictions is inconsistent, and the confidence in the model is therefore too low. This suggests, with further development, deep learning approaches are promising for modeling vegetation evolution in Africa, potentially even up to the degree to support anticipatory action with drought impact modeling. [1] Requena-Mesa, C., Benson, V., Reichstein, M., Runge, J., & Denzler, J. (2021). EarthNet2021: A large-scale dataset and challenge for Earth surface forecasting as a guided video prediction task. In CVPR 2021 (pp. 1132-1142).[2] Diaconu, C. A., Saha, S., Günnemann, S., & Zhu, X. X. (2022). Understanding the Role of Weather Data for Earth Surface Forecasting Using a ConvLSTM-Based Model. In CVPR 2022 (pp. 1362-1371).[3] Kladny, K. R. W., Milanta, M., Mraz, O., Hufkens, K., & Stocker, B. D. (2022). Deep learning for satellite image forecasting of vegetation greenness. bioRxiv.[4] Robin, C., Requena-Mesa, C., Benson, V., Alonso, L., Poehls, J., Carvalhais, N., & Reichstein, M. (2022). Learning to forecast vegetation greenness at fine resolution over Africa with ConvLSTMs. In Tackling Climate Change with Machine Learning: workshop at NeurIPS 2022. 

Deep learning algorithms can be used to design systems that report data on patients and deliver warnings to medical applications or electronic health information if there are changes in the patient's health. These systems could be created using deep Learning. This helps verify that patients get the proper effective care at the proper time for each specific patient. A healthcare decision support system was presented using the Internet of Things and deep learning methods. In the proposed system, we examined the capability of integrating deep learning technology into automatic diagnosis and IoT capabilities for faster message exchange over the Internet. We have selected the suitable Neural Network structure (number of best-hidden layers and activation function classes) to construct the e-health system. In addition, the e-health system relied on data from doctors to understand the Neural Network. In the validation method, the total evaluation of the proposed healthcare system for diagnostics provides dependability under various patient conditions. Based on evaluation and simulation findings, a dual hidden layer of feed-forward NN and its neurons store the tanh function more effectively than other NN. To overcome challenges, this study will integrate artificial intelligence with IoT. This study aims to determine the NN's optimal layer counts and activation function variations.

With the increase of solar photovoltaic (PV) penetration in power system, the impact of random fluctuation of PV power on the secure operation of power grid becomes more and more serious. High-precision PV power forecasting can effectively promote the grid's accommodation of PV power generation. Cloud is the most important factor affecting the surface irradiance and PV power. For the ultra-short-term solar PV power forecast considering the influence of cloud movement, it is necessary to be able to obtain the surface irradiance according to the sky cloud observation data. Therefore, in order to accurately achieve the real-time mapping relationship between sky image and surface irradiance, a hybrid mapping model based on deep learning applied for solar PV power forecasting is proposed in this article. First, the sky image data are clustered based on the feature extraction of convolutional autoencoder and K -means clustering algorithm after preprocess stage. Second, a hybrid mapping model based on deep learning methods are established for surface irradiance. Finally, the simulation results are compared and evaluated with different deep learning methods (CNN, LSTM, and ANN). The results show that the proposed model in this article has higher accuracy and can maintain robustness under different weather conditions.

Learning ability is the basic characteristic of human intelligence. The July 1, 2005 issue of Science published a list of 125 important questions in science. Among them, the question 94 “What are the limits of learning by machines?”. The annotation “Computers can already beat the world’s best chess players, and they have a wealth of information on the Web to draw on. But abstract reasoning is still beyond any machine”. In recent artificial intelligence has made great progresses. In 1997, the rise of the man-machine war, IBM Supercomputer Deep Blue defeated the chess master Garry Kasparov. On February 14, 2011, IBM’s Watson supercomputer won a practice round against Jeopardy champions Ken Jennings and Brad Rutter. In March 2016, Google DeepMind’s AlphaGo sealed a 4-1 victory over a South Korean Go grandmaster Lee Se-dol. This paper focuses on the machine learning methods of AlphaGo, including reinforcement learning, deep learning, deep reinforcement learning, analysis of the existing problems and the latest research progress. Deep reinforcement learning is the combination of deep learning and reinforcement learning, which can realize the learning algorithm from the perception to action. Simply said, this is the same as human behavior, input sensing information such as vision, and then, direct output action through the deep neural network. Deep reinforcement learning has the potential to learn a variety of skills for the robot to achieve full autonomy. Even though reinforcement learning is practiced successfully, but feature states need to manually set, for complex scene is a difficult thing, especially easy to cause the dimension disaster, and expression is not good. In 2010, Sascha Lange and Martin Riedmiller proposed deep auto-encoder neural networks in reinforcement learning to extract feature, which is used to control the visual correlation. In 2013, DeepMind proposed deep Q-network (DQN) in NIPS 2013, using convolution neural network to extract features, and then applied in reinforcement learning. They continue to improve and published an improved version of DQN on Nature in 2015, which has aroused widespread concern. In order to break through the limits of learning by machines, cognitive machine learning is proposed, which is the combination of machine learning and brain cognition, so that the machine intelligence is constantly evolving, and gradually reaches the human level of artificial intelligence. A cognitive model entitled Consciousness And Memory (CAM) is proposed by author, which consists of memory, consciousness, high-level cognitive functions, perception and motor. High-level cognitive functions of the brain include learning, language, thinking, decision making, emotion, and so on. Learning is a course to accept the stimulus through the nervous system and obtain new behavior, habits and accumulation experience. According to the current research progress of brain science and cognitive science, cognitive machine learning may be interested in learning emergence, procedural memory knowledge learning, learning evolution and so on. For intelligence, so-called evolution is refers to the learning of learning and the structure also follows the change. It is important to record the learning result by structure changing and improve the learning method.

Automatic assessment of pain based on deep learning methods: A systematic review

The increasing dependency on electricity and demand for renewable energy sources means that distributed system operators face new challenges in their grid. Accurate forecasts of electric load can solve these challenges. In recent years deep neural networks have become increasingly popular in research, and researchers have carried out many experiments to create the most accurate deep learning models. Players in the energy sector can exploit the increasing amount of energy-related data collected from smart meters to improve the grid’s operating quality. This review investigates state-of-the-art methodologies relating to energy load forecasting using deep neural networks. A thorough literature search is conducted, which outlines and analyses essential aspects regarding deep learning load forecasts in the energy domain. The literature suggests two main perspectives: demand-side management and grid control on the supply side. Each perspective has multiple applications with its challenges to achieve accurate forecasts; households, buildings, and grids. This paper recommends using a hybrid deep learning multivariate model consisting of a convolutional and recurrent neural network based on the scoping review. The suggested input variables should be historical consumption, weather, and day features. Combining the convolutional and recurrent networks ensures that the model learns as many repeating patterns and features in the data as possible.