Deep patch learning algorithms with high interpretability for regression problems

Deep learning has gained immense popularity in scientific computing, and its algorithms are widely used in complex problem-solving industries. Every deep learning algorithm use different types of neural networks to perform indented tasks. Deep learning (DL) algorithms have emerged from different machine learning and soft computing methodologies. Since then, a number of deep learning (DL) algorithms have been recently introduced in the scientific community and applied in various application fields. Today, the use of DLs has become indispensable due to their intelligence, effective learning, accuracy and reliability in model creation. However, a comprehensive list of DL algorithms has not yet been presented in the scientific literature. This article lists the most popular DL algorithms and their application areas. Deep learning uses ANN artificial neural networks to perform convoluted calculations on huge amounts of data. It is a type of machine learning based on the structure and function of the human brain. Deep learning algorithms train machines by learning from examples. Industries such as healthcare, e-commerce, entertainment and advertising often use deep learning.

Background Machine learning (ML) algorithm support vector machine (SVM) performed better than Thrombolysis in Myocardial Infarction (TIMI) score for ASIAN STEMI patients. However, Deep Learning (DL) effectiveness in the multiethnic ASIAN population has yet to be determined. DL has automatic learning of the feature from a given dataset without the need to conduct feature selection. However, the selected features by the algorithm is black box. Identifying features associated with mortality is essential to recognize characteristics of patients with high risk for better patient management. Purpose To develop a DL algorithm for in-hospital mortality in multiethnic STEMI patients using predictors identified from the SVM algorithm. To investigate DL performance constructed using predictors from SVM feature extraction and expert-recommended predictors. Methods We constructed four algorithms; a) DL and SVM algorithms with predictors identified from the SVM variable importance b) DL and SVM using predictors based on expert recommendation. We used registry data from the National Cardiovascular Disease Database of 11397 patient's. Fifty parameters including demographics, cardiovascular risk, medications and clinical variables were considered. The Area under the curve (AUC) is the performance evaluation metric. Algorithms were validated against the TIMI and tested using the same validation data. SVM variable importance with backward elimination was used to select and rank important variables. Results DL algorithms outperform SVM and TIMI on the validation dataset; i) DL with SVM selected predictors (15 predictors, AUC = 0.97), ii) DL with expert-recommended predictors (16 predictors, AUC = 0.96), iii) SVM with selected predictors (15 predictors, AUC = 0.92), iv) SVM with expert-recommended predictors (AUC = 0.89) and TIMI (AUC = 0.82). Common predictors across SVM feature selection, expert-recommendation and TIMI are: age, heart rate, Killip class, fasting blood glucose, systolic blood pressure, comorbid diseases and ST-elevation. SVM feature selection also identified diuretics, PCI and pharmacotherapy drugs as predictors that improve mortality prediction in STEMI patients. Our findings suggest that the TIMI score underestimates patients risk of mortality. DL algorithm using selected predictors classified 35% of nonsurvival patients as high risk (risk probabilities >50%) compared to only 12.7% nonsurvival patients by TIMI (score >5) (Figure below). Conclusions In the ASIAN population, patients with STEMI can be better classified using the DL algorithm compared to the ML and TIMI score. Combining ML feature selection with DL allows the identification of distinct factors in a unique ASIAN population for better mortality prediction than relying solely on an expert recommendation as it is a very subjective approach. Continuous validation on population-specific algorithms using DL and ML is needed before implementing in a real clinical setting. Funding Acknowledgement Type of funding sources: Public grant(s) – National budget only. Main funding source(s): Technology Development Fund 1 TIMI performance on validation setDL performance on validation set

Feature mining for encrypted malicious traffic detection with deep learning and other machine learning algorithms

BACKGROUNDTraditional methods of developing predictive models in inflammatory bowel diseases (IBD) rely on using statistical regression approaches to deriving clinical scores such as the Crohn's disease (CD) activity index. However, traditional approaches are unable to take advantage of more complex data structures such as repeated measurements. Deep learning methods have the potential ability to automatically find and learn complex, hidden relationships between predictive markers and outcomes, but their application to clinical prediction in CD and IBD has not been explored previously.AIMTo determine and compare the utility of deep learning with conventional algorithms in predicting response to anti-tumor necrosis factor (anti-TNF) therapy in CD.METHODSThis was a retrospective single-center cohort study of all CD patients who commenced anti-TNF therapy (either adalimumab or infliximab) from January 1, 2010 to December 31, 2015. Remission was defined as a C-reactive protein (CRP) < 5 mg/L at 12 mo after anti-TNF commencement. Three supervised learning algorithms were compared: (1) A conventional statistical learning algorithm using multivariable logistic regression on baseline data only; (2) A deep learning algorithm using a feed-forward artificial neural network on baseline data only; and (3) A deep learning algorithm using a recurrent neural network on repeated data. Predictive performance was assessed using area under the receiver operator characteristic curve (AUC) after 10× repeated 5-fold cross-validation.RESULTSA total of 146 patients were included (median age 36 years, 48% male). Concomitant therapy at anti-TNF commencement included thiopurines (68%), methotrexate (18%), corticosteroids (44%) and aminosalicylates (33%). After 12 mo, 64% had CRP < 5 mg/L. The conventional learning algorithm selected the following baseline variables for the predictive model: Complex disease behavior, albumin, monocytes, lymphocytes, mean corpuscular hemoglobin concentration and gamma-glutamyl transferase, and had a cross-validated AUC of 0.659, 95% confidence interval (CI): 0.562-0.756. A feed-forward artificial neural network using only baseline data demonstrated an AUC of 0.710 (95%CI: 0.622-0.799; P = 0.25 vs conventional). A recurrent neural network using repeated biomarker measurements demonstrated significantly higher AUC compared to the conventional algorithm (0.754, 95%CI: 0.674-0.834; P = 0.036).CONCLUSIONDeep learning methods are feasible and have the potential for stronger predictive performance compared to conventional model building methods when applied to predicting remission after anti-TNF therapy in CD.

BackgroundMachine learning (ML) algorithm support vector machine (SVM) performed better than Thrombolysis in Myocardial Infarction (TIMI) score for ASIAN STEMI patients. However, Deep Learning (DL) effectiveness in the multiethnic ASIAN population has yet to be determined. DL has automatic learning of the feature from a given dataset without the need to conduct feature selection. However, the selected features by the algorithm is black box. Identifying features associated with mortality is essential to recognize characteristics of patients with high risk for better patient management.PurposeTo develop a DL algorithm for in-hospital mortality in multiethnic STEMI patients using predictors identified from the SVM algorithm. To investigate DL performance constructed using predictors from SVM feature extraction and expert-recommended predictors.MethodsWe constructed four algorithms; a) DL and SVM algorithms with predictors identified from the SVM variable importance b) DL and SVM using predictors based on expert recommendation. We used registry data from the National Cardiovascular Disease Database of 11397 patient's. Fifty parameters including demographics, cardiovascular risk, medications and clinical variables were considered. The Area under the curve (AUC) is the performance evaluation metric. Algorithms were validated against the TIMI and tested using the same validation data. SVM variable importance with backward elimination was used to select and rank important variables.ResultsDL algorithms outperform SVM and TIMI on the validation dataset; i) DL with SVM selected predictors (15 predictors, AUC = 0.97), ii) DL with expert-recommended predictors (16 predictors, AUC = 0.96), iii) SVM with selected predictors (15 predictors, AUC = 0.92), iv) SVM with expert-recommended predictors (AUC = 0.89) and TIMI (AUC = 0.82). Common predictors across SVM feature selection, expert-recommendation and TIMI are: age, heart rate, Killip class, fasting blood glucose, systolic blood pressure, comorbid diseases and ST-elevation. SVM feature selection also identified diuretics, PCI and pharmacotherapy drugs as predictors that improve mortality prediction in STEMI patients. Our findings suggest that the TIMI score underestimates patients risk of mortality. DL algorithm using selected predictors classified 35% of nonsurvival patients as high risk (risk probabilities >50%) compared to only 12.7% nonsurvival patients by TIMI (score >5) (Figure below).ConclusionsIn the ASIAN population, patients with STEMI can be better classified using the DL algorithm compared to the ML and TIMI score. Combining ML feature selection with DL allows the identification of distinct factors in a unique ASIAN population for better mortality prediction than relying solely on an expert recommendation as it is a very subjective approach. Continuous validation on population-specific algorithms using DL and ML is needed before implementing in a real clinical setting.Funding AcknowledgementType of funding sources: Public grant(s) – National budget only. Main funding source(s): Technology Development Fund 1TIMI performance on validation setDL performance on validation set

Recent technological advancements allow deep learning to be employed in practically every aspect of life. Because deep learning techniques are so precise, they can be used in medicine to classify and detect various diseases. The coronavirus (SARSCoV2) epidemic has recently affected global health systems. One of the ways to diagnose SARSCoV2 is with a chest X-ray. This paper proposes a deep learning technique to distinguish SARSCoV2 positive and normal cases. In this study, we fine-tuned the deep learning models and hyperparameters, and the fine-tuned deep learning models performed significantly better. To classify X-ray images, we developed a system based on deep learning algorithms that includes five models: Xception, VGG19, ResNet50, DenseNet121, and Inception. We offer deep learning models and algorithms that have been trained and evaluated to support medical efforts and reduce medical staff workload when dealing with SARSCoV2. In addition, the classification model that was proposed yields positive results because it makes use of accurate classification of the SARSCoV2 disease based on medical images. Additionally, the performance of our proposed CNN classification method for medical imaging was evaluated using various edge-based neural networks. The accuracy of tertiary classification with CNN will decrease as the number of classes in the training network grows. In tertiary classification, which includes normal and SARSCoV2 positive images, the proposed model achieved a 0.9897 accuracy. The proposed algorithm achieves a high level of classification accuracy when using the DenseNet121 model for binary classification.

Wind energy has the possibility for bringing out energy in a very constant and sustainable manner being a notable and eligible source. Although, wind energy does include numerous challenges like, the halted asset of wind plants, early investment costs, and therefore, the strain in discovering areas of wind efficiency. The major objective for proposing this work is to determine the power efficiency of wind turbines, which will also aid in the formulation of a proposal to reduce wind turbine maintenance costs. During this research, data analysis of turbine generators is performed on day to day wind speed info using machine learning and deep learning algorithms. A way is put forward by us to support deep learning and machine learning algorithms which can predict different values of power reliably. Hence, the execution of machine and deep learning algorithms are analyzed. For forecasting for a longer term these algorithms may be used for wind generation rate with historical relation to wind speed info. Index Terms: Wind turbine,machine learning algorithm

In recent years, the widespread utilization of biological data processing technology has been driven by its cost-effectiveness. Consequently, next-generation sequencing (NGS) has become an integral component of biological research. NGS technologies enable the sequencing of billions of nucleotides in the entire genome, transcriptome, or specific target regions. This sequencing generates vast data matrices. Consequently, there is a growing demand for deep learning (DL) approaches, which employ multilayer artificial neural networks and systems capable of extracting meaningful information from these extensive data structures. In this study, the aim was to obtain optimized parameters and assess the prediction performance of deep learning and machine learning (ML) algorithms for binary classification in real and simulated whole genome data using a cloud-based system. The ART-simulated data and paired-end NGS (whole genome) data of Ch22, which includes ethnicity information, were evaluated using XGBoost, LightGBM, and DL algorithms. When the learning rate was set to 0.01 and 0.001, and the epoch values were updated to 500, 1000, and 2000 in the deep learning model for the ART simulated dataset, the median accuracy values of the ART models were as follows: 0.6320, 0.6800, and 0.7340 for epoch 0.01; and 0.6920, 0.7220, and 0.8020 for epoch 0.001, respectively. In comparison, the median accuracy values of the XGBoost and LightGBM models were 0.6990 and 0.6250 respectively. When the same process is repeated for Chr 22, the results are as follows: the median accuracy values of the DL models were 0.5290, 0.5420 and 0.5820 for epoch 0.01; and 0.5510, 0.5830 and 0.6040 for epoch 0.001, respectively. Additionally, the median accuracy values of the XGBoost and LightGBM models were 0.5760 and 0.5250, respectively. While the best classification estimates were obtained at 2000 epochs and a learning rate (LR) value of 0.001 for both real and simulated data, the XGBoost algorithm showed higher performance when the epoch value was 500 and the LR was 0.01. When dealing with class imbalance, the DL algorithm yielded similar and high Recall and Precision values. Conclusively, this study serves as a timely resource for genomic scientists, providing guidance on why, when, and how to effectively utilize deep learning/machine learning methods for the analysis of human genomic data.

Skin cancer detection by deep learning and sound analysis algorithms: A prospective clinical study of an elementary dermoscope

Deep Learning (DL) algorithms have become the de facto Machine Learning (ML) algorithm for large scale data analysis. DL algorithms are computationally expensive -- even distributed DL implementations which use MPI require days of training (model learning) time on commonly studied datasets. Long running DL applications become susceptible to faults -- requiring development of a fault tolerant system infrastructure, in addition to fault tolerant DL algorithms. This raises an important question: What is needed from MPI for designing fault tolerant DL implementations? In this paper, we address this problem for permanent faults. We motivate the need for a fault tolerant MPI specification by an in-depth consideration of recent innovations in DL algorithms and their properties, which drive the need for specific fault tolerance features. We present an in-depth discussion on the suitability of different parallelism types (model, data and hybrid); a need (or lack thereof) for check-pointing of any critical data structures; and most importantly, consideration for several fault tolerance proposals (user-level fault mitigation (ULFM), Reinit) in MPI and their applicability to fault tolerant DL implementations. We leverage a distributed memory implementation of Caffe, currently available under the Machine Learning Toolkit for Extreme Scale (MaTEx). We implement our approaches by extending MaTEx-Caffe for using ULFM-based implementation. Our evaluation using the ImageNet dataset and AlexNet, and GoogLeNet neural network topologies demonstrates the effectiveness of the proposed fault tolerant DL implementation using OpenMPI based ULFM.

The aim of this study is to develop a predictive model utilizing deep learning and machine learning techniques that will inform clinical decision-making by predicting the 1-year postoperative recovery of patients with lumbar disk herniation. The clinical data of 470 inpatients who underwent tubular microdiscectomy (TMD) between January 2018 and January 2021 were retrospectively analyzed as variables. The dataset was randomly divided into a training set (n = 329) and a test set (n = 141) using a 10-fold cross-validation technique. Various deep learning and machine learning algorithms including Random Forests, Extreme Gradient Boosting, Support Vector Machines, Extra Trees, K-Nearest Neighbors, Logistic Regression, Light Gradient Boosting Machine, and MLP (Artificial Neural Networks) were employed to develop predictive models for the recovery of patients with lumbar disk herniation 1 year after surgery. The cure rate score of lumbar JOA score 1 year after TMD was used as an outcome indicator. The primary evaluation metric was the area under the receiver operating characteristic curve (AUC), with additional measures including decision curve analysis (DCA), accuracy, sensitivity, specificity, and others. The heat map of the correlation matrix revealed low inter-feature correlation. The predictive model employing both machine learning and deep learning algorithms was constructed using 15 variables after feature engineering. Among the eight algorithms utilized, the MLP algorithm demonstrated the best performance. Our study findings demonstrate that the MLP algorithm provides superior predictive performance for the recovery of patients with lumbar disk herniation 1 year after surgery.

In recent years, differential privacy technology has been introduced into deep learning to satisfy the protection of personal privacy data. However, the existing differential privacy technology will weaken the generalization ability of deep learning networks. To address the problem, a differential privacy noise Bayesian gradient descent algorithm (Dif-NoBa) in Deep Learning was proposed. For the method, the privacy protection algorithm acts on a single neuron in the neural network. Therefore, no matter how complex the network is, it only focuses on a single neuron, so that each neuron can satisfy its own differential privacy, and then the whole network can meet the differential privacy, effectively solving the problem of insufficient generalization ability of neural network caused by adding arbitrary noise.

Introduction: The diagnosis of myocardial infarction (MI) needs to be swift and accurate, but definitively diagnosing it based on the first test encountered in clinical practice, the electrocardiogram (ECG), is not an easy task. The purpose of the study was to develop a deep learning (DL) algorithm using multitask learning method to differentiate patients experiencing MI from those without coronary artery disease using image-based ECG data. Methods: A DL model was developed based on 11,227 ECG images. We developed a new ECG interpretation algorithm through signal-guided multitask learning, building on a previously published single-task algorithm. The utility of this model was evaluated by testing 51 physicians in interpreting ECGs with and without the assistance of the DL model. Results: The proposed model demonstrated superior performance, achieving 90.56% accuracy, 83.82% sensitivity, 93.02% specificity, 81.44% precision, and an F1 score of 82.61% in discriminating MI ECG. Overall, the median accuracy of ECG interpretation improved from 62% to 68% with the DL algorithm. Trainees and specialists in internal medicine experienced significant accuracy increases (60–66% for trainees, 72–80% for specialists). In the MI group, NSTEMI accuracy was notably lower than STEMI (33% vs. 80%, p < 0.001), but the DL algorithm improved interpretative capabilities in both NSTEMI and STEMI. Conclusions: Signal-guided multitask DL algorithm demonstrated superior performance compared with previous single-task algorithm. The DL algorithm supports the physicians’ decision discriminating MI ECGs from non-MI ECGs. The improvement was consistent in subgroups of STEMI and NSTEMI.

BackgroundDeep learning (DL) based solutions have been proposed for interpretation of several imaging modalities including radiography, CT, and MR. For chest radiographs, DL algorithms have found success in the evaluation of abnormalities such as lung nodules, pulmonary tuberculosis, cystic fibrosis, pneumoconiosis, and location of peripherally inserted central catheters. Chest radiography represents the most commonly performed radiological test for a multitude of non-emergent and emergent clinical indications. This study aims to assess accuracy of deep learning (DL) algorithm for detection of abnormalities on routine frontal chest radiographs (CXR), and assessment of stability or change in findings over serial radiographs.Methods and findingsWe processed 874 de-identified frontal CXR from 724 adult patients (> 18 years) with DL (Qure AI). Scores and prediction statistics from DL were generated and recorded for the presence of pulmonary opacities, pleural effusions, hilar prominence, and enlarged cardiac silhouette. To establish a standard of reference (SOR), two thoracic radiologists assessed all CXR for these abnormalities. Four other radiologists (test radiologists), unaware of SOR and DL findings, independently assessed the presence of radiographic abnormalities. A total 724 radiographs were assessed for detection of findings. A subset of 150 radiographs with follow up examinations was used to asses change over time. Data were analyzed with receiver operating characteristics analyses and post-hoc power analysis.ResultsAbout 42% (305/ 724) CXR had no findings according to SOR; single and multiple abnormalities were seen in 23% (168/724) and 35% (251/724) of CXR. There was no statistical difference between DL and SOR for all abnormalities (p = 0.2–0.8). The area under the curve (AUC) for DL and test radiologists ranged between 0.837–0.929 and 0.693–0.923, respectively. DL had lowest AUC (0.758) for assessing changes in pulmonary opacities over follow up CXR. Presence of chest wall implanted devices negatively affected the accuracy of DL algorithm for evaluation of pulmonary and hilar abnormalities.ConclusionsDL algorithm can aid in interpretation of CXR findings and their stability over follow up CXR. However, in its present version, it is unlikely to replace radiologists due to its limited specificity for categorizing specific findings.

In this paper, we study the reliability of a novel deep learning framework for internal gross target volume (IGTV) delineation from 4-D computed tomography (4DCT), which is applied to patients with lung cancer treated by stereotactic body radiation therapy (SBRT). Seventy seven patients who underwent SBRT followed by 4DCT scans were incorporated in this retrospective study. The IGTV_DL was delineated using a novel deep machine learning algorithm with a linear exhaustive optimal combination framework. For the purpose of comparison, three other IGTVs based on common methods was also delineated. We compared the relative volume difference (RVI), matching index (MI), and encompassment index (EI) for the above IGTVs. Then, multiple parameter regression analysis was performed to assess the tumor volume and motion range as clinical influencing factors in the MI variation. The results demonstrated that the deep learning algorithm with linear exhaustive optimal combination framework has a higher probability of achieving optimal MI compared with other currently widely used methods. For patients after simple breathing training by keeping the respiratory frequency in 10 breath per minute (BPM), the four phase combinations of 0%, 30%, 50% and 90% can be considered as a potential solution for an optimal combination to synthesize IGTV in all respiration amplitudes.