Improving stroke diagnosis accuracy using hyperparameter optimized deep learning

The availability of data is an important thing in the decision-making process. With the trend of using Machine Learning and Data Mining, humans are facilitated in learning and finding hidden knowledge in the data. One technique used in the classification of data is the Deep Learning method. Deep Learning is widely used because of its ability to solve problems with satisfying results. In other words, Deep Learning has shown to be the one of the most powerful and widely used methods. However, the key of success depends on the setting the right parameters. With this reason, we need a way to optimize the use of hyperparameter in Deep Learning. The method commonly used in this case is Grid Search. However, this method requires a long time and high computation. While other methods such as Random Search and Bayesian Optimization, both are very reliable and efficient compared to Grid Search. The results of the experiments conducted in this study indicate that Bayesian Optimization is better than Random Search in performing hyperparameter optimization processes in Deep Learning.

Relationship of CT Radiomics and Dose Texture to Radiation-Induced Swallow Dysfunction in Head and Neck Cancer

This study explores the potential of the GPBoost approach for groundwater quality assessment in comparison to three other gradient boosting-based algorithms. Three methods, random search, grid search, and Bayesian optimization were used to find the optimal values of various hyperparameters with all four-gradient boosting-based algorithms. One hundred and two samples of Entropy weighted water quality index with 14 input parameters are used for assessing groundwater quality. The calculated EWQI values for drinking range between 80.4 and 394.96 in pre-monsoon and 39.6 to 338.79 during the post-monsoon period. Moreover, spatial distribution maps displayed that the central portions of the study area fall under medium water quality. The performances of models were compared based on multiple statistical criteria, including Correlation Coefficient (CC), root mean square error (RMSE), and mean absolute error (MAE). The results reveal that the CC value by all modeling approaches is more than 0.93, suggesting a comparable performance by all methods. Results in terms of RMSE values in predicting the EWQI values suggest GPBoost (random search) model performed better than the other three models, thus suggesting a competitive performance by GPBoost in comparison to other gradient boosting-based approaches. Relative importance analysis provided by random and grid search methods highlights the significance of NO3 −, Mg2+, TDS, EC, and TH as important input parameters for predicting EWQI.

ABSTRACTLung cancer causes a high mortality rate in the world than any other cancers. That can be minimised if the symptoms and cancer cells have been detected early. One of the techniques used to detect lung cancer is by computed tomography (CT) scan. CT scan images have been used in this study to identify one of the lesion characteristics named ground glass opacity (GGO). It has been used to determine the level of malignancy of the lesion. There were three phases in identifying GGO: image cropping, feature extraction using grey level co-occurrence matrices (GLCM) and classification using Naïve Bayes Classifier. In order to improve the classification results, the most significant feature was sought by feature selection using gain ratio evaluation. Based on the results obtained, the most significant features could be identified by using feature selection method used in this research. The accuracy rate increased from 83.33% to 91.67%, the sensitivity from 82.35% to 94.11% and the specificity from 84.21% to 89.47%.

COVID-19 is known in recent times as a severe syndrome of respiratory organ (Lungs) and has gradually produced pneumonia, a lung disorder all around the world. As coronavirus is continually spreading rapidly globally, the computed tomography (CT) technique has been made important and essential for quick diagnosis of this dangerous syndrome. Hence, it is necessitated to develop a precise computer-based technique for assisting medical clinicians in identifying the COVID-19 influenced patients with the help of CT scan images. Therefore, the multilayer perceptron neural networks optimized with Garra Rufa Fish optimization using images of CT scan is proposed in this paper for the classification of COVID-19 patients (COV-19-MPNN-GRF-CTI). The input images are taken from SARS-COV-2 CT-scan dataset. Initially, the input images are pre-processed utilizing convolutional auto-encoder (CAE) to enhance the quality of the input images by eliminating noises. The pre-processed images are fed to Residual Network (ResNet-50) for extracting the global and statistical features. The extraction over the features of CT scan images is made through ResNet-50 and subsequently input to multilayer perceptron neural networks (MPNN) for CT images classification as COVID-19 and Non-COVID-19 patients. Here, the layer of Batch Normalization of the MPNN is separated and added with ResNet-50 layer. Generally, MPNN classifier does not divulge any adoption of optimization approach for calculating the optimal parameters and accurately classifying the extracted features of CT images. The Garra Rufa Fish (GRF) optimization algorithm performs to optimize the weight parameters of MPNN classifiers. The proposed approach is executed in MATLAB. The performance metrics, such as sensitivity, precision, specificity, [Formula: see text]-measure, accuracy and error rate, are examined. Then the performance of the proposed COV-19-MPNN-GRF-CTI method provides 22.08%, 24.03%, 34.76% higher accuracy, 23.34%, 26.45%, 34.44% higher precision, 33.98%, 21.95%, 34.78% lower error rate compared with the existing methods, like multi-task deep learning using CT image analysis for COVID-19 pneumonia classification and segmentation (COV-19-MDP-CTI), COVID-19 classification utilizing CT scan depending on meta-classifier approach (COV-19-SEMC-CTI) and deep learning-based COVID-19 prediction utilizing CT scan images (COV-19-CNN-CTI), respectively.

Background and Objectives: Brain Tumor Fusion-based Segments and Classification-Non-enhancing tumor (BTFSC-Net) is a hybrid system for classifying brain tumors that combine medical image fusion, segmentation, feature extraction, and classification procedures. Materials and Methods: to reduce noise from medical images, the hybrid probabilistic wiener filter (HPWF) is first applied as a preprocessing step. Then, to combine robust edge analysis (REA) properties in magnetic resonance imaging (MRI) and computed tomography (CT) medical images, a fusion network based on deep learning convolutional neural networks (DLCNN) is developed. Here, the brain images’ slopes and borders are detected using REA. To separate the sick region from the color image, adaptive fuzzy c-means integrated k-means (HFCMIK) clustering is then implemented. To extract hybrid features from the fused image, low-level features based on the redundant discrete wavelet transform (RDWT), empirical color features, and texture characteristics based on the gray-level cooccurrence matrix (GLCM) are also used. Finally, to distinguish between benign and malignant tumors, a deep learning probabilistic neural network (DLPNN) is deployed. Results: according to the findings, the suggested BTFSC-Net model performed better than more traditional preprocessing, fusion, segmentation, and classification techniques. Additionally, 99.21% segmentation accuracy and 99.46% classification accuracy were reached using the proposed BTFSC-Net model. Conclusions: earlier approaches have not performed as well as our presented method for image fusion, segmentation, feature extraction, classification operations, and brain tumor classification. These results illustrate that the designed approach performed more effectively in terms of enhanced quantitative evaluation with better accuracy as well as visual performance.

Hyper-Parameter Optimization (HPO) occupies a fundamental role in Deep Learning systems due to the number of hyper-parameters (HPs) to be set. The state-of-the-art of HPO methods are Grid Search, Random Search and Bayesian Optimization. The first two methods try all possible combinations and random combination of the HPs values, respectively. This is performed in a blind manner, without any information for choosing the new set of HPs values. Bayesian Optimization (BO), instead, keeps track of past results and uses them to build a probabilistic model mapping HPs into a probability density of the objective function. Bayesian Optimization builds a surrogate probabilistic model of the objective function, finds the HPs values that perform best on the surrogate model and updates it with new results. In this paper, we improve BO applied to Deep Neural Network (DNN) by adding an analysis of the results of the network on training and validation sets. This analysis is performed by exploiting rule-based programming, and in particular by using Probabilistic Logic Programming. The resulting system, called Symbolic DNN-Tuner, logically evaluates the results obtained from the training and the validation phase and, by applying symbolic tuning rules, fixes the network architecture, and its HPs, therefore improving performance. We also show the effectiveness of the proposed approach, by an experimental evaluation on literature and real-life datasets.

Optimizing hyperparameters is crucial for enhancing the performance of deep learning (DL) models. The process of configuring optimal hyperparameters, known as hyperparameter tuning, can be performed using various methods. Traditional approaches like grid search and random search have significant limitations. In contrast, Bayesian optimization (BO) utilizes a surrogate model and an acquisition function to intelligently navigate the hyperparameter space, aiming to provide deeper insights into performance disparities between naïve and advanced methods. This study evaluates BO's efficacy compared to baseline methods such as random search, manual search, and grid search across multiple DL architectures, including multi-layer perceptron (MLP), convolutional neural network (CNN), and LeNet, applied to the Modified National Institute of Standards and Technology (MNIST) and CIFAR-10 datasets. The findings indicate that BO, employing the tree-structured parzen estimator (TPE) search method and expected improvement (EI) acquisition function, surpasses alternative methods in intricate DL architectures such as LeNet and CNN. However, grid search shows superior performance in smaller DL architectures like MLP. This study also adopts a multi-objective (MO) perspective, balancing conflicting performance objectives such as accuracy, F1 score, and model size (parameter count). This MO assessment offers a comprehensive understanding of how these performance metrics interact and influence each other, leading to more informed hyperparameter tuning decisions.

Machine learning pipeline potentially consists of several stages of operations like data preprocessing, feature engineering and machine learning model training. Each operation has a set of hyper-parameters, which can become irrelevant for the pipeline when the operation is not selected. This gives rise to a hierarchical conditional hyper-parameter space. To optimize this mixed continuous and discrete conditional hierarchical hyper-parameter space, we propose an efficient pipeline search and configuration algorithm which combines the power of Reinforcement Learning and Bayesian Optimization. Empirical results show that our method performs favorably compared to state of the art methods like Auto-sklearn, TPOT, Tree Parzen Window, and Random Search.

Emerging as one of the most contemporary machine learning techniques, deep learning has shown success in areas such as image classification, speech recognition, and even playing games through the use of hierarchical architecture which includes many layers of non-linear information. In this paper, a powerful deep learning pipeline, intelligent deep learning (iDeepLe) is proposed for both regression and classification tasks. iDeepLe is written in Python with the help of various API libraries such as Keras, TensorFlow, and Scikit-Learn. The core idea of the pipeline is inspired by the sequential modeling with considering numerous layers of neurons to build the deep architecture. Each layer in the sequential deep model can perform independently as a module with minimum finitudes and does not limit the performance of the other layers. iDeepLe has the ability of employing grid search, random search, and Bayesian optimization to tune the most significant predictor input variables and hyper-parameters in the deep model via adaptive learning rate optimization algorithms for both accuracy and complexity, while simultaneously solving the unknown parameters of the regression or the classification model. The parallel pipeline of iDeepLe has the capacity to handle big data problems using Apache Spark, Apache Arrow, High Performance Computing (HPC) and GPU-enabled machines as well. In this paper, to show the importance of the optimization in deep learning, an exhaustive study of the impact of hyper-parameters in a simple and a deep model using optimization algorithms with adaptive learning rate was carried out.

Multichannel surface electromyography-based hand gesture recognition plays an important role in the development of intuitive prosthetic and human-machine interfaces. It requires high accuracy, minimal noise degradation, and electrode drift tolerance for real-world deployment. The goal is to find the best tuned classifier configurations that balance accuracy and real-world robustness, comparing grid search, random search, and Bayesian optimization for the four classical classifiers: K-Nearest Neighbors (KNN), Decision Tree (DT), Random Forest (RF), and Support Vector Machine (SVM). Their performance is evaluated by repeated stratified 5 × 5 cross-validation with 25 runs and leave-one-subject-out testing. Furthermore, we use a multi-objective Pareto front analysis considering three factors: (1) accuracy, (2) adaptation to simulated drift, and (3) relative accuracy degradation under noise. KNN reaches stable 98.4% accuracy on all optimizers, but Bayesian optimization performs better than other optimization techniques in terms of accuracy and other performance metrics when using SVM, DT, and RF classifiers. Additionally, Bayesian optimization gets KNN into the Pareto front with 98.44% accuracy, 98.9% adaptability, and a minimal 0.10% decrease in noise, which likewise clearly improves SVM and RF. These results provide guidance about how to choose effective, real-time electromyography classifiers not only for accuracy but also through trade-offs of accuracy, adaptability, and noise tolerance in prosthetic and rehabilitation systems.

We aimed to evaluate a computer-aided diagnosis (CADx) system for lung nodule classification focussing on (i) usefulness of the conventional CADx system (hand-crafted imaging feature + machine learning algorithm), (ii) comparison between support vector machine (SVM) and gradient tree boosting (XGBoost) as machine learning algorithms, and (iii) effectiveness of parameter optimization using Bayesian optimization and random search. Data on 99 lung nodules (62 lung cancers and 37 benign lung nodules) were included from public databases of CT images. A variant of the local binary pattern was used for calculating a feature vector. SVM or XGBoost was trained using the feature vector and its corresponding label. Tree Parzen Estimator (TPE) was used as Bayesian optimization for parameters of SVM and XGBoost. Random search was done for comparison with TPE. Leave-one-out cross-validation was used for optimizing and evaluating the performance of our CADx system. Performance was evaluated using area under the curve (AUC) of receiver operating characteristic analysis. AUC was calculated 10 times, and its average was obtained. The best averaged AUC of SVM and XGBoost was 0.850 and 0.896, respectively; both were obtained using TPE. XGBoost was generally superior to SVM. Optimal parameters for achieving high AUC were obtained with fewer numbers of trials when using TPE, compared with random search. Bayesian optimization of SVM and XGBoost parameters was more efficient than random search. Based on observer study, AUC values of two board-certified radiologists were 0.898 and 0.822. The results show that diagnostic accuracy of our CADx system was comparable to that of radiologists with respect to classifying lung nodules.

User reviews on Google Play Store reflect satisfaction and expectations regarding digital services, including E-Government applications. This study aims to optimize IndoBERT performance in sentiment classification through fine-tuning and hyperparameter exploration using three methods: Grid Search, Random Search, and Bayesian Optimization. Experiments were conducted on Sinaga Mobile app reviews, evaluated using accuracy, precision, recall, F1-score, learning curve, and confusion matrix. The results show that Grid Search with a learning rate of 5e-5 and a batch size of 16 provides the best results, with an accuracy of 90.55%, precision of 91.16%, recall of 90.55%, and F1-score of 89.75%. The learning curve indicates stable training without overfitting. This study provides practical contributions as a guide for improving IndoBERT in Indonesian sentiment analysis and as a foundation for developing NLP-based review monitoring systems to enhance public digital services.

Aim/Purpose: Traditional hyperparameter tuning methods’ inefficiency and low accuracy in diabetes prediction. Application and evaluation of Bayesian optimization aim to increase the accuracy of diabetes prediction models. Background: Bayesian optimization (BO) has emerged as a powerful technique for hyperparameter tuning in machine learning models with its methodical approach for optimizing complex functions with expensive evaluations. Selecting the optimal hyperparameters for models can significantly increase prediction accuracy in the healthcare sector, particularly for diabetes prediction, hence improving patient outcomes and resource management. Methodology: Among the well-known diabetes databases used in the study is the Pima Indian Diabetes Database. Among the machine learning models developed and whose hyperparameters are modified via Bayesian optimization are Support Vector Machines (SVM), Random Forest, and Gradient Boosting Machines (GBM). The optimization process is compared with traditional tuning methods to assess improvements in model performance. Contribution: Problems include the increasing prevalence of diabetes worldwide and the role vital function prediction models play in early diagnosis and therapy. Especially when dealing with enormous datasets common in healthcare, grid search, and random search are sometimes computationally taxing and inefficient. However, Bayesian optimization promises a more practical and economical approach by selecting hyperparameters iteratively based on earlier evaluations. Findings: Bayesian optimization not only yields faster calculation times but also outperforms traditional methods. More precisely, models adapted by Bayesian optimization show greater sensitivity and specificity, which are crucial for accurate and prompt diabetes diagnosis. Recommendation for Researchers: By highlighting the specific elements and network characteristics that are behind the anomalies it discovers, on a real-time or large scale dataset. Future Research: This work can be improvised using several recent artificial intelligence algorithms with the integration of IoT on real-time datasets.

We established prediction models based on the combination of spectral and different advanced image features to improve the prediction accuracy of solid-soluble content (SSC) of apple. Eight optimal wavelengths were selected using a new variable selection method called variable combination population analysis (VCPA). Image textural features of the first three principal component score images were obtained using a gray level co-occurrence matrix (GLCM) and a local binary pattern (LBP). Next, a random frog algorithm was developed to select optimal textural features for further analysis. A support vector regression (SVR) model based on spectral and different textural features was developed to predict the SSC of the apple. The model based on eight optimal wavelengths and nine optimal GLCM features of principal component images yielded the best result with the determination coefficient for prediction (Rp2) of 0.9193, root mean square error for prediction (RMSEP) of 0.2955, and the ratio of the standard deviation of the prediction set to the root mean square error of prediction (RPD) with a value of 3.50. These results revealed that the spectral combined with optimal GLCM features from principal component images coupled with the SVR model has the potential for prediction of the SSC of apple.