Development of a Diagnostic Model for Pancreatic Ductal Adenocarcinoma Using Nature-Inspired Optimization Algorithm and Machine Learning Techniques

Application of optimized machine learning techniques for prediction of occupational accidents

Abstract: Accurate prediction of maximum scour depth (MSD) at sluice gates is critical for guaranteeing the stability and safety of hydraulic systems. Traditional empirical formulas often fail to capture the non-linear interactions between flow dynamics, sediment characteristics, and structural configurations. This study addresses these limitations by leveraging advanced machine learning (ML) techniques, specifically tree-based models, to enhance predictive accuracy. The performance of three tree-based models–Extra Trees (ERT), CatBoost (CAT), and Histogram-Based Gradient Boosting (HGB)–was examined using two hyperparameter tuning methods: Tree-Structured Parzen Estimator (TPE) and Particle Swarm Optimization (PSO). The models underwent 100 simulations to quantify uncertainty and variability in performance metrics. The results indicate that CAT_PSO (optimized with PSO) exhibits superior predictive performance compared to empirical formulas and other ML techniques. CAT_PSO achieved the highest mean CORR (correlation coefficient) of 0.9644 and mean NSE (Nash-Sutcliffe Efficiency) of 0.9272. HGB models demonstrated slightly lower performance compared to CAT and ERT, with higher variability in predictions. Further analysis explored the influence of individual input factors on model performance. The inclusion of more variables, such as tailwater depth and sediment size, generally enhanced model performance. The study also developed new empirical equations for MSD estimation by considering both multiplicative and additive models, progressively incorporating additional input features. These new formulas show improved predictive accuracy over empirical methods, though they still fall short of the performance achieved by the ML models.

The number of deaths caused by alcohol-related diseases may be reduced by predicting alcohol use disorder (AUD). Many researchers have worked on AUD prediction using machine learning (ML) techniques. However, to the best of our knowledge, there is a lack of a comprehensive systematic literature review (SLR) that summarizes the existing studies on AUD prediction using ML in the last ten years. To address this knowledge gap, this article provides an SLR of academic articles on AUD prediction using ML techniques dated from January 2010 to July 2021. This SLR highlights technical decision analysis related to five aspects: data collection site, characteristics, and type of dataset; data sampling and data pre-processing techniques; feature types and feature engineering techniques; and characteristics of ML techniques and evaluation metrics. Six bibliographic databases were searched, and the identified studies were rigorously reviewed based on the above five aspects. In the selected studies, public datasets were not used very often for AUD prediction. Given that, the current paper identified two different types of data collection sites for review. Imbalanced class distribution in datasets was the primary focus of the pre-processing and sampling steps. Various features, including demographics, family history, drinking behaviour, and electronic health records, were introduced as the more widely used AUD prediction features. The filter, wrapper, and embedded methods were identified as the primary feature selection methods. Support vector machine was the most widely employed algorithm for predicting AUD; however, the lack of deep neural network techniques is evident in this field. Moreover, considering gender disparities, early detection of AUD, and identifying trajectories towards AUD are suggested for future work. For the purpose of evaluating the performance of the prediction approaches, most studies considered the overall accuracy and the area under the receiver operating characteristic curve. Nevertheless, external validation was not performed in any of the selected studies. This paper also discusses challenges and open issues of AUD prediction for future research. This SLR represents a valuable resource for scholars investigating the prediction of AUD.

Air quality, both indoors and outdoors, is crucial for public health as it affects respiratory conditions and overall well-being. Machine learning (ML) techniques offer innovative solutions for monitoring and predicting air quality. However, choosing the right ML algorithms and evaluation metrics is essential for creating accurate air quality prediction models, as these choices directly impact the accuracy and reliability of the results. This study aims to explore and summarize the literature on the use of ML techniques in predicting and optimizing air quality, addressing the urgent issue of air pollution. The review analyzed papers from two electronic databases, namely Scopus and Science Direct. Information on ML techniques for predicting air quality and identifying the main sources of pollution was extracted from 26 studies. The study focuses on common ML techniques employed in air quality prediction, including classification, deep learning, regression, ensemble learning, and combinations of regression and deep learning. It also identifies the evaluation metrics used to assess the performance of these models, such as recall, root mean square error (RMSE), mean absolute error (MAE), and coefficient of determination (R2). By synthesizing existing knowledge, this study provides a comprehensive understanding of the metrics researchers use to measure the overall effectiveness of ML algorithms. It serves as a benchmark for future research and guides the selection of appropriate evaluation metrics in the field of air quality prediction.

Machine Learning (ML) techniques are pervasive tools in various emerging commercial applications, but have to be accommodated by powerful computer systems to process very large data. Although general-purpose CPUs and GPUs have provided straightforward solutions, their energy-efficiencies are limited due to their excessive supports for flexibility. Hardware accelerators may achieve better energy-efficiencies, but each accelerator often accommodates only a single ML technique (family). According to the famous No-Free-Lunch theorem in the ML domain, however, an ML technique performs well on a dataset may perform poorly on another dataset, which implies that such accelerator may sometimes lead to poor learning accuracy. Even if regardless of the learning accuracy, such accelerator can still become inapplicable simply because the concrete ML task is altered, or the user chooses another ML technique.In this study, we present an ML accelerator called PuDianNao, which accommodates seven representative ML techniques, including k-means, k-nearest neighbors, naive bayes, support vector machine, linear regression, classification tree, and deep neural network. Benefited from our thorough analysis on computational primitives and locality properties of different ML techniques, PuDianNao can perform up to 1056 GOP/s (e.g., additions and multiplications) in an area of 3.51 mm^2, and consumes 596 mW only. Compared with the NVIDIA K20M GPU (28nm process), PuDianNao (65nm process) is 1.20x faster, and can reduce the energy by 128.41x.

Optimizing machine learning methods for groundwater quality prediction: Case study in District Bagh, Azad Kashmir, Pakistan.

In this study, we aimed to find an optimized approach to improving facial and masked facial recognition using machine learning and deep learning techniques. Prior studies only used a single machine learning model for classification and did not report optimal parameter values. In contrast, we utilized a grid search with hyperparameter tuning and nested cross-validation to achieve better results during the verification phase. We performed experiments on a large dataset of facial images with and without masks. Our findings showed that the SVM model with hyperparameter tuning had the highest accuracy compared to other models, achieving a recognition accuracy of 0.99912. The precision values for recognition without masks and with masks were 0.99925 and 0.98417, respectively. We tested our approach in real-life scenarios and found that it accurately identified masked individuals through facial recognition. Furthermore, our study stands out from others as it incorporates hyperparameter tuning and nested cross-validation during the verification phase to enhance the model's performance, generalization, and robustness while optimizing data utilization. Our optimized approach has potential implications for improving security systems in various domains, including public safety and healthcare.

<p><b>The client-centric multi-cloud has become a popular cloud ecosystem because it allows enterprise users to share the workload across multiple cloud providers to achieve high-quality services with lower operation costs and higher application resilience. From the perspective of application providers, the location of cloud resources for application deployment significantly impacts the deployment cost and performance of applications, e.g., request response time. This gives rise to the problem of location-aware application deployment in multi-cloud to select suitable cloud resources from widely distributed multi-cloud data centers to balance the cost and performance. Existing research works did not pay full attention to the key impact of the location for application deployment. Therefore, it is urgent to study the problem both theoretically and in practice. In this thesis, innovative optimization methods and machine learning techniques are proposed for three common scenarios, namely composite application deployment, application replication and deployment, and elastic application deployment.</b></p> <p>First, this thesis studies the composite application deployment problem with the goal to minimize the average response time of composite applications subject to a budget constraint. We propose a Hybrid Genetic Algorithm (GA)-based approach, i.e., H-GA, for solving the problem with an extremely large search space. H-GA features a newly designed and domain-tailored service clustering algorithm, repair algorithm, solution representation, population initialization, and genetic operators. Experiments show that H-GA can outperform significantly several state-of-the-art approaches, achieving up to about 8% performance improvement in terms of response time, and 100% budget satisfaction in the meantime.</p> <p>Second, this thesis studies the application replication and deployment problem with the goal to minimize the total deployment cost of all application replicas subject to a stringent requirement on average response time. We propose two approaches under different optimization frameworks to solve the problem. With user requests dispatched to the closest application replicas, we develop an approach under a GA framework for Application Replication and Deployment (ARD), i.e., GA-ARD. GAARD features problem-specific solution representation, fitness measurement, and population initialization, which are effective to optimize the deployment of application replicas in multi-cloud. The experiments show that GA-ARD outperforms common application replication and placement strategies in the industry. With user requests flexibly dispatched among different application replicas, we develop another approach under a two-stage optimization framework, i.e., MCApp. MCApp can optimize both replica deployment and request dispatching by combining mixed-integer linear programming with domain-tailored large neighborhood search. Our experiments show that MCApp can achieve up to 25% reduction in total deployment cost compared with several recently developed approaches.</p> <p>Third, this thesis studies the elastic application deployment problem to minimize the deployment cost over a time span such as a billing day while satisfying the constraint on average response time. The goal of adapting resources for application deployment in response to dynamic and distributed workloads motivates us to adopt deep reinforcement learning (DRL) techniques. The proposed approach, namely DeepScale, applies a deep Q-network (DQN) to capture the optimal scaling policy that can perform online resource scaling. DeepScale also includes a long short-term memory-based prediction model to allow the DQN to consider predicted future requests while making cost-effective scaling decisions. Besides, we design a penalty-based reward function and a safety-aware action executor to ensure that any scaling decisions made by DRL can satisfy the response time constraint. The experiments show that DeepScale can significantly reduce the deployment cost of applications compared with the state-of-the-art baselines, including Amazon auto-scaling service and recently proposed RL-based algorithms. In the meanwhile, DeepScale can effectively satisfy the constraint on the average response time.</p> <p>In summary, this thesis studies three new problems for location-aware application deployment in multi-cloud. We propose four novel approaches under different optimization and machine learning frameworks, i.e., HGA, GA-ARD, MCApp, and DeepScale, for solving these problems. New constraint handling techniques are developed to satisfy the practical deployment requirements of enterprise applications.</p>

Landslide disaster risk reduction necessitates the investigation of different geotechnical causal factors for slope failures. Machine learning (ML) techniques have been proposed to study causal factors across many application areas. However, the development of ensemble ML techniques for identifying the geotechnical causal factors for slope failures and their subsequent prediction has lacked in literature. The primary goal of this research is to develop and evaluate novel feature selection methods for identifying causal factors for slope failures and assess the potential of ensemble and individual ML techniques for slope failure prediction. Twenty-one geotechnical causal factors were obtained from 60 sites (both landslide and non-landslide) spread across a landslide-prone area in Mandi, India. Relevant causal factors were evaluated by developing a novel ensemble feature selection method that involved an average of different individual feature selection methods like correlation, information-gain, gain-ratio, OneR, and F-ratio. Furthermore, different ensemble ML techniques (Random Forest (RF), AdaBoost (AB), Bagging, Stacking, and Voting) and individual ML techniques (Bayesian network (BN), decision tree (DT), multilayer perceptron (MLP), and support vector machine (SVM)) were calibrated to 70% of the locations and tested on 30% of the sites. The ensemble feature selection method yielded six major contributing parameters to slope failures: relative compaction, porosity, saturated permeability, slope angle, angle of the internal friction, and in-situ moisture content. Furthermore, the ensemble RF and AB techniques performed the best compared to other ensemble and individual ML techniques on test data. The present study discusses the implications of different causal factors for slope failure prediction.

Predicting survival outcomes in critical accidents has been a focal point in machine learning research. This study addresses several limitations of existing methods, including insufficient management of data imbalance, lack of emphasis on hyperparameter tuning, and proneness to overfitting. Many existing models struggle to generalize effectively on imbalanced datasets or depend on default hyperparameter settings, resulting in biased predictions. By integrating Principal Component Analysis (PCA), hyperparameter optimization, and resampling methods, as well as combining Edited Nearest Neighbors (ENN) with the Synthetic Minority Oversampling Technique (SMOTE), the model significantly improves predictive accuracy and model generalization. An ensemble model combining seven machine learning algorithms—Logistic Regression, Support Vector Machine, KNN, Random Forest, XGBoost, LightGBM, and CatBoost—was applied to predict survival outcomes. Stochastic Weighted Averaging (SWA) was applied to mitigate overfitting and enhance generalization. The accuracy increased from 91.97% to 94.89% after SWA was applied in this specific scenario. The combination of PCA-based dimensionality reduction, hyperparameter tuning, and resampling techniques (ENN + SMOTE) ensured the model handled data imbalance and optimized predictive accuracy. The final model demonstrated excellent performance, with Area Under the Curve (AUC) and Average Precision (AP) values both reaching 0.98, indicating high accuracy and precision. These improvements were validated using the Titanic dataset in a binary classification problem of predicting passenger survival. The results emphasize that ensemble learning, enhanced by SWA, offers a powerful framework for handling imbalanced and complex datasets, providing significant advancements in predictive modeling accuracy. This study provides insights into how machine learning techniques can be effectively combined to solve classification challenges in real-world scenarios.

Recent progress on machine learning techniques for flow field optimization toward performance and commercial scale proton exchange membrane fuel cell: A systematic review

ABSTRACTThis paper reviews the activity-travel behaviour literature that employs Machine Learning (ML) techniques for empirical analysis and modelling. Machine Learning algorithms, which attempt to build intelligence utilizing the availability of large amounts of data, have emerged as powerful tools in the fields of pattern recognition and big data analysis. These techniques have been applied in activity-travel behaviour studies since the early ’90s when Artificial Neural Networks (ANN) were employed to model mode choice decisions. AMOS, an activity-based modelling system developed in the mid-’90s, has ANN at its core to model and predict individual responses to travel demand management measures. In the dawn of 2000, ALBATROSS, a comprehensive activity-based travel demand modelling system, was proposed by Arentze and Timmermans using Decision Trees. Since then researchers have been exploring ML techniques like Support Vector Machines (SVM), Decision Trees (DT), Neural Networks (NN), Bayes Classifiers, and more recently, Ensemble Learners to model and predict activity-travel behaviour. A large number of publications over the years and an upward trend in the number of published articles over time indicate that Machine Learning is a promising tool for activity-travel behaviour analysis and prediction. This article, first of its kind in the literature, reviews these studies and explores the trends in activity-travel behaviour research that apply ML techniques. The review finds that mode choice decisions have received wide attention in the literature on ML applications. It was observed that most of the studies identify the lack of interpretability as a serious shortcoming in ML techniques. However, very few studies have attempted to improve the interpretability of the models. Further, some studies report the importance of feature engineering in ML-based studies, but very few studies adopt feature engineering before model development. Spatiotemporal transferability of models is another issue that has received minimal attention in the literature. In the end, the paper discusses possible directions for future research in the area of activity-travel behaviour modelling using ML techniques.

Machine learning (ML) techniques learn a system by observing it. Events and occurrences in the network define what is expected of the network’s operation. It is for this reason that ML techniques are used in the computer network security field to detect unauthorized intervention. In the event of suspicious activity, the result of the ML analysis deviates from the definition of expected normal network activity and the suspicious activity becomes apparent. Support vector machines (SVM) are ML techniques that have been used to profile normal network activity and classify it as normal or abnormal. They are trained to configure an optimal hyperplane that classifies unknown input vectors’ values based on their positioning on the plane. We propose to use SVM models to detect malicious behavior within low-power, low-rate and short range networks, such as those used in the Internet of Things (IoT). We evaluated two SVM approaches, the C-SVM and the OC-SVM, where the former requires two classes of vector values (one for the normal and one for the abnormal activity) and the latter observes only normal behavior activity. Both approaches were used as part of an intrusion detection system (IDS) that monitors and detects abnormal activity within the smart node device. Actual network traffic with specific network-layer attacks implemented by us was used to create and evaluate the SVM detection models. It is shown that the C-SVM achieves up to 100% classification accuracy when evaluated with unknown data taken from the same network topology it was trained with and 81% accuracy when operating in an unknown topology. The OC-SVM that is created using benign activity achieves at most 58% accuracy.

Many enzymatic biosensing devices have been developed for specific and selective nitrate detection over the last two decades. They generally use the nitrate-nitrite redox reaction to measure the nitrate. Since the activity of the enzyme used in the structure of the biosensor reduces over time, the operator should replace the enzyme immobilized on the working electrode frequently, which increases the detection costs and limits their commercialization capabilities. In this study, artificial neural networks (ANNs) have been used for nitrate concentration prediction in the samples considering both electrochemical data and enzyme activity decrement over time. Genetic algorithm (GA) and particle swarm optimization (PSO) were used to optimize the architecture of the ANNs used for the decision-making of the biosensor. This study showed that both GA-ANN and PSO-ANN learning algorithms resulted in promising nitrate prediction performance with coefficients of determination (R²) higher than 0.93 and 0.94, respectively. Moreover, the biosensor could be used up to two weeks after the enzyme immobilization for nitrate determination in more than 500 samples after each time of electrode preparation. Finally, a comparison between the findings of this study and former studies that used support vector machines and fuzzy inference systems revealed that ANNs optimized with evolutionary and swarm intelligence techniques can provide more reliable prediction results.

Machine learning and the electrocardiogram over two decades: Time series and meta-analysis of the algorithms, evaluation metrics and applications