Phishing Detection Using Machine Learning: A Model Development and Integration

This paper compares the performance of five different supervised machine learning (SL) algorithms to identify the practical model for tracking copper using solution cathode glow discharge optical emission spectroscopy (SCGD-OES). A total of 4500 SCGD-OES spectra were collected, corresponding to 15 different copper concentrations ranging from 10.3 mg l−1 to 59.6 mg l−1, for training and validating five models: K-nearest neighbors (KNN), random forest (RF), artificial neural network (ANN), convolutional neural network (CNN), and recurrent neural network (RNN). Then, performance of these models were evaluated based on accuracy, precision of predictions and training time. In terms of accuracy, the relative error (RE) was 3% for the CNN model and 4% for the ANN model. Conversely, the RE increased from 21% and 43% for the RF and RNN models to 103% for the KNN model due to the unpredictability of Cu concentration outside the training range. Regarding precision, the relative standard deviation (RSD) values were similar across all models, with the lowest RSD being 3% for the CNN model and the highest being 5% for the RF model. The training time indicated that the KNN and RF models provided the best learning speed, while the CNN model had the slowest learning speed, taking 5 times longer than the ANN model. The accuracy of the ANN and CNN models is in agreement with inductively coupled plasma-optical emission spectroscopy (ICP-OES), with a average difference not exceeding 6% when tracking six different Cu concentrations, both inside and outside the training range. Additionally, this research investigated the weight distribution in the hidden layers of the ANN, CNN, and RNN models and feature importances in the RF model to enhance understanding of the internal structure of models and assist analysts in selecting the practical model for tracking heavy metals based on SCGD-OES.

BackgroundLumbar drainage is widely used in the clinic; however, forecasting lumbar drainage-related meningitis (LDRM) is limited. We aimed to establish prediction models using supervised machine learning (ML) algorithms.MethodsWe utilized a cohort of 273 eligible lumbar drainage cases. Data were preprocessed and split into training and testing sets. Optimal hyper-parameters were archived by 10-fold cross-validation and grid search. The support vector machine (SVM), random forest (RF), and artificial neural network (ANN) were adopted for model training. The area under the operating characteristic curve (AUROC) and precision-recall curve (AUPRC), true positive ratio (TPR), true negative ratio (TNR), specificity, sensitivity, accuracy, and kappa coefficient were used for model evaluation. All trained models were internally validated. The importance of features was also analyzed.ResultsIn the training set, all the models had AUROC exceeding 0.8. SVM and the RF models had an AUPRC of more than 0.6, but the ANN model had an unexpectedly low AUPRC (0.380). The RF and ANN models revealed similar TPR, whereas the ANN model had a higher TNR and demonstrated better specificity, sensitivity, accuracy, and kappa efficiency. In the testing set, most performance indicators of established models decreased. However, the RF and AVM models maintained adequate AUROC (0.828 vs. 0.719) and AUPRC (0.413 vs. 0.520), and the RF model also had better TPR, specificity, sensitivity, accuracy, and kappa efficiency. Site leakage showed the most considerable mean decrease in accuracy.ConclusionsThe RF and SVM models could predict LDRM, in which the RF model owned the best performance, and site leakage was the most meaningful predictor.

To predict the potential geographic distribution of Oncomelania hupensis in Yunnan Province using random forest (RF) and maximum entropy (MaxEnt) models, so as to provide insights into O. hupensis surveillance and control in Yunnan Province. The O. hupensis snail survey data in Yunnan Province from 2015 to 2016 were collected and converted into O. hupensis snail distribution site data. Data of 22 environmental variables in Yunnan Province were collected, including twelve climate variables (annual potential evapotranspiration, annual mean ground surface temperature, annual precipitation, annual mean air pressure, annual mean relative humidity, annual sunshine duration, annual mean air temperature, annual mean wind speed, ≥ 0 ℃ annual accumulated temperature, ≥ 10 ℃ annual accumulated temperature, aridity and index of moisture), eight geographical variables (normalized difference vegetation index, landform type, land use type, altitude, soil type, soil textureclay content, soil texture-sand content and soil texture-silt content) and two population and economic variables (gross domestic product and population). Variables were screened with Pearson correlation test and variance inflation factor (VIF) test. The RF and MaxEnt models and the ensemble model were created using the biomod2 package of the software R 4.2.1, and the potential distribution of O. hupensis snails after 2016 was predicted in Yunnan Province. The predictive effects of models were evaluated through cross-validation and independent tests, and the area under the receiver operating characteristic curve (AUC), true skill statistics (TSS) and Kappa statistics were used for model evaluation. In addition, the importance of environmental variables was analyzed, the contribution of environmental variables output by the models with AUC values of > 0.950 and TSS values of > 0.850 were selected for normalization processing, and the importance percentage of environmental variables was obtained to analyze the importance of environmental variables. Data of 148 O. hupensis snail distribution sites and 15 environmental variables were included in training sets of RF and MaxEnt models, and both RF and MaxEnt models had high predictive performance, with both mean AUC values of > 0.900 and all mean TSS values and Kappa values of > 0.800, and significant differences in the AUC (t = 19.862, P < 0.05), TSS (t = 10.140, P < 0.05) and Kappa values (t = 10.237, P < 0.05) between two models. The AUC, TSS and Kappa values of the ensemble model were 0.996, 0.954 and 0.920, respectively. Independent data verification showed that the AUC, TSS and Kappa values of the RF model and the ensemble model were all 1, which still showed high performance in unknown data modeling, and the MaxEnt model showed poor performance, with TSS and Kappa values of 0 for 24%(24/100) of the modeling results. The modeling results of 79 RF models, 38 MaxEnt models and their ensemble models with AUC values of > 0.950 and TSS values of > 0.850 were included in the evaluation of importance of environmental variables. The importance of annual sunshine duration (SSD) was 32.989%, 37.847% and 46.315% in the RF model, the MaxEnt model and their ensemble model, while the importance of annual mean relative humidity (RHU) was 30.947%, 15.921% and 28.121%, respectively. Important environment variables were concentrated in modeling results of the RF model, dispersed in modeling results of the MaxEnt model, and most concentrated in modeling results of the ensemble model. The potential distribution of O. hupensis snails after 2016 was predicted to be relatively concentrated in Yunnan Province by the RF model and relatively large by the MaxEnt model, and the distribution of O. hupensis snails predicted by the ensemble model was mostly the joint distribution of O. hupensis snails predicted by RF and MaxEnt models. Both RF and MaxEnt models are effective to predict the potential distribution of O. hupensis snails in Yunnan Province, which facilitates targeted O. hupensis snail control.

Generating high-resolution daily soil moisture by using spatial downscaling techniques: a comparison of six machine learning algorithms

Noninvasive assessment of skin structure using hyperspectral images has been intensively studied in recent years. Due to the high computational cost of the classical methods, such as the inverse Monte Carlo (IMC), much research has been done with the aim of using machine learning (ML) methods to reduce the time required for estimating parameters. This study aims to evaluate the accuracy and the estimation speed of the ML methods for this purpose and compare them to the traditionally used inverse adding-doubling (IAD) algorithm. We trained three models – an artificial neural network (ANN), a 1D convolutional neural network (CNN), and a random forests (RF) model – to predict seven skin parameters. The models were trained on simulated data computed using the adding-doubling algorithm. To improve predictive performance, we introduced a stacked dynamic weighting (SDW) model combining the predictions of all three individually trained models. SDW model was trained by using only a handful of real-world spectra on top of the ANN, CNN and RF models that were trained using simulated data. Models were evaluated based on the estimated parameters’ mean absolute error (MAE), considering the surface inclination angle and comparing skin spectra with spectra fitted by the IAD algorithm. On simulated data, the lowest MAE was achieved by the RF model (0.0030), while the SDW model achieved the lowest MAE on in vivo measured spectra (0.0113). The shortest time to estimate parameters for a single spectrum was 93.05 μs. Results suggest that ML algorithms can produce accurate estimates of human skin optical parameters in near real-time.

Land-use regression model and machine learning algorithms in assessing outdoor noise exposure in Hong KongBackground/Aim: There are epidemiological studies in reporting the association between environmental noise exposure and various health outcomes, including cardiovascular diseases and cognitive decline. However, evidence in using land-use regression (LUR) model in predicting noise is limited and not conclusive, while machine learning algorithms are seemed to be a novel method in improving the explanative power of noise prediction. In this study, random forest (RF) model was adopted to estimate outdoor noise level in Hong Kong and compare the performance between LUR and RF. Methods: A total of 102 measurement sites with 27 environment-related variables were used to estimate the spatial environmental noise variation. Two separate measurements for A-weighted equivalent sound pressure levels over 24 hours (Leq,24h) were conducted at each site during the 2019-2020 period. Land use types, traffic information and geographic information systems were chosen to the related variables for building models. Ordinary linear regression was adopted to be the traditional land-use regression model. Results: The annual mean of 102 measurement sites of Leq,24h was 66.3 dB(A). The proportion of variation (R2) explained by the random forest model in full-frequency noise was higher than the traditional LUR (R2: RF: 0.76, LUR: 0.69). Random forest performed also better in lower root mean squared errors (RF: 2.91 dB(A), LUR: 3.22 dB(A)). Further results on the noise level in different time periods and frequencies will be described in the conference. Conclusion: Random forest model is suggested to be a better alternative in estimating noise exposure. Further study on the application of the model on epidemiological studies can be conducted.

Air pollution in Macau has become a serious problem following the Pearl River Delta’s (PRD) rapid industrialization that began in the 1990s. With this in mind, Macau needs an air quality forecast system that accurately predicts pollutant concentration during the occurrence of pollution episodes to warn the public ahead of time. Five different state-of-the-art machine learning (ML) algorithms were applied to create predictive models to forecast PM2.5, PM10, and CO concentrations for the next 24 and 48 h, which included artificial neural networks (ANN), random forest (RF), extreme gradient boosting (XGBoost), support vector machine (SVM), and multiple linear regression (MLR), to determine the best ML algorithms for the respective pollutants and time scale. The diurnal measurements of air quality data in Macau from 2016 to 2021 were obtained for this work. The 2020 and 2021 datasets were used for model testing, while the four-year data before 2020 and 2021 were used to build and train the ML models. Results show that the ANN, RF, XGBoost, SVM, and MLR models were able to provide good performance in building up a 24-h forecast with a higher coefficient of determination (R2) and lower root mean square error (RMSE), mean absolute error (MAE), and biases (BIAS). Meanwhile, all the ML models in the 48-h forecasting performance were satisfactory enough to be accepted as a two-day continuous forecast even if the R2 value was lower than the 24-h forecast. The 48-h forecasting model could be further improved by proper feature selection based on the 24-h dataset, using the Shapley Additive Explanations (SHAP) value test and the adjusted R2 value of the 48-h forecasting model. In conclusion, the above five ML algorithms were able to successfully forecast the 24 and 48 h of pollutant concentration in Macau, with the RF and SVM models performing the best in the prediction of PM2.5 and PM10, and CO in both 24 and 48-h forecasts.

Predictive artificial intelligence (AI) models for the assessment of factors and trends related to disease prevalence, prognosis, and the risk prediction of mortality in the early phases, seem to be effective techniques. Machine learning (ML) algorithms as a branch of AI methods have had a considerable role in controlling and management of COVID-19 and previous pandemics. The present study estimates the mortality risk among COVID-19 patients based on ML algorithms, prediction of outcomes in new cases, and also determines the top significant features related to morality in the patients. In this retrospective research, 7115 patients with confirmed COVID-19 disease were included in the study. The information was collected from 35 different hospitals across Khorasan-Razavi province, Northeast of Iran during 20 February 2020 - 21 June 2021. We employed random forests (RF), decision tree (DT), artificial neural networks (ANN), support vector machines (SVM), logistic regression (LR), and boosted trees (BT) ML algorithms. Our findings indicated that the RF model had better performance in different criteria. The RF algorithm had area under curve (AUC) of 88%, accuracy of 96%, precision of 96%, sensitivity of 99%, and specificity of 80%. Intubation, SpO2, age, time from symptoms, loss of consciousness, reception season, and diabetes were the seven ordered and significant predictors of the risk of COVID-19 patients’ death. RF model can aid in clinical decision-making and limited resource allocation. This model needs to be externally validated in larger populations, more features, and multicenter settings. Received: August 4, 2023Accepted: September 13, 2023

Background Predicting the prognosis of patients with acute myocardial infarction (AMI) combined with diabetes mellitus (DM) is crucial due to high in-hospital mortality rates. This study aims to develop and validate a mortality risk prediction model for these patients by interpretable machine learning (ML) methods. Methods Data were sourced from the Medical Information Mart for Intensive Care IV (MIMIC-IV, version 2.2). Predictors were selected by Least absolute shrinkage and selection operator (LASSO) regression and checked for multicollinearity with Spearman’s correlation. Patients were randomly assigned to training and validation sets in an 8:2 ratio. Seven ML algorithms were used to construct models in the training set. Model performance was evaluated in the validation set using metrics such as area under the curve (AUC) with 95% confidence interval (CI), calibration curves, precision, recall, F1 score, accuracy, negative predictive value (NPV), and positive predictive value (PPV). The significance of differences in predictive performance among models was assessed utilising the permutation test, and 10-fold cross-validation further validated the model’s performance. SHapley Additive exPlanations (SHAP) and Local Interpretable Model-agnostic Explanations (LIME) were applied to interpret the models. Results The study included 2,828 patients with AMI combined with DM. Nineteen predictors were identified through LASSO regression and Spearman’s correlation. The Random Forest (RF) model was demonstrated the best performance, with an AUC of 0.823 (95% CI: 0.774–0.872), high precision (0.867), accuracy (0.873), and PPV (0.867). The RF model showed significant differences (p < 0.05) compared to the K-Nearest Neighbours and Decision Tree models. Calibration curves indicated that the RF model’s predicted risk aligned well with actual outcomes. 10-fold cross-validation confirmed the superior performance of RF model, with an average AUC of 0.828 (95% CI: 0.800–0.842). Significant Variables in RF model indicated that the top eight significant predictors were urine output, maximum anion gap, maximum urea nitrogen, age, minimum pH, maximum international normalised ratio (INR), mean respiratory rate, and mean systolic blood pressure. Conclusion This study demonstrates the potential of ML methods, particularly the RF model, in predicting in-hospital mortality risk for AMI patients with DM. The SHAP and LIME methods enhance the interpretability of ML models.

A method for accurately classifying donkey breeds has been developed by integrating single nucleotide polymorphism (SNPs) data with machine learning algorithms. The approach includes preprocessing donkey genomic sequencing data, addressing data imbalance with the Synthetic Minority Over-sampling Technique (SMOTE), and utilizing an improved Leave-One-Out Cross-Validation (LOOCV) for dataset partitioning. Support Vector Machine (SVM), K-Nearest Neighbors (KNN), and Random Forest (RF) models were constructed and evaluated. The results demonstrated that different chromosomes significantly influence classifier performance. For instance, chromosome Chr2 showed the highest classification accuracy with KNN, while chromosome Chr19 performed best with SVM and RF models. After enhancing data quality and addressing imbalances, classification performance improved substantially, with accuracy, precision, recall, and F1 score showing increases of up to 15% in certain models, particularly on key chromosomes. This method offers an effective solution for donkey breed classification and provides technical support for the conservation and development of donkey genetic resources.

BackgroundA considerable portion of patients with fever of unknown origin (FUO) present concomitant lymphadenopathy. Diseases within the spectrum of FUO accompanied by lymphadenopathy include lymphoma, infections, and rheumatic diseases. Particularly, lymphoma has emerged as the most prevalent etiology of FUO with associated lymphadenopathy. Distinguishing between benign and malignant lymph node lesions is a major challenge for physicians and an urgent clinical concern for patients. However, conventional imaging techniques, including PET/CT, often have difficulty accurately distinguishing between malignant and benign lymph node lesions. This study utilizes PET/CT radiomics to differentiate between lymphoma and benign lymph node lesions in patients with FUO, aiming to improve diagnostic accuracy.ResultsData were collected from 204 patients who underwent 18F-FDG PET/CT examinations for FUO, including 114 lymphoma patients and 90 patients with benign lymph node lesions. Patients were randomly divided into training and testing groups at a ratio of 7:3. A total of 15 effective features were obtained by the least absolute shrinkage and selection operator (LASSO) algorithm. Machine learning models were constructed using logistic regression (LR), support vector machine (SVM), random forest (RF), and k-nearest neighbors (KNN) algorithms. In the training group, the area under the curve (AUC) values for predicting lymphoma and benign cases by LR, SVM, RF, and KNN models were 0.936, 0.930, 0.998, and 0.938, respectively. There were statistically significant differences in AUC between the RF and other models (all P < 0.001). In the testing group, the AUC values for the four models were 0.860, 0.866, 0.915, and 0.891, respectively, with no statistically significant differences observed among them (all P > 0.05). The decision curve analysis (DCA) curves of the RF model outperformed those of the other three models in both the training and testing groups.ConclusionsPET/CT radiomics demonstrated promising performance in discriminating lymphoma from benign lymph node lesions in patients with FUO, with the RF model showing the best performance.

Fire is a natural disaster that can be caused by many different reasons. Recently, more environmentally friendly and innovative extinguishing methods have started to be tested, some of which are also used. For this purpose, a sound wave fire-extinguishing system was created and firefighting tests were performed. With the data obtained, as a result of 17,442 tests, a data set was created. In this study, five different machine learning methods were used by using the data set created. These are artificial neural network, k-nearest neighbor, random forest, stacking and deep neural network methods. Stacking method is an ensemble method created by using artificial neural network, k-nearest neighbor, random forest models together. Classification of extinction and non-extinction states of the flame was made with the models created with these methods. The accuracy of models in classification should be analyzed in detail in order to be used as a decision support system in the sound wave fire-extinguishing system. Hence, the classification processes were carried out through the 10-fold cross-validation method. As a result of these tests, the performance analysis of the models was carried out, and the results showed that the highest classification accuracy belongs to the stacking model with 97.06&#x0025;. The classification accuracy was determined 96.58&#x0025; in random forest method, 96.03&#x0025; in artificial neural network model, 94.88&#x0025; in deep neural network model and 92.62&#x0025; in k-NN model. The performance of the methods was compared by analyzing the performance metrics of machine learning methods. Thanks to the decision support system to be obtained based on the results of the analyzes, the sound wave fire-extinguishing system can be used efficiently.

BackgroundTranscutaneous transhepatic gallbladder drainage (PTGBD) has shown significant efficacy in the treatment of elderly patients with acute cholecystitis. The goal of this study is to develop a machine learning-based web calculator aimed at predicting the optimal timing for cholecystectomy (LC) after PTGBD in elderly patients with acute cholecystitis (AC) to achieve precise personalized medicine.MethodsA retrospective analysis of 979 elderly patients with acute cholecystitis admitted to Jinzhou Central Hospital and the First Affiliated Hospital of Jinzhou Medical University from 2013 to 2024 was performed, and a total of 680 patients were included in the model development. Patients were divided into delayed (347 cases, surgery > 6 weeks post-PTGBD) and non-delayed (333 cases) groups based on the interval between PTGBD and LC. Minimal Absolute Contraction and Selection Operator (LASSO) and logistic analysis were used to determine the predictors of postponement of LC in elderly patients with AC after PTGBD. Next, we used eight ML algorithms, namely Logistic Regression (LR), Decision Tree (DT), Random Forest (RF), Extreme gradient boosting (XGB), Support Vector Machine (SVM), Multilayer Perceptron (MLP), K-nearest Neighbor (KNN), Gaussian Naive Bayes (GNB), to train and develop ML models using a 10x cross-validation method. The performance of the model was evaluated by a variety of indicators, including the area under the receiver operating characteristic curve (ROC), calibration curve, decision curve, PR curve, and confusion matrix. In addition, model interpretation is performed through Shapley Additive Interpretation (SHAP) analysis to clarify the importance of each feature of the model and its basis for decision-making. Finally, we chose to use the best model to develop a web-based calculator that could be used to predict the likelihood of delaying LC after PTGBD in elderly AC patients.ResultsIn multivariate logistic regression analysis, age, sex, gallbladder wall thickness, time between onset and PTGBD, white blood cell count (WBC), C-reactive protein (CRP), and neutrophil-to-lymphocyte ratio (NLR) were identified as independent predictors of delayed LC in elderly patients with AC after PTGBD. In the training set, the area under the receiver operating characteristic curve (AUC) values for these models ranged from 0.808 to 0.914, with the random forest (RF) model showing the highest AUC value. Through the evaluation of decision curve analysis (DCA), precision-recall (PR) curve and calibration curve, the RF model showed superior clinical decision support and prediction performance compared with the other seven models. Finally, we used the RF model to build an online network calculator, which aims to accurately assist doctors in making more informed and accurate clinical decisions and promote the wide application of the model in clinical practice (https://zw17786325639.shinyapps.io/Postpone/).ConclusionsThis study developed and validated an RF model network calculator based on clinical indicator information to assess the likelihood of delaying LC after PTGBD in elderly patients with acute cholecystitis. This tool is expected to assist physicians in making more appropriate clinical decisions for patients.

Accurate evaluation of thyroid nodules is crucial for effective management; however, methods such as ultrasonography and Fine Needle Aspiration Cytology (FNAC) can be subjective and operator-dependent. Indeterminate thyroid nodules (ITNs) complicate diagnosis, coming at the expense of time, money, and potentially additional FNA samplings, causing more discomfort for the patients. Recent advancements in artificial intelligence (AI) assisted ultrasound diagnosis system have demonstrated excellent diagnostic performance and the potential to aid in the differentiation of ITNs. This study aims to develop an AI classifier that integrates the AI-assisted ultrasound diagnosis system, FNAC, and demographic data to enhance the differentiation of benign and malignant thyroid nodules, and to compare the diagnostic performance of the models, with a focus on diagnosing ITNs. In the present research, 620 thyroid nodules were collected from a single medical center and divided into training and testing cohorts (Testing1). We developed five AI models using distinct classification algorithms (Logistic Regression, Support Vector Machine, K-Nearest Neighbor, Random Forest, and Gradient Boosting Machine) that integrate demographic data, cytological findings, and an AI-assisted ultrasound diagnostic system for thyroid nodule assessment. These models underwent prospective validation (Testing2, n = 243) to identify the optimal model. A subsequent prospective study (Testing3) involving 70 thyroid nodules further evaluated the model's performance, where the selected optimal model was compared against FNAC combined with BRAF V600E mutation analysis. After validation with the Testing1 and Testing2 cohorts, the Random Forest (RF) model demonstrated the best overall performance among the five classifiers. The area under the curve (AUC) for the RF model to diagnose thyroid nodules was 0.994 in the training cohort, 0.993 in the testing cohort, and 0.977 in the prospective data. In addition, for 42 included ITNs in the prospective data, the accuracy, sensitivity, and specificity of the RF model were 90.48%, 89.47%, and 91.30%, respectively. In the Testing 3 cohort, the RF model demonstrated superior diagnostic performance compared to both the standalone AI ultrasound auxiliary diagnostic system and FNAC alone. Its accuracy was comparable to FNAC combined with BRAF V600E mutation analysis. Conclusion: Our developed thyroid nodule AI diagnostic model shows favorable predictive value. It can serve as a decision support tool for non-thyroid specialists and assist thyroid surgeons in the management of ITN. Our developed thyroid nodule AI diagnostic model shows favorable predictive value. It can serve as a decision support tool for non-thyroid specialists and assist thyroid surgeons in the management of ITN.

Hypothyroidism (hypo means “under” or “below normal”) is the most common thyroid disease. It affects people worldwide irrespective of sex, age, race, and level of education, and wealth. The hypothyroid disease diagnosis in the early stage will help take precautions in preventing future complications. This work focuses on implementing a Machine Learning (ML)-based real-time hypothyroid disease classification framework. The hypothyroid dataset from UCI was employed to build the ML models. The K-Nearest Neighbor (KNN), Random Forest (RF), and Decision Tree (DT) ML models were implemented and achieved an accuracy of 98% for all models. Later the Min-Max Scaler data normalization technique was applied to improve the performance and achieved average accuracy of 99% for both RF and DT models. The hypothyroid dataset has imbalanced data. In such a case, F1-score is an important metric to evaluate the model performance. K-fold cross validation technique is applied to cross-validate the models. The average F1-score was found to be 0.94 for the RF model, whereas 0.92 for DT and 0.9 for KNN. This performance indicates RF model is performing well for predicting hypothyroidism and also outperforms the models reported in the literature.