Prediction of Heart Failure by using Machine Learning and Feature Selection

The objective of this document is to identify the most accurate machine learning (ML) models for predicting the value of a real estate property, based on a systematic literature review (SLR). It was conducted on research published between 2022 and 2023 that analyzed the accuracy of ML models in real estate valuation. Information was extracted regarding the ML models used, the databases employed, and the models highlighted for their accuracy. A variety of ML models used in real estate valuation were identified, including Random Forest (RF), XGBoost, Gradient Boosting Machine (GBM), Linear Regression (LR), and Lasso Regression. RF and LR models stood out as the most accurate in the analyzed research, showing that the accuracy of ML models varies depending on the database, property characteristics, and valuation context. It was concluded that ML models like RF and LR are promising tools for improving the accuracy of real estate valuation. Additionally, the choice of the best ML model depends on factors such as the database, property characteristics, and specific valuation objectives. Further research is needed on the application of ML in real estate valuation, considering aspects such as model interpretability and the real estate market context.

This paper investigates the importance of explainability versus accuracy of machine learning (ML) models. We propose that greater task uncertainty makes people want to interact more with the ML model, which increases the importance of explainability relative to accuracy. We focus on the use of ML models for product cost estimation during new product development. The paper provides mixed-methods evidence on the trade-off between explainability and accuracy of ML models. Specifically, we find support for an inverse relationship between explainability and accuracy from the perspective of cost experts. We also find that the accurate but complex and less explainable ML model of gradient boosted regression (GBR) was preferred in only a few situations; mostly, the more basic, better explainable models of multiple linear regression (MLR) and case-based reasoning (CBR) were preferred, although these were less accurate. This suggests that lack of explainability can indeed be a major limitation for the application of ML models. Furthermore, we investigate specific characteristics that could increase task uncertainty and the importance of explainability in our context: project unpredictability, product cost granularity, predecessor product availability, target cost gap, and product development phase.

The prediction of wind energy generation is important to enhance the performance and dependability of renewable energy systems due to the rising demand for wind-generated electricity and advancements in wind energy technology competitiveness. This study leverages advanced machine learning (ML) and some other statistical and deep learning based time series forecasting models to enhance the accuracy of wind energy predictions. This comprehensive analysis includes nine ML models—Linear Regression, Random Forests (RF), Gradient Boosting Machines (GBM), Support Vector Machines (SVM), K-Nearest Neighbors (KNN), AdaBoost, XGBoost, Support Vector Regression (SVR), and Neural Networks—as well as Four time-series forecasting models—ARIMA, Temporal Convolutional Networks (TCNs), Long Short-Term Memory (LSTM) networks and GRU. Each ML model underwent rigorous cross-validation to ensure optimal performance. The assessment criteria utilized here comprised the Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), and the R² Score. It was found that among the nine ML models, Random Forests, GBM and KNN consistently provided superior accuracy and robustness, making them the top choices for wind energy prediction whereas the performance of linear regression, SVM and SVR were very poor for the considered dataset. From the experiment, Random Forest, GBM, and KNN showed the best performance with low MSE values of 0.77, 1.95, and 1.51 respectively, while other models had MSEs above 7.5, with AdaBoost reaching 30. Their RMSEs (0.88, 1.40, 1.23) and MAEs (0.093, 0.73, 0.10) also indicate strong predictive accuracy compared to the rest.In this paper, time series forecasting, TCNs, LSTM and GRU networks showed strong capabilities in capturing temporal dependencies and trends within the wind energy data. Visualization techniques were employed to compare model performances comprehensively, providing clear insights into their predictive power. Therefore, this present study offers a robust framework for researchers and practitioners aiming to leverage machine learning and time series forecasting in the realm of renewable energy prediction.

Classification using machine learning algorithms in physical fitness tests carried out by students in educational centers can help prevent obesity and other related diseases. This research aims to evaluate physical fitness using percentiles of the tests and machine learning algorithms with hyperparameter optimization. The process followed was knowledge discovery in databases (KDD). Data were collected from 1525 students (784 women, 741 men) aged 6 to 17, selected non-probabilistically from five public schools. For the evaluation, anthropometric parameters such as age, weight, height, sitting height, abdominal circumference, relaxed arm circumference, oxygen saturation, resting heart rate, and maximum expiratory flow were considered. Physical Fitness tests included sitting flexibility, kangaroo horizontal jump, and 20-meter fly speed. Within the percentiles observed, we took three cut-off points as a basis for the present research: > P75 (above average), p25 to p75 (average), and < P25 (below average). The following machine learning algorithms were used for classification: Random Forest, Support Vector Machine, Decision tree, Logistic Regression, Naive Bayes, K-nearest neighbor, XGBboost, Neural network, Cat Boost, LGBM, and Gradient Boosting. The algorithms were hyperparameter optimized using GridSearchCV to find the best configurations. In conclusion, the importance of hyperparameter optimization in improving the accuracy of machine learning models is highlighted. Random Forest performs well in classifying the “High” and “Low” categories in most tests but struggles to correctly classify the “Normal” category for both male and female students.

Development of compressive strength prediction platform for concrete materials based on machine learning techniques

Digital imaging

Chronic pain around the temporomandibular joint (TMJ) and masticatory muscles is a primary symptom of temporomandibular disorders (TMD). However, the clinical significance of magnetic resonance imaging (MRI) features in predicting TMD-related pain remains unclear. This study aimed to develop and interpret machine learning (ML) models based on MRI characteristics for predicting pain severity in patients with TMD. The present retrospective study included 584 patients with TMD between January 2022 and December 2024, yielding a total of 755 TMJ MRI data sets. Pain severity was classified using the visual analogue scale (VAS). Demographic variables (age, sex) and MRI features-including lesion side, disc position, disc morphology, disc signal, disc perforation, bilaminar zone tear, joint space, joint effusion, condylar movement, bony changes and morphology/signal of the lateral pterygoid muscle-were collected. Eleven ML models based on demographic and MRI features were developed: logistic regression (LR), support vector machine (SVM), random forest (RF), extreme gradient boosting (XGBoost), light gradient boosting machine (LightGBM), adaptive boosting (AdaBoost), gradient boosting classifier (GBC), bagging classifier (BC), extremely randomised trees (ETC), decision tree classifier (DTC) and multilayer perceptron (MLP). Model performance was evaluated using multiple metrics, including the area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, specificity and F1 score. Precision-recall (PR) curves and calibration curves were plotted to assess discrimination and model calibration. Decision curve analysis (DCA) was conducted to evaluate the clinical net benefit across a range of threshold probabilities. Model interpretability was enhanced using Shapley Additive Explanations (SHAP), which quantified the contribution of each feature to individual predictions. Feature selection was conducted based on mean SHAP values, and separate LightGBM models were constructed using the Top 3, 5, and 9 most important features, as well as the full-feature set, for performance comparison. The data set was randomly divided into a training set (n = 604) and a test set (n = 151). Among the 11 ML models, the LightGBM model demonstrated the best predictive performance, with an AUC of 0.899, and was therefore identified as the optimal model. SHAP analysis identified age, disc position and condylar movement as the top three contributing features. Feature selection analysis indicated that selecting the top nine SHAP-ranked variables led to the highest diagnostic performance, with an AUC of 0.829. This study developed an interpretable, high-performing MRI-based ML model incorporating SHAP analysis to integrate imaging and clinical features for objective pain assessment, which may help identify high-risk TMD patients and guide personalised treatment strategies.

Parkinson’s disease (PD) is a neurodegenerative disease cause by a deficiency of dopamine. Investigators have identified the voice as the underlying symptom of PD. Advanced vocal disorder studies provide adequate treatment and support for accurate PD detection. Machine learning (ML) models have recently helped to solve problems in the classification of chronic diseases. This work aims to analyze the effect of selecting features on ML efficiency on a voice-based PD detection system. It includes PD classification models of Random forest, decision Tree, neural network, logistic regression and support vector machine. The feature selection is made by RF mean-decrease in accuracy and mean-decrease in Gini techniques. Random forest kerb feature selection (RFKFS) selects only 17 features from 754 attributes. The proposed technique uses validation metrics to assess the performance of ML models. The results of the RF model with feature selection performed well among all other models with high accuracy score of 96.56% and a precision of 88.02%, a sensitivity of 98.26%, a specificity of 96.06%. The respective validation score has an Non polynomial vector (NPV) of 99.47%, a Geometric Mean (GM) of 97.15%, a Youden’s index (YI) of 94.32%, and a Matthews’s correlation method (MCC) 90.84%. The proposed model is also more robust than other models. It was also realised that using the RFKFS approach in the PD results in an effective and high-performing medical classifier.

Background: Osteoporosis is a systemic skeletal disorder marked by reduced bone density and microarchitectural deterioration, leading to increased fracture risk. While the dual-energy X-ray absorptiometry (DEXA) scan is the World Health Organization (WHO)-recommended diagnostic standard, its limitations necessitate alternative methods. Emerging magnetic resonance imaging (MRI) techniques, radiomics, and machine learning promise to enhance osteoporosis diagnosis through detailed analysis of lumbar MRI apparent diffusion coefficient (ADC) maps, potentially revolutionizing early detection and treatment strategies. Objectives: In this study, we are going to evaluate the performance of machine learning (ML) models using radiomics features of lumbar MRI ADC map for osteoporosis detection, and to identify significant features and their diagnostic thresholds. Specific performance metrics such as accuracy, sensitivity, specificity, and Area Under the receiver operating characteristic (ROC) Curve (AUC) were assessed. Patients and Methods: This retrospective study employed a cross-sectional design, with a total of 140 cases, including 21 with osteoporosis. The study's inclusion criteria consisted of concurrent lumbar MRI and DEXA within a year, while exclusion criteria included infectious or neoplastic lumbar lesions, fractures, instrumentation, significant osteodegenerative changes, cases where the first four lumbar vertebrae were not included in the imaging field, and absence of diffusion-weighted imaging. Manual segmentation of lumbar vertebrae from ADC maps was performed to create a comprehensive dataset, comprising 5,580 radiomics features per case. Subsequently, the top five features selected by fast correlation-based filter (FCBF) were used to test the performance of seven Machine Learning algorithms (k-Nearest neighbors, decision tree, random forest, logistic regression, support vector machine, naive bayes, and neural network). Statistical tests and ROC curve analysis were conducted to determine the significance and thresholds of these features. Results: The study included 140 cases, with 132 females (94.3%) and 8 males (5.7%), and a mean age of 65.32 ± 8.50 years. The mean BMI was 31.43 ± 5.53 kg/m² for females and 26 ± 3.59 kg/m² for males. In terms of demographic differences, no significant age difference was found between the osteoporotic and non-osteoporotic groups (P = 0.889). However, the osteoporotic group had significantly lower mean body weight (64.90 ± 10.13 kg vs. 74.68 ± 13.94 kg, P = 0.003) and BMI (27.40 ± 4.38 kg/m² vs. 31.77 ± 5.52 kg/m², P = 0.001) compared to the non-osteoporotic group. The median interval between DEXA and lumbar MRI was 1 month (range 0.1 - 11.87 months). The Neural Network model demonstrated the highest performance with an AUC of 0.616 and a classification accuracy of 0.764 using all features. The Naive Bayes model, using the top five features selected by FCBF, showed the highest performance with an AUC of 0.913, accuracy of 0.907, sensitivity of 0.667, and specificity of 0.95. All ML models’ performance were elevated by feature selection. Independent t-tests and Mann-Whitney U tests identified 521 and 670 significant features, respectively (P &lt; 0.05). ROC analysis revealed 58 features with AUC values above 0.70. Conclusion: This study's findings suggest that ML models, particularly the Naive Bayes algorithm, can effectively use lumbar ADC map radiomics to diagnose osteoporosis. These findings could enhance early detection and treatment strategies, potentially improving patient outcomes and reducing the burden of osteoporotic fractures. This study also established threshold values for significant features.

Objective Machine learning (ML) is expected to play an increasing role within primary health care (PHC) in coming years. No peer-reviewed studies exist that evaluate the diagnostic accuracy of ML models compared to general practitioners (GPs). The aim of this study was to evaluate the diagnostic accuracy of an ML classifier on primary headache diagnoses in PHC, compare its performance to GPs, and examine the most impactful signs and symptoms when making a prediction. Design A retrospective study on diagnostic accuracy, using electronic health records from the database of the Primary Health Care Service of the Capital Area (PHCCA) in Iceland. Setting Fifteen primary health care centers of the PHCCA. Subjects All patients that consulted a physician, from 1 January 2006 to 30 April 2020, and received one of the selected diagnoses. Main outcome measures Sensitivity, Specificity, Positive Predictive Value, Matthews Correlation Coefficient, Receiver Operating Characteristic (ROC) curve, and Area under the ROC curve (AUROC) score for primary headache diagnoses, as well as Shapley Additive Explanations (SHAP) values of the ML classifier. Results The classifier outperformed the GPs on all metrics except specificity. The SHAP values indicate that the classifier uses the same signs and symptoms (features) as a physician would, when distinguishing between headache diagnoses. Conclusion In a retrospective comparison, the diagnostic accuracy of the ML classifier for primary headache diagnoses is superior to GPs. According to SHAP values, the ML classifier relies on the same signs and symptoms as a physician when making a diagnostic prediction. Keypoints Little is known about the diagnostic accuracy of machine learning (ML) in the context of primary health care, despite its considerable potential to aid in clinical work. This novel research sheds light on the diagnostic accuracy of ML in a clinical context, as well as the interpretation of its predictions. If the vast potential of ML is to be utilized in primary health care, its performance, safety, and inner workings need to be understood by clinicians.

Lung cancer remains a leading cause of global mortality, with early detection being critical for improving the patient survival rates. However, applying machine learning and deep learning effectively for lung cancer prediction using symptomatic and lifestyle data requires the careful consideration of feature selection and model optimization, which is not consistently addressed in the existing research. This research addresses this gap by systematically evaluating and comparing the predictive efficacy of several machine learning and deep learning models, employing rigorous data preprocessing, including feature selection with Pearson’s correlation, outlier removal, and normalization, on a patient symptom and lifestyle factor dataset from Kaggle. Machine learning classifiers, including Decision Trees, K-Nearest Neighbors, Random Forest, Naïve Bayes, AdaBoost, Logistic Regression, and Support Vector Machines, were implemented using Weka simultaneously with neural network models with 1, 2, and 3 hidden layers, which were developed in Python within a Jupyter Notebook environment. The model performance was assessed using K-fold cross-validation and 80/20 train/test splitting. The results highlight the importance of feature selection for enhancing the model accuracy and demonstrate that the single-hidden-layer neural network, trained for 800 epochs, achieved a prediction accuracy of 92.86%, outperforming the machine learning models. This study contributes to developing more effective computational methods for early lung cancer detection, ultimately supporting improved patient outcomes.

Summary A meticulous interpretation of steady-state or unsteady-state relative permeability (Kr) experimental data is required to determine a complete set of Kr curves. In this work, different machine learning (ML) models were developed to assist in a faster estimation of these curves from steady-state drainage coreflooding experimental runs. These ML algorithms include gradient boosting (GB), random forest (RF), extreme gradient boosting (XGB), and deep neural network (DNN) with a main focus on and comparison of the two latter algorithms (XGB and DNN). Based on existing mathematical models, a leading-edge framework was developed where a large database of Kr and capillary pressure (Pc) curves were generated. This database was used to perform thousands of coreflood simulation runs representing oil-water drainage steady-state experiments. The results obtained from these simulation runs, mainly pressure drop along with other conventional core analysis data, were used to estimate analytical Kr curves based on Darcy’s law. These analytically estimated Kr curves along with the previously generated Pc curves were fed as features into the ML model. The entire data set was split into 80% for training and 20% for testing. The k-fold cross-validation technique was applied to increase the model’s accuracy by splitting 80% of the training data into 10 folds. In this manner, for each of the 10 experiments, nine folds were used for training and the remaining fold was used for model validation. Once the model was trained and validated, it was subjected to blind testing on the remaining 20% of the data set. The ML model learns to capture fluid flow behavior inside the core from the training data set. In terms of applicability of these ML models, two sets of experimental data were needed as input; the first was the analytically estimated Kr curves from the steady-state drainage coreflooding experiments, while the other was the Pc curves estimated from centrifuge or mercury injection capillary pressure (MICP) measurements. The trained/tested model was then able to estimate Kr curves based on the experimental results fed as input. Furthermore, to test the performance of the ML model when only one set of experimental data is available to an end user, a recurrent neural network (RNN) algorithm was trained/tested to predict Kr curves in the absence of Pc curves as an input. The performance of the three developed models (XGB, DNN, and RNN) was assessed using the values of the coefficient of determination (R2) along with the loss calculated during training/validation of the model. The respective crossplots along with comparisons of ground truth vs. artificial intelligence (AI)-predicted curves indicated that the model is capable of making accurate predictions with an error percentage between 0.2% and 0.6% on history-matching experimental data for all three tested ML techniques. This implies that the AI-based model exhibits better efficiency and reliability in determining Kr curves when compared to conventional methods. The developed ML models by no means replace the need to conduct drainage coreflooding or centrifuge experiments but act as an alternative to existing commercial platforms that are used to interpret experimental data to predict Kr curves. The two main advantages of the developed ML models are their capability of predicting Kr curves within a matter of a few minutes as well as with limited intervention from the end user. The results also include a comparison between classical ML approaches, shallow neural networks, and DNNs in terms of accuracy in predicting the final Kr curves. The research presented here is an extension of the state-of-the-art framework proposed by Mathew et al. (2021). However, the two main aspects of the current study are the application of deep learning for the prediction of Kr curves and the application of feature engineering. The latter not only reduces the training/testing time for the ML models but also enables the end user to obtain the final predictions with the least set of experimental data. The various models discussed in this research work currently focus on the prediction of Kr curves for drainage steady-state experiments; however, the work can be extended to capture the imbibition cycle as well.

647 Background: The primary treatment of most mNSTC is chemotherapy followed by surgery if the residual disease (RD) is &gt;1 cm. However, conventional imaging lacks the specificity to characterize the tissue, often leading to overtreatment. This study hypothesizes that integrating CT-driven radiomics features with plasma miR371 and miR375 will enhance the predictive accuracy of Machine Learning (ML) models to predict teratoma, viable germ cell (vGCT) and fibrosis/necrosis (F/N) in mNSTC patients with RD. Methods: 111 lesions from52 patients, including residual teratoma (n=57), F/N (n=33), vGCT (n=10), and additional seminoma (n=11) for training purposes were included, split into training (N=78) and test cohorts (N=33). Lesions were lymph nodes (n=87), lung (n=21), and brain (n=3) with a median size of 1.6 cm (Q1-Q3 interval=1.2-2.73 cm). 3D Slicer version 5.6.1 was used to segment the RD &gt; 1 cm (short axis) and extract radiomics features. Plasma miRNA levels before resection were measured by RT-PCR. Random Forest (RF), Support Vector Machine (SVM), Gradient Boosting (GB), and CatBoost (CB) ML models were evaluated to define the operating characteristics of radiomics alone (R-only) and in combination with miR371 (371) and/or miR375 (375) levels in predicting teratoma, vGCT and F/N. Results: For predicting teratoma, the best models were RF (R+375 and R+371+375), CB (R+371+375), and GB (R+371 and R+371+375). While adding miR371 or miR375 to R-only slightly improved AUC across models, the best results were achieved with the R+375+371 dataset. CB achieved AUCs ranging from 0.94 to 0.97 in training and 0.81 to 0.93 in test sets, with its highest AUC of 0.93 (95% CI: 0.78-0.97) on the R+375+371 dataset to differentiate all three classes. Similarly, GB demonstrated strong performance, achieving its highest AUC of 0.93 (95% CI: 0.79-0.96) on the R+375+371 dataset (Table). Conclusions: Integration of plasma miR371, miR375 and radiomics improved accuracy of predicting histologies across all ML models. These methods could be used to characterize the histology of RD in mNSTC patients to better inform treatment decisions. Further refinement, including incorporation of histological findings of the primary tumor, will be reported. AUC values of different ML algorithms on training and test sets. TRAINING SET TEST SET Model ±SD R R+375 R+371 R+375+371 Model (95% CI) R R+375 R+371 R+375+371 RF 0.93±0.05 0.95±0.04 0.95±0.03 0.96±0.04 RF 0.8(0.59-0.89) 0.85(0.72-0.93) 0.87(0.76-0.95) 0.91(0.78-0.95) SVM 0.84±0.06 0.84±0.09 0.89±0.11 0.89±0.09 SVM 0.72(0.54-0.80) 0.74(0.56-0.82) 0.83(0.69-0.92) 0.84(0.76-0.94) GB 0.94±0.04 0.91±0.08 0.95±0.05 0.97±0.03 GB 0.84(0.61-0.96) 0.89(0.77-0.97) 0.89(0.79-0.96) 0.93(0.79-0.96) CB 0.95±0.03 0.94±0.03 0.94±0.04 0.97±0.03 CB 0.81(0.6-0.93) 0.86(0.73-0.94) 0.89(0.78-0.97) 0.93(0.78-0.97)

In recent years, cardiovascular disease (CVD) has emerged as a leading cause of mortality worldwide, highlighting the urgent need for reliable predictive models to support early diagnosis and preventive care. This study conducts a comparative analysis of various machine learning models to assess CVD risk, focusing on their ability to accurately identify individuals at high risk based on clinical and demographic data. We evaluate multiple supervised learning algorithms, including logistic regression, random forests, support vector classifiers, K-nearest neighbors, gradient boosting, and AdaBoost, comparing their predictive performance in terms of accuracy, precision, recall, and F1-score. Additionally, we propose a novel hybrid model that combines Random Forest (RF) for feature selection and Deep Neural Networks (DNN) for classification, aiming to leverage the strengths of both approaches for more accurate CVD prediction. Our findings demonstrate that ensemble models such as random forests and gradient boosting achieve superior performance, with high accuracy (0.99) and balanced precision and recall values, outperforming simpler models like logistic regression and support vector classifiers. The hybrid model further enhances prediction accuracy, achieving 92.4% accuracy, 91.7% precision, 93.0% recall, and an AUC-ROC score of 96.0%. The analysis also underscores the importance of data preprocessing techniques, including normalization and handling of missing values, in optimizing model accuracy and stability. Notably, K-nearest neighbors also performed exceptionally well with a high F1-score across classes, highlighting its robustness for this task. This study provides a detailed examination of each model's strengths and limitations, including the proposed hybrid model, offering valuable insights for healthcare practitioners and data scientists in selecting effective machine learning models for CVD risk prediction. By integrating these models into healthcare systems, real-time risk prediction can be enhanced, ultimately supporting clinical decision-making and advancing personalized care in cardiovascular health. Keywords— cardiovascular disease, machine learning, risk assessment, predictive models, supervised learning, healthcare

Accurate detection and classification of brain tumors are essential for timely diagnosis and effective treatment planning. This study presents an integrated framework leveraging both machine learning (ML) and deep learning (DL) models for brain tumor detection and classification using MRI images. Two publicly available datasets are utilized: one for binary classification (tumor vs. no tumor) and another for multiclass classification (glioma, meningioma, and pituitary tumors). Comprehensive preprocessing steps, including resizing, feature extraction using the Gray Level Co-occurrence Matrix (GLCM), and feature selection via Chi-square testing, were employed to optimize the dataset for modeling. Machine learning models such as Decision Trees, K-Nearest Neighbors (KNN), Support Vector Machines (SVM), and AdaBoost were compared with deep learning architectures like Convolutional Neural Networks (CNNs) and the pre-trained VGG16 model. Hyperparameter optimization techniques, including grid search and the Adam optimizer, were used to enhance model performance. The models were evaluated using metrics such as accuracy, precision, recall, F1-score, Mean Squared Error (MSE), and Mean Absolute Error (MAE). Results indicate that the VGG16 model consistently outperformed other approaches, achieving high validation accuracy. This study highlights the potential of integrating ML and DL techniques for accurate and efficient brain tumor detection and classification, offering valuable tools for medical diagnostics.