Three data mining models to predict bank telemarketing

Recurrent spontaneous miscarriage refers to the repeated loss of two or more clinically detected pregnancies occurring within 24 weeks of gestation. No identifiable cause has been identified for nearly 50% of these cases. This group is referred to as idiopathic recurrent spontaneous miscarriage (IRSM) or miscarriage of unknown origin. Due to lack of robust scientific evidence, guidelines on the diagnosis and management of IRSM are not well defined and often contradictory. This motivates us to explore the vibrational fingerprints of endometrial tissue in these women. Endometrial tissues were collected from women undergoing IRSM (n = 20) and controls (n = 20) corresponding to the window of implantation. Attenuated total reflectance-Fourier transform infrared (ATR-FTIR) spectra were obtained within the range of 400–4000 cm−1 using Agilent Cary 630 FTIR spectrometer. Raman spectra were also generated within the spectral window of 400–4000 cm−1 using Thermo Fisher Scientific, DXR Raman spectrophotometer. Based on the limited molecular information provided by a single spectroscopic tool, fusion strategy combining Raman and ATR-FTIR spectroscopic data of IRSM is proposed. The significant features were extracted applying principal component analysis (PCA) and wavelet threshold denoising (WTD) and fused spectral data used as input into support vector machine (SVM), adaptive boosting (AdaBoost) and decision tree (DT) models. Altered molecular vibrations associated with proteins, glutamate, and lipid metabolism were observed in IRSM using Raman spectroscopy. FTIR analysis indicated changes in the molecular vibrations of lipids and proteins, collagen dysregulation and impaired glucose metabolism. Combination of both spectroscopic data using mid-level fusion (MLF: 92% using AdaBoost and DT models) and high-level fusion (HLF: 92% using SVM models) methods showed improved IRSM classification accuracy as compared to individual spectral models. Our results indicate that spectral fusion technology hold promise in enhancing diagnostic accuracy of IRSM in clinical settings. Validation of these findings in a larger patient population is underway.

Bond rating is regarded as an important event for measuring financial risk of companies and for determining the investment returns of investors. As a result, it has been a popular research topic for researchers to predict companies' credit ratings by applying statistical and machine learning techniques. The statistical techniques, including multiple regression, multiple discriminant analysis (MDA), logistic models (LOGIT), and probit analysis, have been traditionally used in bond rating. However, one major drawback is that it should be based on strict assumptions. Such strict assumptions include linearity, normality, independence among predictor variables and pre-existing functional forms relating the criterion variablesand the predictor variables. Those strict assumptions of traditional statistics have limited their application to the real world. Machine learning techniques also used in bond rating prediction models include decision trees (DT), neural networks (NN), and Support Vector Machine (SVM). Especially, SVM is recognized as a new and promising classification and regression analysis method. SVM learns a separating hyperplane that can maximize the margin between two categories. SVM is simple enough to be analyzed mathematical, and leads to high performance in practical applications. SVM implements the structuralrisk minimization principle and searches to minimize an upper bound of the generalization error. In addition, the solution of SVM may be a global optimum and thus, overfitting is unlikely to occur with SVM. In addition, SVM does not require too many data sample for training since it builds prediction models by only using some representative sample near the boundaries called support vectors. A number of experimental researches have indicated that SVM has been successfully applied in a variety of pattern recognition fields. However, there are three major drawbacks that can be potential causes for degrading SVM's performance. First, SVM is originally proposed for solving binary-class classification problems. Methods for combining SVMs for multi-class classification such as One-Against-One, One-Against-All have been proposed, but they do not improve the performance in multi-class classification problem as much as SVM for binary-class classification. Second, approximation algorithms (e.g. decomposition methods, sequential minimal optimization algorithm) could be used for effective multi-class computation to reduce computation time, but it could deteriorate classification performance. Third, the difficulty in multi-class prediction problems is in data imbalance problem that can occur when the number of instances in one class greatly outnumbers the number of instances in the other class. Such data sets often cause a default classifier to be built due to skewed boundary and thus the reduction in the classification accuracy of such a classifier. SVM ensemble learning is one of machine learning methods to cope with the above drawbacks. Ensemble learning is a method for improving the performance of classification and prediction algorithms. AdaBoost is one of the widely used ensemble learning techniques. It constructs a composite classifier by sequentially training classifiers while increasing weight on the misclassified observations through iterations. The observations that are incorrectly predicted by previous classifiers are chosen more often than examples that are correctly predicted. Thus Boosting attempts to produce new classifiers that are better able to predict examples for which the current ensemble's performance is poor. In this way, it can reinforce the training of the misclassified observations of the minority class. This paper proposes a multiclass Geometric Mean-based Boosting (MGM-Boost) to resolve multiclass prediction problem. Since MGM-Boost introduces the notion of geometric mean into AdaBoost, it can perform learning process considering the geometric mean-based accuracy and errors of multiclass. This study applies MGM-Boost to the real-world bond rating case for Korean companies to examine the feasibility of MGM-Boost. 10-fold cross validations for threetimes with different random seeds are performed in order to ensure that the comparison among three different classifiers does not happen by chance. For each of 10-fold cross validation, the entire data set is first partitioned into tenequal-sized sets, and then each set is in turn used as the test set while the classifier trains on the other nine sets. That is, cross-validated folds have been tested independently of each algorithm. Through these steps, we have obtained the results for classifiers on each of the 30 experiments. In the comparison of arithmetic mean-based prediction accuracy between individual classifiers, MGM-Boost (52.95%) shows higher prediction accuracy than both AdaBoost (51.69%) and SVM (49.47%). MGM-Boost (28.12%) also shows the higher prediction accuracy than AdaBoost (24.65%) and SVM (15.42%)in terms of geometric mean-based prediction accuracy. T-test is used to examine whether the performance of each classifiers for 30 folds is significantly different. The results indicate that performance of MGM-Boost is significantly different from AdaBoost and SVM classifiers at 1% level. These results mean that MGM-Boost can provide robust and stable solutions to multi-classproblems such as bond rating.

The implementation of Data Mining in the education sector aims to develop methods that are able to discover valuable knowledge from data generated in the educational environment. This can be used to increase learning efficiency by paying more attention to students who are predicted to have low grades. However, in its application, each algorithm shows different performance depending on the attributes and dataset used. In this study, a dataset of semester grades and final school exam scores was used. Some of the prediction techniques used are decision trees, support vector machines, and neural networks. Of the four scenarios for the science major at SMAN 2 and SMAN 3 Pangkalpinang with 3 different models, the Mean Squared Error value shows that the test results are in accordance with the testing dataset and can be used as predictions of students' final grades, namely the decision tree model and support vector machine. For the Social Sciences major at SMAN 2 and SMAN 3 Pangkalpinang with 3 different models, the Mean Squared Error value shows that the test results are in accordance with the testing dataset and can be used as a prediction of students' final grades, namely the support vector machine model.

Magnetocardiography (MCG) is a novel non-invasive technique that detects subtle magnetic fields generated by cardiomyocyte electrical activity, offering sensitive detection of myocardial ischemia. This study aimed to assess the ability of MCG to predict impaired myocardial perfusion using single-photon emission computed tomography (SPECT). A total of 112 patients with chest pain underwent SPECT and MCG scans, from which 65 MCG output parameters were analyzed. Using least absolute shrinkage and selection operator (LASSO) regression to screen for significant MCG variables, three machine learning models were established to detect impaired myocardial perfusion: random forest (RF), decision tree (DT), and support vector machine (SVM). The diagnostic performance was evaluated based on the sensitivity, specificity, accuracy, positive predictive value (PPV), negative predictive value (NPV), and area under the receiver operating characteristic curve (AUC). Five variables, the ratio of magnetic field amplitude at R-peak and positive T-peak (RoART+), R and T-peak magnetic field angle (RTA), maximum magnetic field angle (MAmax), maximum change in current angle (CCAmax), and change positive pole point area between the T-wave beginning and peak (CPPPATbp), were selected from 65 automatic output parameters. RTA emerged as the most critical variable in the RF, DT, and SVM models. All three models exhibited excellent diagnostic performance, with AUCs of 0.796, 0.780, and 0.804, respectively. While all models showed high sensitivity (RF = 0.870, DT = 0.826, SVM = 0.913), their specificity was comparatively lower (RF = 0.500, DT = 0.300, SVM = 0.100). Machine learning models utilizing five key MCG variables successfully predicted impaired myocardial perfusion, as confirmed by SPECT. These findings underscore the potential of MCG as a promising future screening tool for detecting impaired myocardial perfusion. ChiCTR2200066942, https://www.chictr.org.cn/showproj.html?proj=187904.

Near infrared spectroscopy was used in discrimination of intact bovine teeth in terms of animal sex, diet, tooth type and place of origin. Discriminant Analysis (DA) models were developed and tested using a stratified random 70:30 data split to calibration and test sets. Of the discriminant techniques of Soft Independent Modelling of Class Analogy (SIMCA), Partial Least Squares – DA, Artificial Neural Networks (ANN) and Support Vector Machine (SVM), SVM and PLS-DA models performed best in most instances, with pretreatment choice impacting technique success. For test set prediction of animal sex, an accuracy of 95% was achieved using a PLS-DA model, and similar performance achieved using a SVM model. SVM, ANN and SIMCA models predicted three categories of tooth type (deciduous, permanent unerupted and erupted) with similar accuracies (89, 79 and 82%, respectively), however, permanent unerupted teeth were conflated with permanent erupted teeth. Prediction accuracy for ANN, SVM and PLS-DA models were also similar for discrimination of teeth from animals on a grain versus grass diet (PLS-DA 82%). There was considerable variation between models in prediction of geographic origin. Of the seven locations, two were discriminated with true positive rate around 70% using SVM and ANN models. Potential applications of this technology are discussed.

Flooding causes several threats with outcomes which include peril to human and animal life, damage to property, and adversity to agricultural fields. Therefore, flood prediction is of prime importance for reducing loss of life and devastation to property. To model complex nature of hydrologic processes artificial neural network (ANN) tool is effectively being utilized for modelling different nonlinear relationships, and has proved to be an appropriate method for flood prediction. Present study investigated relative accuracy of radial basis function neural network (RBFNN) and support vector machine (SVM) models combined with Firefly Algorithm (FA) in predicting river flood discharge and contrasted with that of regular ANN, RBFNN and SVM models. Monthly river flow data of Silchar and Dholai stations located in Cachar district of Assam, India are utilized for the present study. For assessing model performance, coefficient of determination (R2), mean square error (MSE) and root mean square error (RMSE) were measured. Evaluation of outcomes shows that both RBF-FA (radial basis function - firefly algorithm) and SVM-FA (support vector machine - firefly algorithm) hybrid models give more precise forecasting results than RBFNN, FFBPNN (feed forward back propagation neural network) and SVM models. Yet, it can be observed that SVM-FA model give better forecasting outputs with R2 value 0.9818 than RBF-FA model. Results also reveal that simple SVM model performs marginally better than simple ANN model.

BackgroundThis study aims to develop a screening model for subjective cognitive decline (SCD) based on machine learning techniques.MethodsA retrospective cohort study collected clinical psychological factor data from the “Active Health” screening app under the National Key R&D Program. The final dataset included 598 samples, with an SCD incidence rate of 26.12%. The data were randomly divided into a training set (n = 418). A validation set (n = 180) at a ratio of 7:3. In the training set, prediction models for SCD were constructed using logistic regression (LR), Naive Bayes, support vector machine (SVM), decision tree, and neural network algorithms. Model performance on the validation set was assessed by calculating the area under the ROC curve (AUC), accuracy, sensitivity, specificity, precision, recall, and F1 score. SHAP values were used for model interpretability analysis.ResultsThe SVM model showed good performance in the training set, with an AUC of 0.82, indicating strong predictive ability. Information Overload (IO), Self-Perception (SP), Energy Level (EL), Depressive Emotion (DE), Gender (SEX), Risk Decision (RD), and Short-Term Memory (STM) were important feature variables for SCD occurrence.ConclusionThis study successfully developed an SVM-based model for screening the risk of SCD. The SVM model demonstrated superior predictive performance compared to Naïve Bayes, Decision Tree, Neural Network, and traditional LR models.

This study tries to propose a wine recommendation algorithm based on partitioning and Stacking Integration Strategy for Chinese wine consumers. The approaches follow the idea of partitioning, decomposing traditional recommendation task into several subtasks according to wine attributes, using neural network, support vector machine (SVM), decision tree, random forest, optimized random forest, Adaboost and XGBoost as recommendation models. Then, based on Stacking integration method, five models are screened out for each recommendation index as the base classifier, and the decision tree or logistic regression model is selected as the meta-learner to construct a two-layer Stacking integration framework. Finally, the optimal recommendation algorithm be built for recommendation subtasks according to the prediction accuracy. The result showed that the Stacking integrated recommendation model was suitable for the recommendation of eight attributes including colour, sweetness, foamability, mouthfeel, aroma type, year, packaging and brand, while SVM model was suitable to recommend aroma concentration and price, and the XGboost model was most appropriate for origin. This study would subserve consumers to choose the wine more easily and conveniently and provide support for wine companies to improve customer satisfaction with consumer services. The study expands the approach of concerning research and proposes a specific multi-model recommendation strategy based on artificial intelligence models to recommend multiattribute commodities.

Breast cancer remains a leading cause of mortality among women globally, necessitating accurate and early diagnosis techniques. This study explores the effectiveness of Support Vector Machine (SVM) techniques for diagnosing breast cancer, utilizing the Object-Oriented Analysis and Design Method (OOADM) for system development. The research employed the Wisconsin Breast Cancer Dataset from the UCI Machine Learning Repository, comprising ten features. The dataset was divided into 80% for training and 20% for testing the SVM model. Performance metrics such as classification accuracy, Area Under the Curve (AUC), sensitivity, specificity, and precision were used to evaluate the SVM model, which was also compared against a Decision Tree (DT) model. The results indicated that the SVM model achieved superior performance with an accuracy of 94%, AUC of 98%, sensitivity of 95%, specificity of 87%, and precision of 93%. In comparison, the DT model showed an accuracy of 89%, AUC of 95%, sensitivity of 90%, specificity of 85%, and precision of 90%. The findings underscore the potential of SVM in enhancing breast cancer diagnostic accuracy, thereby supporting early detection and treatment.

Classification of heroin, methamphetamine, ketamine and their additives by attenuated total reflection-Fourier transform infrared spectroscopy and chemometrics.

BackgroundCranial, facial, nasal, and maxillary widths have been shown to be significantly affected by the individual’s sex. The present study aims to use measurements of dental arch and maxillary skeletal base to determine sex, employing supervised machine learning.Materials and methodsMaxillary and mandibular tomographic examinations from 100 patients were analyzed to investigate the inter-premolar width, inter-molar width, maxillary width, inter-pterygoid width, nasal cavity width, nostril width, and maxillary length, obtained through Cone Beam Computed Tomography scans. The following machine learning algorithms were used to build the predictive models: Logistic Regression, Gradient Boosting Classifier, K-Nearest Neighbors (KNN), Support Vector Machine (SVM), Multi-Layer Perceptron Classifier (MLP), Decision Tree, and Random Forest Classifier. A 10-fold cross-validation approach was adopted to validate each model. Metrics such as area under the curve (AUC), accuracy, recall, precision, and F1 Score were calculated for each model, and Receiver Operating Characteristic (ROC) curves were constructed.ResultsUnivariate analysis showed statistical significance (p < 0.10) for all skeletal and dental variables. Nostril width showed greater importance in two models, while Inter-molar width stood out among dental measurements. The models achieved accuracy values ranging from 0.75 to 0.85 on the test data. Logistic Regression, Random Forest, Decision Tree, and SVM models had the highest AUC values, with SVM showing the smallest disparity between cross-validation and test data for accuracy metrics.ConclusionTransverse dental arch and maxillary skeletal base measurements exhibited strong predictive capability, achieving high accuracy with machine learning methods. Among the evaluated models, the SVM algorithm exhibited the best performance. This indicates potential usefulness in forensic sex determination.

Background and aimsThe birth weight of a newborn is a crucial factor that affects their overall health and future well-being. Low birth weight (LBW) is a widespread global issue, which the World Health Organization defines as weighing less than 2,500 g. LBW can have severe negative consequences on an individual’s health, including neonatal mortality and various health concerns throughout their life. To address this problem, this study has been conducted using BDHS 2017–2018 data to uncover important aspects of LBW using a variety of machine learning (ML) approaches and to determine the best feature selection technique and best predictive ML model.MethodsTo pick out the key features, the Boruta algorithm and wrapper method were used. Logistic Regression (LR) used as traditional method and several machine learning classifiers were then used, including, DT (Decision Tree), SVM (Support Vector Machine), NB (Naïve Bayes), RF (Random Forest), XGBoost (eXtreme Gradient Boosting), and AdaBoost (Adaptive Boosting), to determine the best model for predicting LBW. The model’s performance was evaluated based on the specificity, sensitivity, accuracy, F1 score and AUC value.ResultsResult shows, Boruta algorithm identifies eleven significant features including respondent’s age, highest education level, educational attainment, wealth index, age at first birth, weight, height, BMI, age at first sexual intercourse, birth order number, and whether the child is a twin. Incorporating Boruta algorithm’s significant features, the performance of traditional LR and ML methods including DT, SVM, NB, RF, XGBoost, and AB were evaluated where LR, had a specificity, sensitivity, accuracy and F1 score of 0.85, 0.5, 85.15% and 0.915. While the ML methods DT, SVM, NB, RF, XGBoost, and AB model’s respective accuracy values were 85.35%, 85.15%, 84.54%, 81.18%, and 84.41%. Based on the specificity, sensitivity, accuracy, F1 score and AUC, RF (specificity = 0.99, sensitivity = 0.58, accuracy = 85.86%, F1 score = 0.9243, AUC = 0.549) outperformed the other methods. Both the classical (LR) and machine learning (ML) models’ performance has improved dramatically when important characteristics are extracted using the wrapper method. The LR method identified five significant features with a specificity, sensitivity, accuracy and F1 score of 0.87, 0.33, 87.12% and 0.9309. The region, whether the infant is a twin, and cesarean delivery were the three key features discovered by the DT and RF models, which were implemented using the wrapper technique. All three models had the identical F1 score of 0.9318. However, “child is twin” was recognized as a significant feature by the SVM, NB, and AB models, with an F1 score of 0.9315. Ultimately, with an F1 score of 0.9315, the XGBoost model recognized “child is twin” and “age at first sex” as relevant features. Random Forest again beat the other approaches in this instance.ConclusionsThe study reveals Wrapper method as the optimal feature selection technique. The ML method outperforms traditional methods, with Random Forest (RF) being the most effective predictive model for Low-Birth-Weight prediction. The study suggests that policymakers in Bangladesh can mitigate low birth weight newborns by considering identified risk factors.

BackgroundWith diabetes incidence growing globally and metformin still being the first-line for its treatment, metformin’s toxicity and overdose have been increasing. Hence, its mortality rate is increasing. For the first time, we aimed to study the efficacy of machine learning algorithms in predicting the outcome of metformin poisoning using two well-known classification methods, including support vector machine (SVM) and decision tree (DT).MethodsThis study is a retrospective cohort study of National Poison Data System (NPDS) data, the largest data repository of poisoning cases in the United States. The SVM and DT algorithms were developed using training and test datasets. We also used precision-recall and ROC curves and Area Under the Curve value (AUC) for model evaluation.ResultsOur model showed that acidosis, hypoglycemia, electrolyte abnormality, hypotension, elevated anion gap, elevated creatinine, tachycardia, and renal failure are the most important determinants in terms of outcome prediction of metformin poisoning. The average negative predictive value for the decision tree and SVM models was 92.30 and 93.30. The AUC of the ROC curve of the decision tree for major, minor, and moderate outcomes was 0.92, 0.92, and 0.89, respectively. While this figure of SVM model for major, minor, and moderate outcomes was 0.98, 0.90, and 0.82, respectively.ConclusionsIn order to predict the prognosis of metformin poisoning, machine learning algorithms might help clinicians in the management and follow-up of metformin poisoning cases.

Aggregate stability is a useful soil physical dynamic index of soil resistivity to surface wind and water erosion in all ecosystems, especially, in arid and semi-arid regions. Two machine learning techniques including support vector machines (SVMs) and artificial neural networks (ANNs) were used to develop predictive models for the estimation of geometric mean diameter (GMD) of soil aggregates. An empirical multiple linear regression (MLR) model was also constructed as the benchmark to compare their performances. Furthermore, the influence of feature space dimension reduction using parallel genetic algorithm (PGA) on the prediction accuracy of all investigated techniques was evaluated. The ANN model achieved greater accuracy in GMD prediction as compared to the MLR and SVM models. The obtained ERROR% value in GMD prediction using the ANN model was 6.9%, while it was 15.7 and 10.6% for the MLR and SVM models, respectively. Feature selection using PGA improved the prediction accuracy of all investigated techniques. The coefficient of determination (R2) values between the measured and the predicted GMD values using PGA-based MLR, SVM, and ANN models increased by 20.0, 12.2, and 8.8% in comparison with the proposed MLR, SVM, and ANN models. In conclusion, it appears that the PGA-based ANN model could be considered as an alternative to conventional regression models for the GMD prediction.

Online Peer-to-Peer (P2P) lending has achieved explosive development recently, which could be beneficial to both sides of individual lending. In this study, a data mining (DM) approach to predict the performance of P2P loan before funded is proposed. Using data from the Lending Club, we explore the characteristics of loan and its applicant and use random forest to do the feature selection in the modeling phase. The Difference from other risk prediction models is that the prediction is classified into three or four categories, rather than just two the default and not default classes. Then we compare five DM models: two decision trees (DTs), two neural networks (NNs) and one support vector machine (SVM) and use two metrics: average percent hit rate and area of the lift cumulative curve to evaluate the prediction results. The Empirical result shows that the term of loan, annual income, the amount of loan, debt-to-income ratio, credit grade and revolving line utilization play an important role in loan defaults. And SVM, Classification and Regression Tree (CART) and Multi-layer perceptron (MPL)'s prediction performance are almost equal.