Credit evaluation with a data mining approach based on gradient boosting decision tree

Universities annually accept new students at the beginning of the new school year. In the acceptance of prospective students on the Seleksi Prestasi Akademik Nasional Perguruan Tinggi Keagamaan Islam Negeri (SPAN PTKIN) di State Islamic University Of Sunan Ampel Surabaya, many prospective students who do not register will have an impact on income of the State Islamic University Of Sunan Ampel Surabaya institution. If the institution can find out early on the probability of a prospective student who will resign, then the management can take action to retain the prospective student. To overcome this, data mining classification can be carried out. The methods used in this classification are decision trees and naïve bayes. The number of students who did not re register compared to reregister resulted in the data being imbalanced. Data imbalances can affect the accuracy of the classification results. The imbalance of the data used can result in an unsuitable model. The solution to handle the data imbalance is to use the SMOTE and ADASYN oversampling methods. The purpose of this study was to compare performance of the SMOTE and ADASYN methods. The results show that the SMOTE method can balance the data in a balanced way compared to ADASYN. From the test results, the SMOTE method is more suitable to use than the ADASYN method because the ROCAUC SMOTE value is higher than ADASYN.

Bone age assessment is an important indicator of child maturity. Generally, a person evaluated for bone age is mostly in the puberty stage. Compared to toddlers and post-puberty stages, the data of bone age at the puberty stage are hence much easier to obtain. As a result, the amounts of bone age data collected at the toddlers and post-puberty stages are often less than that of bone age data at the pulpy stage. This causes a data imbalance problem that affects the prediction accuracy. To deal with this problem, a data imbalance immunity bone age assessment (DIIBAA) system is proposed which integrates a CNN-based autoencoder and CNN-based scoring network. The DIIBAA system builds three autoencoders for the bone age data of toddlers, puberty, and post-puberty stages. Since the three types of autoencoders do not interfere with each other, there is no data imbalance problem in the DIIBAA system. After that, the outputs of the three autoencoders are input to the scoring network, and the auto-encoder produced the image with the highest score is regarded as the final prediction result. In the experiments, imbalanced training data with a positive and negative sample ratio of 1:2 were used, which has been alleviated compared to the original very imbalanced data. Meanwhile, the DenseNet169-based autoencoders are employed in the experiments, and the obtained evaluation accuracies are improved.

R-GDORUS technology: Effectively solving the Raman spectral data imbalance in medical diagnosis

Colorectal Cancer (CRC) is a highly prevalent malignancy globally, and early prediction is crucial for improving prognosis. This study used a multidimensional CRC dataset (n=1000) provided by the Kaggle platform, which contains 14 clinical and lifestyle characteristics. First, data imbalance was mitigated through Random Oversampling (ROM) and standardization. Subsequently, a comprehensive evaluation was performed on seven baseline machine learning models, including Gradient Boosting Decision Tree (GBDT), eXtreme Gradient Boosting (XGBoost) and so on. Based on performance metrics such as accuracy and F1 score, GBDT and XGBoost were subsequently selected as the optimal base learners. Finally, the predictive probability features generated by the base learners are fed into the meta-learners such as Random Forest (RF), K Nearest Neighbor (KNN) and Multi-Layer Perceptron (MLP) for secondary modeling. The interpretability of the model is achieved through the Shapley Additive exPlanations (SHAP) value, which quantifies the marginal contribution of each feature to the prediction. Experiments show that the RF integration architecture based on GBDT and XGBoost baseline models has the best performance (accuracy of 0.9527 and AUC of 0.9923). SHAP analysis showed that Activity_Level and BMI were core predictors with limited contribution from gender, confirming the prioritization of exercise and weight management in CRC prevention. The framework demonstrated excellent robustness and maintained its predictive advantage even when inefficient base models e.g., Logistic Regression (LR) were introduced. This study provides an interpretable machine learning paradigm for CRC risk stratification with potential for clinical translation.

This research has three parts addressing classification, especially the imbalanced data problem, which is one of the most popular and essential issues in the domain of classification. The first part is to study the Adaptive Boosting (AdaBoost) algorithm. AdaBoost is an effective solution for classification, but it still needs improvement in the imbalanced data problem. This part proposes a method to improve the AdaBoost algorithm using the new weighted vote parameters for the weak classifiers. Our proposed weighted vote parameters are determined not only by the global error rate but also by the classification accuracy rate of the positive class, which is our primary interest. The imbalanced index of the data is also a factor in constructing our algorithms. The numeric studies show that our proposed algorithms outperform the traditional ones, especially regarding the evaluation criterion of the F--1 Measure. Theoretic proofs of the advantages of our proposed algorithms are presented. The second part treats the Relevance Vector Machine (RVM), which is a supervised learning algorithm extended from the Support Vector Machine (SVM) based on the Bayesian sparsity model. Compared with the regression problem, RVM classification is challenging to conduct because there is no closed-form solution for the weight parameter posterior. The original RVM classification algorithm uses Newton's method in optimization to obtain the mode of weight parameter posterior, then approximates it by a Gaussian distribution in Laplace's method. This original model would work, but it just applies the frequency methods in a Bayesian framework. This part first proposes a Generic Bayesian RVM classification, which is a pure Bayesian model. We conjecture that our algorithm achieves convergent estimates of the quantities of interest compared with the nonconvergent estimates of the original RVM classification algorithm. Furthermore, a fully Bayesian approach with the hierarchical hyperprior structure for RVM classification is proposed, which improves the classification performance, especially in the imbalanced data problem. The third part is an extended work of the second one. The original RVM classification model uses the logistic link function to build the likelihood, which makes the model hard to conduct since the posterior of the weight parameter has no closed-form solution. This part proposes the probit link function approach instead of the logistic one for the likelihood function in RVM classification, namely PRVM (RVM with the Probit link function). We show that the posterior of the weight parameter in our model follows the multivariate normal distribution and achieves a closed-form solution. A latent variable is needed in our algorithm to simplify the Bayesian computation greatly, and its conditional posterior follows a truncated normal distribution. Compared with the original RVM classification model, our proposed one is another pure Bayesian approach and it has a more efficient computation process. For the prior structure, we first consider the Normal-Gamma independent prior to propose a Generic Bayesian PRVM algorithm. Furthermore, the Fully Bayesian PRVM algorithm with a hierarchical hyperprior structure is proposed, which improves the classification performance, especially in the imbalanced data problem.

Whirlwind is a natural disaster that often occurs and is difficult to predict from some time before. Early identification is needed to prevent a lot of casualties and losses. Whirlwind caused by instability in the atmosphere. Instability in the atmosphere usually occurs at the beginning of the day and the whirlwind can be identified based on the upper air parameter which can represent atmospheric instability. The purpose of this research is to optimize SVM classification with SMOTE algorithm to handling problems in imbalanced data and this research can minimize casualties and losses or also be a breakthrough for disaster-prone areas to be given early warning. The process for identifying whirlwind has several stages, namely pre-processing, imbalanced data handling, and classification. Pre-processing is normalized data. Whirlwind data has a classification problem, namely inter-class data that is not balanced so it needs to be corrected using the SMOTE Algorithm. Research on the identification of whirlwind using a combination of SMOTE-SVM produces the best accuracy is 98.8 %. When compared to data without the SMOTE algorithm the results obtained are better if the SMOTE method is applied. The specificity value is also better when given the SMOTE method. Based on these results it can be concluded that SMOTE can overcome the problem of imbalanced data in the upper air data by increasing the value of the classification of the whirlwind.

The integration of industrialization and informatization has exposed industrial control systems (ICSs) to increasingly serious security challenges. Currently, the mainstream method to protect the security of ICSs is intrusion detection system (IDS) based on deep-learning. However, these methods depend on a massive amount of high-quality data. Owing to the characteristics and protocol limitations, ICSs data usually experience low-quality and data imbalance problems, which significantly affects the accuracy of IDS.In this study, an IDS for ICS that combines data expansion algorithm and CNN was proposed. A novel normalized neighborhood weighted convex combined random sample (NNW-CCRS) oversampling algorithm was designed, which automatically attenuates the effects of noise and expanding imbalanced data to produce balanced ICS datasets. By reducing the impact of imbalanced ICS data on IDSs, our system effectively protects the security of ICS. Secure Water Treatment dataset (SWaT) was used for experimental validation. The experimental results confirmed that the accuracy of the proposed system improved by approximately 20%, compared to the ICS without data expansion.

The intelligent diagnosis of rotating machinery with big data has been widely studied. However, due to the variability of working conditions and difficulty in marking fault samples, it is difficult to obtain enough high-quality fault marking data for training bearing fault diagnosis models in practical industrial application scenarios. Aiming at the problem of training data imbalance caused by lack of fault samples, a novel metalearning fault diagnosis method (MOFD) is proposed to get the bearing fault diagnosis solution under data imbalance. Firstly, in order to enhance the variety of fault samples, a Feature Space Density Adaptive Synthetic Minority Oversampling Technique (FSDA-SMOTE) is proposed in this paper, which takes the density difference of minority samples in the spatial domain within the class as the constraint of local neighbor similarity to generate new fault samples for data augmentation. In addition, in order to strengthen the model’s learning ability and diagnosis performance under limited fault samples, a residual-attention convolutional neural network (RA-CNN) was constructed to identify the deep features of fault signals, and a metalearning strategy based on parameter gradient optimization was applied to RA-CNN for refining the learning process of the diagnosis model. Finally, the reliability of the proposed method is verified through experimental analysis of public bearing dataset.

Prediction of body mass index status from voice signals based on machine learning for automated medical applications

Machine learning-based landslide susceptibility assessment with optimized ratio of landslide to non-landslide samples

PM 2.5, as an important index about air pollution, whose form and composition is very complicated, has a negative influence on humankind. The paper, according to the meteorological data from Wu Han environmental air automatic monitoring station, conducts an quantitative analysis of independence and the 6 standards of AQI ,based on VAR and Granger causality test, draws a conclusion that inhalable particles and ozone is not the source of the increase of PM 2.5.Finally. The paper establishes VCR model between PM 2.5 and CO, SO2, NO2 ,introduces the impulse response function, explores a random perturbation of a standard deviation shock effects on different indicators for monitoring. KeywordsPM2.5; air pollution; VAR model. The status and variation of air quality influences human survival, life and development. It is one of the hottest issues that draw much attention of the environmental protection and the national people. Compared to the air pollution index, the air quality index can monitor the air quality status more specifically. Among these indexes, PM2.5 is a very important index. How to carry out the relevant analysis of the PM2.5 factors and discuss the PM2.5’s distribution and variation rules through the collection of PM2.5 data, it is beneficial for the prediction, evaluation and empirical analysis and have theoretical and practical significance for making high efficient feasible emission reduction plan. I. DATA SOURCE AND PREPROCESSING A. Source of Data To analyze the influence of PM2.5 with the monitoring data of Wuhan selected from January 1, 2013 to August 26, 2013 in the attachment. B. Data pre-procession In the attachment, there is some missing data about the inhalable particles on April 29, 2013 and July 7, 2013. Besides, there are some common situations of data missing that is described in the attachment. Therefore, firstly, it introduces the basic theory and techniques of the statistical treatment of data missing. Little and Rubin[1] defined the following three different types of data-missing mechanisms; 1)Missing Completely At Random(MCAR), the data missing is irrelevant to the incomplete variable and complete variable 2)Missing At Random(MAR), the data missing completely rely on the complete variable 3)Not Missing at Random, the data missing of incomplete variable rely on the incomplete variable itself, this kind of data missing shall not be neglected. Where the complete variable refers to the variable (attribute) has no missing value. Otherwise, it will be incomplete variable. Usually, the process of missing data uses the filling method of prediction to maximize the utilization of the data obtained and construct the prediction model. In order to ensure the quality of the missing data’s quality through the other data of the corresponding points, it adopts the filling method of prediction to fill the missing data and marked the inhalable particle as inhpar and imports the selected data of the period from January 1, 2013 to April 25, 2013 into Eviews 6.0 software, the results of the data such as stationary test is in the attachment 1, the fitted equation is: ( ) 41.8666 0.5627* ( 1) (0.3169) (5.3167) (7.4048) inhpar t inhpar t = + − The predicted result on April 29, 2013 is 87.4453, rounded off to 87.it is the same as the data before June 7, 2013, the fitting result of the data is ( ) 36.0974 0.4024* ( 1) (0.2006) (5.7782) ( 6.2158) inhpar t inhpar t = − − − Correspondingly, the supplemental result is 74.3438, rounded off to 74. II. THE CORRELATIVE AND INDEPENDENT QUANTITATIVE ANALYSIS OF SIX MONITORING INDEXES IN AQI For the test of the correlation and independence among the monitoring indexes, it has great importance for judging the reasonability of the monitoring index’s setting and the mechanism analysis of the formation and propagation of PM 2.5 etc. A. Quantitative Analysis of Correlation The correlation is aim at describing the overlapped information and the dependence between different indexes, Pearson’s correlation analysis is one of the most important technical methods in the multivariate data analysis. It selects 238 pairs observed data in 6 monitoring indexes of the attachment 1, leading-in Eviews6.0, respectively named as sulfur dioxide (SO2), nitrogen dioxide (NO2), particulate matter (Inhpar), carbon monoxide (CO), smell (O3), PM 2.5 (PM), the results are as follows: 2015 AASRI International Conference on Industrial Electronics and Applications (IEA 2015) © 2015. The authors Published by Atlantis Press 308 TABLE I. 6 MONITORING INDEX DATA DESCRIPTIVE STATISTICS PM SO2 O3 NO2 INHPAR CO Mean 108.737 28.1891 52.0511 63.1051 79.1610 25.8067 Median 84.5000 24.0000 46.0000 59.0000 74.0000 23.0000 Maximum 426.000 85.0000 156.000 123.000 282.000 101.000 Minimum 22.0000 5.00000 3.00000 20.0000 18.0000 10.0000 Std.Dev 74.6960 17.7239 33.7949 24.4750 33.1286 14.1098 Skewness 1.32742 0.68193 0.73441 0.50745 1.79423 1.96787 Kurtosis 4.43919 2.54436 2.96273 2.18114 9.87485 8.16541 Jarque-Bera 90.4344 20.5049 21.4088 16.8634 591.383 418.201 Probability 0.00000 0.00004 0.00002 0.00021 0.00000 0.00000 Sum 25927.0 6709.0

Semantic scene classification, robotic state recognition, and many other real-world applications involve multi-label classification with imbalanced data. In this paper, we address these problems by using an enrichment process in neural net training. The enrichment process can manage the imbalanced data and train the neural net with high classification accuracy. Experimental results on a robotic arm controller show that our method has better generalization performance than traditional neural net training in solving the multi-label and imbalanced data problems.

Although imbalanced data have been studied for many years, the problem of data imbalance is still a major problem in the development of machine learning and artificial intelligence. The development of deep learning and artificial intelligence has further expanded the impact of imbalanced data, so studying imbalanced data classification is of practical significance. We propose an image oversampling algorithm based on the influence function and sample weights. Our scheme not only synthesizes high-quality minority class samples but also preserves the original features and information of minority class images. To address the lack of visually reasonable features in SMOTE when synthesizing images, we improve the pre-training model by removing the pooling layer and the fully connected layer in the model, extracting the important features of the image by convolving the image, executing SMOTE interpolation operation on the extracted important features to derive the synthesized image features, and inputting the features into a DCGAN network generator, which maps these features into the high-dimensional image space to generate a realistic image. To verify that our scheme can synthesize high-quality images and thus improve classification accuracy, we conduct experiments on the processed CIFAR10, CIFAR100, and ImageNet-LT datasets.

The parameters of support vector machines (SVMs) such as kernel parameters and the penalty parameter have a great influence on the accuracy and complexity of the classification models. In the past, different evolutionary optimization algorithms were employed for optimizing SVMs; in this paper, we propose a social ski-driver (SSD) optimization algorithm which is inspired from different evolutionary optimization algorithms for optimizing the parameters of SVMs, with the aim of improving the classification performance. To cope with the problem of imbalanced data which is one of the challenging problems for building robust classification models, the proposed algorithm (SSD-SVM) was enhanced to deal with imbalanced data. In this study, eight standard imbalanced datasets were used for testing our proposed algorithm. For verification, the results of the SSD-SVM algorithm are compared with grid search, which is a conventional method of searching parameter values, and particle swarm optimization (PSO). The experimental results show that the SSD-SVM algorithm is capable of finding near-optimal values of SVMs parameters. The results also demonstrated high classification performance compared to the PSO algorithm.

In machine learning, classification and regression are two of the most noteworthy key topics since they occur extensively in numerous real-world applications. However, real life data is hardly ever found balanced, rather skewed data is the common occurrence. This poses some serious challenges to the standard techniques of classification and regression. The performance and effectiveness of these techniques are substantially affected by overfitting, creating a bias towards the majority class. In recent years, quite a few number of methods have been introduced for classification of imbalanced data. But most of them are designed for binary classes and it is difficult or inefficient to extend them for multiple classes. Moreover, data imbalance problem occurs frequently in regression analysis too, with only a handful of algorithms robust enough to tackle to this problem. In this paper, we propose an effective recursive method for multi-class classification with imbalanced data. Our proposed algorithm partitions and balances the data, and is applied recursively coupled with ensemble techniques. Furthermore, we also extend our proposed method to solve the data imbalance problem in regression analysis. Experimental results demonstrate that the proposed recursive technique is effective and improves the performance when compared to existing methods for classification and regression with imbalanced distribution.