Regularized Stacked Autoencoder with Dropout-Layer to Overcome Overfitting in Numerical High-Dimensional Sparse Data

Area Under the ROC Curve (AUC) is an objective indicator of evaluating classification performance for imbalanced data. In order to deal with large-scale imbalanced streaming data, especially high-dimensional sparse data, this paper proposes a Sparse Stochastic Online AUC Optimization (SSOAO) method. Specifically, we first turn the standard online AUC optimization problem into a stochastic saddle point problem, then optimizing AUC by solving stochastic saddle point problem through AdaGrad optimizer. A sparse regularization term is also added for learning sparse data with high dimension. Comprehensive evaluation has been carried out on the recent benchmark. The experimental results show that the proposed SSOAO has the comparable performance on low-dimensional data, and outperforms other popular AUC optimization methods on high-dimensional sparse imbalanced streaming data. Both time and space complexity for model updating are reduced from \(O(d^2)\) to O(d), which equal to the data dimension.

Fuzzy partition based soft subspace clustering and its applications in high dimensional data

Since multi-omics data (high-dimensional data, including many variables, like genomics, transcriptomics and proteomics) are high-dimensional (have many variables), heterogeneous (are data of different types), and interdependent (variables can affect each other), there is a significant challenge in integrating various data types in a synergistic manner (without, it may not be feasible). This paper is an attempt to tackle the problem of predicting drug sensitivity by offering a hybrid machine learning (ML, algorithms that learn patterns by analyzing data) and deep learning (DL, a type of ML based on neural network architectures to learn patterns of data) model of not only predicting drug sensitivity, but also using Genomics of Drug Sensitivity in Cancer (GDSC) data. The method involves applying sophisticated preprocessing, strict feature selection techniques (Random Forest, tree-based algorithm; LASSO regression, feature selection that chooses features by pruning irrelevant ones; Mutual Information which is a dependency measure between variables) as well as a hybrid model design favoring classical ML models (Random Forest, SVM--Support Vector Machine) over the deep learning style (LSTM-Long short-term memory networks, a sequence-based version of recurrent neural network). The metrics of the evaluation give a very high R 2 (coefficient of determination, a measure of model fit) of 0.991, and the evaluations are very small MAE ( Mean absolute error: square root of average squared error), RMSE (Root mean squared error: square root of average squared error), and MSE ( Mean squared error: square root of average squared error), resulting in approximately 12 percent higher accuracy than conventional approximations: Ridge Regression, XGBoost (Extreme gradient boosting), and Random Forest alone. This also shows the strength (capacity to work with diverse circumstances), generalizability (capacity to be used in new information), and enhanced predictive capabilities of the hybrid method of modeling nonlinear associations in multi-omics information, thus enhancing precision medicine (health care customized to patients in terms of genetic, environmental, and lifestyle determinants) uses.

A Comprehensive Survey on Deep Graph Representation Learning

In this paper, we propose an incremental clustering algorithm called INCLUS for high dimensional sparse transactional data using a newly defined similarity measure and a notion of cluster representatives based on locally frequent items of each cluster. INCLUS seeks structures in transactional data with respect to the support and similarity constraints specified by the users. The effectiveness and the order-independence property of INCLUS are empirically studied and compared with two state- of-art algorithms. Though it is a one-pass algorithm without any iterative refinement, INCLUS is not only effective and scalable, but also insensitive to the order of transactions, which is a crucial property for an incremental algorithm. globally frequent items are neglected during clustering, clusters embedded in such transactions cannot be discovered. In this paper, we define a new notion of cluster representative based on frequent items within each cluster that is easy to comprehend by users, and introduce a new similarity measure especially suitable for sparse transactional data. Using these, an incremental clustering algorithm INCLUS is proposed to cope with high cardinality of transactional datasets. INCLUS is a structure seeking algorithm as it produces clusters based on the support and similarity constraints specified by the users. The domains of input parameters (support and similarity thresholds) are well defined and can be easily chosen by users. INCLUS is significantly different from LargeItem, SUMMARY and TrK-Means which also employ frequent items for clustering. LargeItem uses global optimization to minimize both overlapping of frequent items among clusters and the number of infrequent items for a clustering. INCLUS uses frequent items for local optimization such that similarities for a clustering are maximized with respect to the similarity constraints. In SUMMARY, the frequent items used are globally frequent in the whole data set while in INCLUS they are only locally frequent within a cluster. Unlike INCLUS, TrK-Means does not use similarity constraints and it produces k or more clusters based on the user input of k. Squeezer (7) is an incremental clustering algorithm for categorical data. It uses a different similarity function from INCLUS, which is defined as the number of matched attribute values in a pair of tuples.

The use of machine learning (ML) has been increasing rapidly in the medical imaging field, including computer-aided diagnosis (CAD), radiomics, and medical image analysis. Recently, an ML area called deep learning emerged in the computer vision field and became very popular in many fields. It started from an event in late 2012, when a deep-learning approach based on a convolutional neural network (CNN) won an overwhelming victory in the best-known worldwide computer vision competition, ImageNet Classification. Since then, researchers in virtually all fields, including medical imaging, have started actively participating in the explosively growing field of deep learning. In this paper, the area of deep learning in medical imaging is overviewed, including (1) what was changed in machine learning before and after the introduction of deep learning, (2) what is the source of the power of deep learning, (3) two major deep-learning models: a massive-training artificial neural network (MTANN) and a convolutional neural network (CNN), (4) similarities and differences between the two models, and (5) their applications to medical imaging. This review shows that ML with feature input (or feature-based ML) was dominant before the introduction of deep learning, and that the major and essential difference between ML before and after deep learning is the learning of image data directly without object segmentation or feature extraction; thus, it is the source of the power of deep learning, although the depth of the model is an important attribute. The class of ML with image input (or image-based ML) including deep learning has a long history, but recently gained popularity due to the use of the new terminology, deep learning. There are two major models in this class of ML in medical imaging, MTANN and CNN, which have similarities as well as several differences. In our experience, MTANNs were substantially more efficient in their development, had a higher performance, and required a lesser number of training cases than did CNNs. "Deep learning", or ML with image input, in medical imaging is an explosively growing, promising field. It is expected that ML with image input will be the mainstream area in the field of medical imaging in the next few decades.

The biological age of the brain differs from its chronological age (CA) and can be used as biomarker of neural/cognitive disease processes and as predictor of mortality. Brain age (BA) is often estimated from magnetic resonance images (MRIs) using machine learning (ML) that rarely indicates how regional brain features contribute to BA. Leveraging an aggregate training sample of 3 418 healthy controls (HCs), we describe a ridge regression model that quantifies each region's contribution to BA. After model testing on an independent sample of 651 HCs, we compute the coefficient of partial determination R¯p2 for each regional brain volume to quantify its contribution to BA. Model performance is also evaluated using the correlation r between chronological and biological ages, the mean absolute error (MAE ) and mean squared error (MSE) of BA estimates. On training data, r=0.92, MSE=70.94 years, MAE=6.57 years, and R¯2=0.81; on test data, r=0.90, MSE=81.96 years, MAE=7.00 years, and R¯2=0.79. The regions whose volumes contribute most to BA are the nucleus accumbens (R¯p2=7.27%), inferior temporal gyrus (R¯p2=4.03%), thalamus (R¯p2=3.61%), brainstem (R¯p2=3.29%), posterior lateral sulcus (R¯p2=3.22%), caudate nucleus (R¯p2=3.05%), orbital gyrus (R¯p2=2.96%), and precentral gyrus (R¯p2=2.80%). Our ridge regression, although outperformed by the most sophisticated ML approaches, identifies the importance and relative contribution of each brain structure to overall BA. Aside from its interpretability and quasi-mechanistic insights, our model can be used to validate future ML approaches for BA estimation.

BackgroundThe Centers for Medicare and Medicaid Services projects that health care costs will continue to grow over the next few years. Rising readmission costs contribute significantly to increasing health care costs. Multiple areas of health care, including readmissions, have benefited from the application of various machine learning algorithms in several ways.ObjectiveWe aimed to identify suitable models for predicting readmission charges billed by hospitals. Our literature review revealed that this application of machine learning is underexplored. We used various predictive methods, ranging from glass-box models (such as regularization techniques) to black-box models (such as deep learning–based models).MethodsWe defined readmissions as readmission with the same major diagnostic category (RSDC) and all-cause readmission category (RADC). For these readmission categories, 576,701 and 1,091,580 individuals, respectively, were identified from the Nationwide Readmission Database of the Healthcare Cost and Utilization Project by the Agency for Healthcare Research and Quality for 2013. Linear regression, lasso regression, elastic net, ridge regression, eXtreme gradient boosting (XGBoost), and a deep learning model based on multilayer perceptron (MLP) were the 6 machine learning algorithms we tested for RSDC and RADC through 10-fold cross-validation.ResultsOur preliminary analysis using a data-driven approach revealed that within RADC, the subsequent readmission charge billed per patient was higher than the previous charge for 541,090 individuals, and this number was 319,233 for RSDC. The top 3 major diagnostic categories (MDCs) for such instances were the same for RADC and RSDC. The average readmission charge billed was higher than the previous charge for 21 of the MDCs in the case of RSDC, whereas it was only for 13 of the MDCs in RADC. We recommend XGBoost and the deep learning model based on MLP for predicting readmission charges. The following performance metrics were obtained for XGBoost: (1) RADC (mean absolute percentage error [MAPE]=3.121%; root mean squared error [RMSE]=0.414; mean absolute error [MAE]=0.317; root relative squared error [RRSE]=0.410; relative absolute error [RAE]=0.399; normalized RMSE [NRMSE]=0.040; mean absolute deviation [MAD]=0.031) and (2) RSDC (MAPE=3.171%; RMSE=0.421; MAE=0.321; RRSE=0.407; RAE=0.393; NRMSE=0.041; MAD=0.031). The performance obtained for MLP-based deep neural networks are as follows: (1) RADC (MAPE=3.103%; RMSE=0.413; MAE=0.316; RRSE=0.410; RAE=0.397; NRMSE=0.040; MAD=0.031) and (2) RSDC (MAPE=3.202%; RMSE=0.427; MAE=0.326; RRSE=0.413; RAE=0.399; NRMSE=0.041; MAD=0.032). Repeated measures ANOVA revealed that the mean RMSE differed significantly across models with P<.001. Post hoc tests using the Bonferroni correction method indicated that the mean RMSE of the deep learning/XGBoost models was statistically significantly (P<.001) lower than that of all other models, namely linear regression/elastic net/lasso/ridge regression.ConclusionsModels built using XGBoost and MLP are suitable for predicting readmission charges billed by hospitals. The MDCs allow models to accurately predict hospital readmission charges.

An accurate energy consumption prediction becomes crucial with increasing electric vehicle usage for effective power grid management. This research examined the performance of eleven machine learning models for this purpose: Ridge Regression, Lasso Regression, K-Nearest Neighbors, Gradient Boosting, Support Vector Regression, Multi-Layer Perceptron, XGBoost, CatBoost, LightGBM, Gaussian Processes for Regression(GPR) and Extra Trees Regressor, considering real historical data from Colorado. The models were evaluated using different metrics: Mean Absolute Error (MAE), Mean Squared Error (MSE), R², Root Mean Squared Error(RMSE) and Normalized Root Mean Squared Error(NRMSE), with visual analyses through scatter plots and time series plots. The best model observed was the Extra Trees Regressor, which had an MAE of 0.5888, an MSE of 3.2683, R² value of 0.9592, RMSE of 1.8078 and NRMSE of 0.020. Gradient Boosting and KNN also returned good results, although they were slightly more dispersed. Nevertheless, while non-linear models like MLP, XGBoost, CatBoost, LightGBM and linear models such as Ridge and Lasso Regression offer valuable insights, they exhibit shortcomings in estimating energy, especially at extreme levels, highlighting limitations in capturing complex non-linear interactions. This study focuses on their applicability to energy projections to demonstrate how well ensemble and non-linear models may capture intricate patterns in time series. These cutting-edge machine learning techniques might greatly enhance energy demand predictions.

Recent advancements in machine learning and, more specifically, deep learning methodologies showed promising results in Computed tomography (CT) imaging, either by enhancing existing reconstruction methodologies by cleaning the signal from the collected data or used for the complete reconstruction of the images directly from raw data. The loss function used for the training of these deep learning methodologies can significantly impact the training process. This paper compares different deep learning loss functions in the framework of denoising and recon-structing sinograms based on Single-photon emission computed tomography (SPECT) imaging. The loss functions evaluated in the proposed work are Mean Squared Error (MSE), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), Mean Squared Logarithmic Error (MSLE), Huber, Peak Signal-to-Noise ratio (PSRN) and Structural Similarity Index (SSIM). These loss functions were used to train an identical proposed convolutional-based deep learning methodology to denoise sinograms constructed using software phantoms and compared the results. Furthermore, a study is constructed to measure the reconstructed accuracy of the denoised sinograms and compare the results using the ordered subset expectation maximisation (OSEM) method. Based on our analysis, some guidelines are derived for selecting a loss function in denoising data used for inverse problems such as sinogram reconstruction for medical image applications.

Globally, wind energy if properly harnessed, could serve as a source of energy generation in Africa. This study compared the performance of two Machine Learning (ML) algorithms (Linear regression and Random Forest) in predicting wind speed in five major cities in Africa (Yaoundé, Pretoria, Nairobi, Cairo and Abuja). Wind data were collected between January 1, 2000, and December 31, 2022, using the Solar Radiation Data Archive. The data preprocessing was carried out with 80% of the data used for training and 20% for validation. The performance of these ML algorithms was evaluated using Mean Square Error (MSE), Root Mean Square Error (RMSE), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE) and coefficient of determination (R2). The result shows that Nairobi (3.814795 m/s) closely followed by Cairo (3.606453 m/s) has the highest mean wind speed while Yaoundé (1.090512 m/s) has the lowest. Based on the performance metrics used, the two Machine Learning algorithms were competitive. Still, the Linear Regression (LR) algorithm outperformed the Random Forest Algorithm in predicting wind speed in all the selected major African cities. In Yaoundé (RMSE = 0.3892, MAE= 0.3001, MAPE =0.5030), Pretoria (RMSE=1.2339, MAE=0.9480, MAPE=0.7450) Nairobi (RMSE= 0.4223, MAE =0.6499, MAPE =0.1872), Nairobi (RMSE=0.6499, MAE=0.5171, MAPE =0.1872), Cairo (RMSE =1.0909, MAE =0.8544, MAPE =0.3541) and Abuja (RMSE = 0.70245, MAE =0.5441, MAPE= 0.4515) the Linear regression algorithms was found to outperformed Random Forest Regression. Therefore, the Linear regression algorithm is more reliable in predicting wind speed compared with the Random Forest regression.

Potatoes are an important crop in the world; they are the main source of food for a large number of people globally and also provide an income for many people. The true forecasting of potato yields is a determining factor for the rational use and maximization of agricultural practices, responsible management of the resources, and wider regions’ food security. The latest discoveries in machine learning and deep learning provide new directions to yield prediction models more accurately and sparingly. From the study, we evaluated different types of predictive models, including K-nearest neighbors (KNN), gradient boosting, XGBoost, and multilayer perceptron that use machine learning, as well as graph neural networks (GNNs), gated recurrent units (GRUs), and long short-term memory networks (LSTM), which are popular in deep learning models. These models are evaluated on the basis of some performance measures like mean squared error (MSE), root mean squared error (RMSE), and mean absolute error (MAE) to know how much they accurately predict the potato yields. The terminal results show that although gradient boosting and XGBoost algorithms are good at potato yield prediction, GNNs and LSTMs not only have the advantage of high accuracy but also capture the complex spatial and temporal patterns in the data. Gradient boosting resulted in an MSE of 0.03438 and an R2 of 0.49168, while XGBoost had an MSE of 0.03583 and an R2 of 0.35106. Out of all deep learning models, GNNs displayed an MSE of 0.02363 and an R2 of 0.51719, excelling in the overall performance. LSTMs and GRUs were reported to be very promising as well, with LSTMs comprehending an MSE of 0.03177 and GRUs grabbing an MSE of 0.03150. These findings underscore the potential of advanced predictive models to support sustainable agricultural practices and informed decision-making in the context of potato farming.

The prediction of wind energy generation is important to enhance the performance and dependability of renewable energy systems due to the rising demand for wind-generated electricity and advancements in wind energy technology competitiveness. This study leverages advanced machine learning (ML) and some other statistical and deep learning based time series forecasting models to enhance the accuracy of wind energy predictions. This comprehensive analysis includes nine ML models—Linear Regression, Random Forests (RF), Gradient Boosting Machines (GBM), Support Vector Machines (SVM), K-Nearest Neighbors (KNN), AdaBoost, XGBoost, Support Vector Regression (SVR), and Neural Networks—as well as Four time-series forecasting models—ARIMA, Temporal Convolutional Networks (TCNs), Long Short-Term Memory (LSTM) networks and GRU. Each ML model underwent rigorous cross-validation to ensure optimal performance. The assessment criteria utilized here comprised the Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), and the R² Score. It was found that among the nine ML models, Random Forests, GBM and KNN consistently provided superior accuracy and robustness, making them the top choices for wind energy prediction whereas the performance of linear regression, SVM and SVR were very poor for the considered dataset. From the experiment, Random Forest, GBM, and KNN showed the best performance with low MSE values of 0.77, 1.95, and 1.51 respectively, while other models had MSEs above 7.5, with AdaBoost reaching 30. Their RMSEs (0.88, 1.40, 1.23) and MAEs (0.093, 0.73, 0.10) also indicate strong predictive accuracy compared to the rest.In this paper, time series forecasting, TCNs, LSTM and GRU networks showed strong capabilities in capturing temporal dependencies and trends within the wind energy data. Visualization techniques were employed to compare model performances comprehensively, providing clear insights into their predictive power. Therefore, this present study offers a robust framework for researchers and practitioners aiming to leverage machine learning and time series forecasting in the realm of renewable energy prediction.

BackgroundThe World Health Organization predicts that by 2030, chronic obstructive pulmonary disease (COPD) will be the third leading cause of death and the seventh leading cause of morbidity worldwide. Pulmonary function tests (PFT) are the gold standard for COPD diagnosis. Since COPD is an incurable disease that takes a considerable amount of time to diagnose, even by an experienced specialist, it becomes important to provide an analysis of abnormalities in a simple manner. Although many deep learning (DL) methods based on computed tomography (CT) have been developed to identify COPD, the pathological changes of COPD based on CT are multi-dimensional and highly spatially heterogeneous, and their predictive performance still needs to be improved.ObjectiveThe purpose of this study was to develop a DL-based multimodal feature fusion model to accurately estimate PFT parameters from chest CT images and verify its performance.Materials and methodsIn this retrospective study, participants underwent chest CT examination and PFT at the Fourth Clinical Medical College of Xinjiang Medical University between January 2018 and July 2024. In this study, the 1-s forced expiratory volume (FEV1), forced vital capacity (FVC), 1-s forced expiratory volume ratio forced vital capacity (FEV1/FVC), 1-s forced expiratory volume to predicted value (FEV1%), and forced vital capacity to predicted value (FVC%) of PFT parameters were used as predictors and the corresponding chest CT of 3108 participants. The data were randomly assigned to the training group and the validation group at a ratio of 9:1, and the model was cross-validated using 10-fold cross-validation. Each parameter was trained and evaluated separately on the DL network. The mean absolute error (MAE), mean squared error (MSE), and Pearson correlation coefficient (r) were used as evaluation indices, and the consistency between the predicted and actual values was analyzed using the Bland-Altman plot. The interpretability of the model’s prediction process was analyzed using the Grad-CAM visualization technique.ResultsA total of 2408 subjects were included (average age 66 ± 12 years; 1479 males). Among these, 822 cases were used for encoder training to extract image features, and 1,586 cases were used for the development and validation of a multimodal feature fusion model based on a multilayer perceptron (MLP). The MAE, MSE, and r predicted between PFT and model estimates for FEV1 were 0.34, 0.20, and 0.84, respectively. For FVC, the MAE, MSE, and r were 0.42, 0.31, and 0.81, respectively. For FEV1/FVC, the MAE, MSE, and r were 6.64, 0.73, and 0.77, respectively. For FEV1%, the MAE, MSE, and r were 13.42, 3.01, and 0.73, respectively. For FVC%, the MAE, MSE, and r were 13.33, 2.97, and 0.61, respectively. It was observed that there was a strong correlation between the measured and predicted indices of FEV1, FVC, FEV1/FVC, and FEV1%. The Bland-Altman plot analysis showed good consistency between the estimated values and the measured values of all PFT parameters.ConclusionsThe preliminary research results indicate that the MLP-based multimodal feature fusion model has the potential to predict PFT parameters in COPD patients in real time. However, it is worth noting that the study used indicators before the use of bronchodilators, which may affect the interpretation of the results. Future studies should use measurements taken after bronchodilator administration to better align with clinical standards.Supplementary InformationThe online version contains supplementary material available at 10.1186/s12890-025-03957-7.

Genomic selection serves as an effective way for crop genetic breeding, capable of significantly shortening the breeding cycle and improving the accuracy of breeding. Phenotype prediction can help identify genetic variants associated with specific phenotypes. This provides a data-driven selection criterion for genomic selection, making the selection process more efficient and targeted. Deep learning has become an important tool for phenotype prediction due to its abilities in automatic feature learning, nonlinear modeling, and high-dimensional data processing. Current deep learning models have improvements in various aspects, such as predictive performance and computation time, but they still have limitations in capturing the complex relationships between genotype and phenotype, indicating that there is still room for improvement in the accuracy of phenotype prediction. This study innovatively proposes a new method called DeepAT, which mainly includes an input layer, a data feature extraction layer, a feature relationship capture layer, and an output layer. This method can predict wheat yield based on genotype data and has innovations in the following four aspects: (1) The data feature extraction layer of DeepAT can extract representative feature vectors from high-dimensional SNP data. By introducing the ReLU activation function, it enhances the model’s ability to express nonlinear features and accelerates the model’s convergence speed; (2) DeepAT can handle high-dimensional and complex genotype data while retaining as much useful information as possible; (3) The feature relationship capture layer of DeepAT effectively captures the complex relationships between features from low-dimensional features through a self-attention mechanism; (4) Compared to traditional RNN structures, the model training process is more efficient and stable. Using a public wheat dataset from AGT, comparative experiments with three machine learning and six deep learning methods found that DeepAT exhibited better predictive performance than other methods, achieving a prediction accuracy of 99.98%, a mean squared error (MSE) of only 28.93 tones, and a Pearson correlation coefficient close to 1, with yield predicted values closely matching observed values. This method provides a new perspective for deep learning-assisted phenotype prediction and has great potential in smart breeding.