Data Sampling Approaches with Severely Imbalanced Big Data for Medicare Fraud Detection

This study evaluates the use of deep learning and data sampling on a class-imbalanced Big Data problem, i.e. Medicare fraud detection. Medicare offers affordable health insurance to the elderly population and serves more than 15% of the United States population. To increase transparency and help reduce fraud, the Centers for Medicare and Medicaid Services (CMS) have made several data sets publicly available for analysis. Our research group has conducted several studies using CMS data and traditional machine learning algorithms (non-deep learning), but challenges associated with severe class imbalance leave room for improvement. These previous studies serve as baselines as we employ deep neural networks with various data-sampling techniques to determine the efficacy of deep learning in addressing class imbalance. Random over-sampling (ROS), random under-sampling (RUS), and combinations of the two (ROS-RUS) are applied to study how varying levels of class imbalance impact model training and performance. Classwise performance is maximized by identifying optimal decision thresholds, and a strong linear relationship between minority class size and optimal threshold is observed. Results show that ROS significantly outperforms RUS, combining RUS and ROS both maximizes performance and efficiency with a 4× speedup in training time, and the default threshold of 0.5 is never optimal when training data is imbalanced. To the best of our knowledge, this is the first study to provide statistical results comparing ROS, RUS, and ROS-RUS deep learning methods across a range of class distributions. Additional contributions include a unique analysis of thresholding as it relates to the minority class size and state-of-the-art performance on the given fraud detection task.

In a majority–minority classification problem, class imbalance in the dataset(s) can dramatically skew the performance of classifiers, introducing a prediction bias for the majority class. Assuming the positive (minority) class is the group of interest and the given application domain dictates that a false negative is much costlier than a false positive, a negative (majority) class prediction bias could have adverse consequences. With big data, the mitigation of class imbalance poses an even greater challenge because of the varied and complex structure of the relatively much larger datasets. This paper provides a large survey of published studies within the last 8 years, focusing on high-class imbalance (i.e., a majority-to-minority class ratio between 100:1 and 10,000:1) in big data in order to assess the state-of-the-art in addressing adverse effects due to class imbalance. In this paper, two techniques are covered which include Data-Level (e.g., data sampling) and Algorithm-Level (e.g., cost-sensitive and hybrid/ensemble) Methods. Data sampling methods are popular in addressing class imbalance, with Random Over-Sampling methods generally showing better overall results. At the Algorithm-Level, there are some outstanding performers. Yet, in the published studies, there are inconsistent and conflicting results, coupled with a limited scope in evaluated techniques, indicating the need for more comprehensive, comparative studies.

Imbalanced big data means big data where the ratio of a certain class is relatively small compared to other classes. When the machine learning model is trained by using imbalanced big data, the problem with performance drops for the minority class occurs. For this reason, various oversampling methodologies have been proposed, but simple oversampling leads to problem of the overfitting. In this paper, we propose a meta learning methodology for efficient analysis of imbalanced big data. The proposed meta learning methodology uses the meta information of the data generated by the generative model based on Generative Adversarial Networks. It prevents the generative model from becoming too similar to the real data in minority class. Compared to the simple oversampling methodology for analyzing imbalanced big data, it is less likely to cause overfitting. Experimental results show that the proposed method can efficiently analyze imbalanced big data.

Bank telemarketing campaigns often experience low subscription rates due to customer heterogeneity and severe class imbalance, which pose challenges for reliable predictive modeling. This study investigates a data-driven approach that integrates synthetic minority oversampling and cost-sensitive learning to improve the prediction of telemarketing outcomes. Experiments are conducted using the Portuguese Bank Marketing dataset, comprising 41,188 instances with a positive response rate of 11.3%. Eight machine learning models are evaluated under a unified preprocessing pipeline and five-fold stratified cross-validation, including Logistic Regression, Decision Tree, Random Forest, and Ensemble methods. The results show that Ensemble models, particularly CatBoost, XGBoost, and LightGBM, achieve improved performance compared with traditional baselines, with notable gains in minority-class recall and overall discrimination ability. The best-performing model attains an F1-score of 0.540, a recall of 0.812 for the positive class, and a ROC–AUC of 0.908. To enhance interpretability, SHAP-based analysis is applied to quantify feature contributions, identifying campaign duration, previous contact outcomes, and selected macroeconomic indicators as key predictors. These findings indicate that combining resampling strategies with cost-sensitive optimization provides a robust and transparent approach for learning from imbalanced telemarketing data, thereby supporting reproducible and data-driven financial decision-making by explicitly addressing difficulty in minority-class identification under imbalance and class imbalance under cross-entropy training in imbalanced banking data.

A majority of predictive models should be updated regularly, since the most recent data associated with the model may have a different distribution from that of the original training data. This difference may be critical enough to impact the effectiveness of the machine learning model. In our paper, we investigate the relationship between time and predictive model maintenance. Our work incorporates severely imbalanced big data from three Medicare datasets, namely Part D, DMEPOS, and Combined, that have been used in several fraud detection studies. We build training datasets from year-groupings of 2013, 2014, 2015, 2013–2014, 2014–2015, and 2013–2015. Our test datasets are built from the 2016 data. To mitigate some of the adverse effects from the severe class imbalance in these datasets, the performance of five class ratios obtained by Random Undersampling and five learners is evaluated by the Area Under the Receiver Operating Characteristic Curve metric. The models producing the best values are as follows: Logistic Regression with the 2015 year-grouping at a 99:1 class ratio (Part D); Random Forest with the 2014-2015 year-grouping at a 75:25 class ratio (DMEPOS); and Logistic Regression with the full 2015 year-grouping (Combined). Our experimental results show that the largest training dataset (year-grouping 2013–2015) was not among the selected choices, which indicates that the 2013 data may be outdated. Moreover, we note that because the best model is different for Part D, DMEPOS, and Combined, this suggests that these three datasets may actually be sub-domains requiring unique models within the Medicare fraud detection domain.

Currently, there are many problems in imbalanced big data classification based on rough set with virtual reality technology in cloud computing. For example, redundant big data cleaning is not clear, the effect is poor for big data denoising and feature extraction, and the precision of classification is low. In this paper, an imbalanced big data classification is proposed based on Hubness and K nearest neighbor to address such problems. First, the SNM algorithm is used in order to efficient cleaning of redundant big data. Then, wavelet threshold denoising algorithm is used to denoise the big data to improve the denoising effect. Meantime, feature of big data is extracted based on Lyapunov theorem. Moreover, the Hubness and K-nearest neighbor algorithms are used to achieve high precision of imbalanced big data classification. Experiments verify that the proposed method effectively strengthens current cleaning and denoising methods of redundant imbalanced big data, as well as improves accuracy of extraction and classification of big data.

Multi-class imbalanced big data classification on Spark

Climate change poses significant challenges to agriculture, leading to increased crop damage owing to extreme weather conditions. Detecting and analyzing such damage is crucial for mitigating its effects on crop yield. This study proposes a novel autoencoder (AE)-based model, termed “Memory Ganomaly,” designed to detect and analyze weather-induced crop damage under conditions of significant class imbalance. The model integrates memory modules into the Ganomaly architecture, thereby enhancing its ability to identify anomalies by focusing on normal (undamaged) states. The proposed model was evaluated using apple and peach datasets, which included both damaged and undamaged images, and was compared with existing robust Convolutional neural network (CNN) models (ResNet-50, EfficientNet-B3, and ResNeXt-50) and AE models (Ganomaly and MemAE). Although these CNN models are not the latest technologies, they are still highly effective for image classification tasks and are deemed suitable for comparative analyses. The results showed that CNN and Transformer baselines achieved very high overall accuracy (94–98%) but completely failed to identify damaged samples, with precision and recall equal to zero under severe class imbalance. Few-shot learning partially alleviated this issue (up to 75.1% recall in the 20-shot setting for the apple dataset) but still lagged behind AE-based approaches in terms of accuracy and precision. In contrast, the proposed Memory Ganomaly delivered a more balanced performance across accuracy, precision, and recall (Apple: 80.32% accuracy, 79.4% precision, 79.1% recall; Peach: 81.06% accuracy, 83.23% precision, 80.3% recall), outperforming AE baselines in precision and recall while maintaining comparable accuracy. This study concludes that the Memory Ganomaly model offers a robust solution for detecting anomalies in agricultural datasets, where data imbalance is prevalent, and suggests its potential for broader applications in agricultural monitoring and beyond. While both Ganomaly and MemAE have shown promise in anomaly detection, they suffer from limitations—Ganomaly often lacks long-term pattern recall, and MemAE may miss contextual cues. Our proposed Memory Ganomaly integrates the strengths of both, leveraging contextual reconstruction with pattern recall to enhance detection of subtle weather-related anomalies under class imbalance.

The rapid adoption of the internet in Indonesia, with over 200 million active users as of January 2022, has dramatically transformed information dissemination, particularly through social media and online platforms. These platforms, while democratizing information sharing, have also become hotbeds for the spread of misinformation and hoaxes, significantly impacting the political landscape, as seen in the Jakarta gubernatorial election from late 2016 to April 2017. Research by the Indonesian Telematics Society (MASTEL) revealed a high prevalence of hoax content, predominantly socio-political, underscoring the critical need to address this misinformation and hoaxes challenge. This research delves into the challenge of detecting hoaxes in Indonesian political news, particularly focusing on the classification of news as factual or hoax in the presence of class imbalances within datasets. The dataset exhibits a significant class imbalance with 6,947 articles identified as hoaxes and 20,945 as non-hoaxes, Utilizing the IndoBERT model, a specialized variant of the BERT framework pre-trained on the Indonesian language, the study aims to assess its effectiveness in discerning between factual and hoax news. This involves fine-tuning IndoBERT for specific text classification tasks and exploring the impact of various resampling techniques, such as Random Over Sampling and Random Under Sampling, to address class imbalances since the dataset, significantly imbalanced with 6,947 articles labeled as hoaxes and 20,945 as non-hoaxes, necessitated these approaches. The study's findings demonstrate the IndoBERT model's consistent accuracy across different resampling methods like Random Over Sampling (ROS) and Random Under Sampling (RUS), highlighting its effectiveness in handling imbalanced datasets produce the accuracy of hoax detection with the 98.2% accuracy, 97.5% Recall, 97.8% F1-score, and 97.2% Precision. This is particularly relevant for tasks like misinformation detection, where data imbalance is common. The success of IndoBERT, a language-specific BERT model, in text classification for the Indonesian language contributes to the understanding of BERT-based models in diverse linguistic contexts.

With healthcare being critical aspect, health insurance has become an important scheme in minimizing medical expenses. Medicare is an example of such a healthcare insurance initiative in the United States. Following this, the healthcare industry has seen a significant increase in fraudulent activities owing to increased insurance, and fraud has become a significant contributor to rising medical care expenses, although its impact can be mitigated using fraud detection techniques. To detect fraud, machine learning techniques are used. The Centers for Medicaid and Medicare Services (CMS) of the United States federal government released “Medicare Part D” insurance claims is utilized in this study to develop fraud detection system. Employing machine learning algorithms on a class-imbalanced and high dimensional medicare dataset is a challenging task. To compact such challenges, the present work aims to perform feature extraction following data sampling, afterward applying various classification algorithms, to get better performance. Feature extraction is a dimensionality reduction approach that converts attributes into linear or non-linear combinations of the actual attributes, generating a smaller and more diversified set of attributes and thus reducing the dimensions. Data sampling is commonly used to address the class imbalance either by expanding the frequency of minority class or reducing the frequency of majority class to obtain approximately equal numbers of occurrences for both classes. The proposed approach is evaluated through standard performance metrics such as F-measure and AUC score. Thus, to detect fraud efficiently, this study applies autoencoder as a feature extraction technique, synthetic minority oversampling technique (SMOTE) as a data sampling technique, and various gradient boosted decision tree-based classifiers as a classification algorithm. The experimental results show the combination of autoencoders followed by SMOTE on the LightGBM (short for, Light Gradient Boosting Machine) classifier achieved best results.

Percutaneous coronary intervention (PCI) can be complicated by major adverse cardiovascular events (MACE; death, myocardial infarction [MI], and target vessel revascularization). Statistical models facilitated by machine learning (ML) can improve prediction of MACE compared with conventional models. However, two key challenges impair ML performance: (1) class imbalance in outcome distributions; and (2) bias affecting fairness across social subgroups. These challenges are amplified in small datasets. Additionally, no published ML model specifically addresses the influence of social determinants of health (SDoH) on PCI outcomes. We hypothesized that random oversampling would improve sensitivity and fairness when assessing the effect of SDoH on select PCI-associated MACE in a small dataset. We employed multivariable logistic regression to predict 180-day MACE and new MI following urgent PCI in a small dataset (N = 481). Three SDoH were pre-specified variables: race, marital status, and socioeconomic status at extremes of the spectrum (uninsured and Medicaid versus private insurance). Random oversampling was applied to the SDoH and outcomes while maintaining a consistent ratio between positive and negative outcomes. In the imbalanced dataset, the logistic regression revealed no significant association between SDoH and 180-day MACE or new MI (minimum P-value = 0.47). Although sensitivity for event detection was low (≤ 0.26), other metrics-including positive predictive value (PPV)-were commendable (≥ 0.80). ML classifiers trained on oversampled data for race and marital status showed increased sensitivity but decreased PPV as the ratio of adverse-to-favorable outcome increased. Other performance metrics remained stable. Equalized odds disparity decreased with increasing oversampling ratio, indicating improved fairness. However, for socioeconomic-extremes status, the low numbers of adverse events and small sub-group size limited the effectiveness of oversampling. The optimal oversampling ratio cut-point across all outcomes and SDoH was 0.30-0.40 (sensitivity-0.50; PPV-0.52). In small, imbalanced datasets, random oversampling can improve sensitivity and fairness of ML classifiers when evaluating PCI outcomes across race and marital status. However, this improvement comes at the expense of decreased PPV. The decrease in equalized odds disparity indicates improved balance in the performance of the classifiers across these two subgroups.

Class imbalance should not throw you off balance: Choosing the right classifiers and performance metrics for brain decoding with imbalanced data

- Healthcare fraud detection is an important challenge in modern healthcare systems due to the large volume of medical claims and the highly imbalanced nature of fraud datasets. Fraudulent claims represent only a small portion of the total claims, which makes it difficult for traditional machine learning models to accurately identify fraudulent activities. Conventional techniques such as Random Oversampling (ROS), Synthetic Minority Oversampling Technique (SMOTE), and Random Undersampling (RUS) are commonly used to address class imbalance, but they often lead to issues such as overfitting, noise generation, or loss of important information. In this study, a machine learning-based approach is proposed to improve Medicare fraud detection using the Medicare Part B dataset. The proposed framework applies a hybrid resampling technique called SMOTE-ENN, which combines SMOTE for generating synthetic minority samples with Edited Nearest Neighbors (ENN) to remove noisy and irrelevant data instances. Logistic Regression is used as the classification algorithm to detect fraudulent healthcare claims. The model performance is evaluated using multiple metrics, including accuracy, precision, recall, F1-score, AUC-ROC, and Area Under the Precision-Recall Curve (AUPRC). Experimental results demonstrate that the proposed approach achieves an accuracy of 98%, indicating its effectiveness in handling imbalanced datasets and improving fraud detection in healthcare systems. Key Words: Healthcare Fraud Detection, Machine Learning, SMOTE-ENN, Logistic Regression, Imbalanced Data, Medicare Claims.

A major challenge facing data-mining practitioners in the field of bioinformatics is class imbalance, which occurs when instances of one class (called the majority class) vastly outnumber instances of the other (minority) classes. This can result in models with increased bias towards the majority class (minority-class instances predicted as being in the majority class). Data sampling, a process which changes the dataset through removing or adding instances to improve the class balance, can be used to improve the performance of such models on imbalanced data. However, it is not clear what target balance level should be used with data sampling, and what influence class imbalance alone has on classification performance (compared to other issues such as difficulty of learning from the data and dataset size). To resolve this, we propose the Balance-Aware Subsampling technique, which allows researchers to directly compare different balance levels of a dataset while keeping all other factors (such as dataset size and the actual dataset in question) constant. Thus, any changes in performance can be attributed solely to the chosen balance level. We demonstrate this technique using six datasets from the field of bioinformatics, and we also consider three different subsample sizes (that is, the size of the dataset used for building a model) so we can observe the effect of this parameter on classification performance. Our results show that within each level of class imbalance, the average AUC value increases as the subsample size increases. The key exception is the 20:80 (minority:majority) balance level, for which the average AUC value decreases as the subsample size increases from 80 to 120. We also find that within each subsample size, the average AUC value increases as the minority distribution increases, although this does not completely hold for subsample size 40 (in which case, the Näıve Bayes and Random Forest learners show greater performance at the 35:65 balance level than at 50:50), and in general there is not a significant improvement between the 35:65 and 50:50 balance levels. Overall, by using Balance-Aware Subsampling, we are able to directly observe how class imbalance affects performance isolated from all other factors.

• Multi-sine AC signals enable rapid, precise real-time PEMFC fault diagnosis. • SMOTE-Tomek balances and clears noisy data, sharply defining rare fault conditions. • Dual-attention WGAN synthesizes realistic PEMFC fault signals with high fidelity. • Achieves 99.67 % accuracy, surpassing state-of-the-art by 14 % at high imbalance. • Robust diagnosis and high accuracy using limited and imbalanced fault data. Proton exchange membrane fuel cell (PEMFC) faults, especially dehydration and flooding, cause distinct changes in electrochemical behavior. Consequently, real-time monitoring is essential for early and accurate diagnosis. However, acquiring real-world fault data is challenging, and the rarity of such faults results in severe class imbalance. This imbalance limits the performance, reliability, and practical applicability of conventional diagnostic methods. To address these limitations, this study proposes a unified diagnostic framework that integrates multi-sine AC voltage response, boundary-aware resampling, and attention-guided generative modeling. The key innovations of the proposed approach include: (1) Enhanced fault separability through the first application of multi-sine AC voltage response under data imbalance, enabling real-time extraction of critical electrochemical spectral features for early-stage diagnosis; (2) Improved data balance and clearer class boundaries using synthetic minority oversampling with Tomek links, which oversamples minority classes and removes borderline samples; (3) Realistic minority class synthesis using a dual attention Wasserstein generative adversarial networks, where channel attention focuses on diagnostically relevant spectral features and temporal attention models the dynamic evolution of PEMFC electrochemical behavior, ensuring high-quality, diagnostically informative synthetic fault data. The integrated framework achieves 99.67 % overall diagnostic accuracy and, under an extreme 1:200 class imbalance, outperforms state-of-the-art methods by 14 %. This approach enables rapid, data-efficient PEMFC fault diagnosis, strengthening fault management and advancing the performance of energy systems.