DBFS: An effective Density Based Feature Selection scheme for small sample size and high dimensional imbalanced data sets

Selecting a subset of features poses an NP hard problem, necessitating the development of computationally efficient algorithms to identify nearly optimal feature subsets that enhance classifier performance. High-dimensional data, featured with a numerous features, and extensive datasets present significant challenges to feature subset selection. Key concerns include the extensibility of the feature selection methods in terms of high-dimensional data accuracy and processing time for big datasets. To address these problems, we devised a feature selection method based on ensemble clustering. The literature offers numerous computationally efficient greedy feature ranking techniques, each ranking features differently. A Ensemble among these varied rankings can yield a feature ranking method whose time will be less and performance will be high. This research aims to develop effective algorithms that can deal with both high dimensional and also small datasets. Our contributions include the design of Feature Ranking based on Ensemble Clustering (FREC), a rapid and expandable method for feature selection that leverages existing feature ranking algorithms. Implementation results demonstrate that FREC noticeably outperforms several current methods found in the literature on both small and high dimensional datasets.

Microplastic particle ingestion or inhalation by humans is a problem of growing concern. Unfortunately, current research methods that use machine learning to understand their potential harms are obstructed by a lack of available data. Deep learning techniques in particular are challenged by such domains where only small or imbalanced data sets are available. Overcoming this challenge often involves oversampling underrepresented classes or augmenting the existing data to improve model performance. This paper proposes GANsemble: a two-module framework connecting data augmentation with conditional generative adversarial networks (cGANs) to generate class-conditioned synthetic data. First, the data chooser module automates augmentation strategy selection by searching for the best data augmentation strategy. Next, the cGAN module uses this strategy to train a cGAN for generating enhanced synthetic data. We experiment with the GANsemble framework on a small and imbalanced microplastics data set. A Microplastic-cGAN (MPcGAN) algorithm is introduced, and baselines for synthetic microplastics (SYMP) data are established in terms of Fréchet Inception Distance (FID) and Inception Scores (IS). We also provide a synthetic microplastics filter (SYMP-Filter) algorithm to increase the quality of generated SYMP. Additionally, we show the best amount of oversampling with augmentation to fix class imbalance in small microplastics data sets. To our knowledge, this study is the first application of generative AI to synthetically create microplastics data.

Improving classification accuracy using data augmentation on small data sets

Traditional differential expression genes (DEGs) identification models have limitations in small sample size datasets because they require meeting distribution assumptions, otherwise resulting high false positive/negative rates due to sample variation. In contrast, tabular data model based on deep learning (DL) frameworks do not need to consider the data distribution types and sample variation. However, applying DL to RNA-Seq data is still a challenge due to the lack of proper labeling and the small sample size compared to the number of genes. Data augmentation (DA) extracts data features using different methods and procedures, which can significantly increase complementary pseudo-values from limited data without significant additional cost. Based on this, we combine DA and DL framework-based tabular data model, propose a model TabDEG, to predict DEGs and their up-regulation/down-regulation directions from gene expression data obtained from the Cancer Genome Atlas database. Compared to five counterpart methods, TabDEG has high sensitivity and low misclassification rates. Experiment shows that TabDEG is robust and effective in enhancing data features to facilitate classification of high-dimensional small sample size datasets and validates that TabDEG-predicted DEGs are mapped to important gene ontology terms and pathways associated with cancer.

Occam's razor in dimension reduction: Using reduced row Echelon form for finding linear independent features in high dimensional microarray datasets

An imbalanced dataset influences the supervised learning model. Most of the existing real world datasets are imbalanced and often high dimensional. The existing classification methods tend to perform extremely well on the majority class and give least importance to the minority class. Most of the solutions provided for the imbalanced datasets do not fit in for the high dimensional imbalanced datasets. This paper compares the performance of an existing balancing method (cluster concentric circle based under sampling-C3BUS) over low dimensional imbalanced dataset versus high dimensional imbalanced datasets. This work shows that C3BUS works quiet well for low dimensional imbalanced dataset when compared to high dimensional imbalanced dataset and proves that class imbalance and high dimensionality are one of the two main issues in supervised learning process.

Distributed feature selection (DFS) strategy for microarray gene expression data to improve the classification performance

Network intrusion detection system are significant features that contribute to the enterprises and organization network success. In the past decade, Intrusion Detection system (IDS) using several methods have been proposed and implemented to safeguard the networks within an organization to be reliable, secure, and readily available. Although, some of these methods are built utilizing machine learning concepts. IDS utilizing machine learning techniques are accurate, effective and efficient in spotting attacks. However, the machine learning model performance reduces with high dimensional intrusion dataset. Therefore, it is sacrosanct to implement an efficient feature selection technique that can generate a great impact on the classification stage. Also, the feature selection phase is the most serious phase in IDS based on machine learning. This phase is expensive both in time and efforts. Furthermore, most of the proposed machine learning intrusion detection system suffered from low detection accuracy and high false positive rate when the models are experimented on high dimensional dataset. The aim of this paper is to propose a binary firefly algorithm (BFFA) based feature selection for IDS. We first performed normalization in the first stage of the model. Subsequently, the BFFA algorithm was used for feature selection stage. We adopted random forest algorithm for the classification phase. The experiment was performed on high dimensional University of New South Wales-NB 2015 (UNSW-NB15) dataset with seventy five percent of the data used for training the model and twenty percent for testing. The findings showed an accuracy of 99.72%, detection rate of 99.84%, precision of 99.27%, recall of 99.84% and F-score of 99.56%. The results were gauge with the state-of-the-art results and our results were found outstanding.

High-dimensional datasets with low sample sizes (HDLSS) are pivotal in the fields of biology and bioinformatics. One of core objective of HDLSS is to select most informative features and discarding redundant or irrelevant features. This is particularly crucial in bioinformatics, where accurate feature (gene) selection can lead to breakthroughs in drug development and provide insights into disease diagnostics. Despite its importance, identifying optimal features is still a significant challenge in HDLSS. To address this challenge, we propose an effective feature selection method that combines gradual permutation filtering with a heuristic tribrid search strategy, specifically tailored for HDLSS contexts. The proposed method considers inter-feature interactions and leverages feature rankings during the search process. In addition, a new performance metric for the HDLSS that evaluates both the number and quality of selected features is suggested. Through the comparison of the benchmark dataset with existing methods, the proposed method reduced the average number of selected features from 37.8 to 5.5 and improved the performance of the prediction model, based on the selected features, from 0.855 to 0.927. The proposed method effectively selects a small number of important features and achieves high prediction performance.

One of the more prevalent problems when working with bioinformatics datasets is class imbalance, when there are more instances in one class compared to the other class (es). This problem is made worse because frequently, the class of interest is also the minority class. A possible solution is data sampling, a powerful tool for combating class imbalance by adding or removing instances to make the dataset more balanced. In addition to the choice of including data sampling, one of the most important decisions when applying data is what the final class ratio should be. Commonly, the final class ratio when data is applied is 50:50, however it is an open question whether other ratios are more appropriate for certain imbalanced datasets (all datasets in this paper have 25.16% minority instances or less) where a 50:50 ratio requires extreme modification to the dataset. In this work we compare six different data approaches (feature selection with the pair wise combinations of three data techniques and two final class ratios) with feature selection without data with the goal of determining if the inclusion of data is beneficial and if so, what should be the final class ratio. In order to test the six data approaches and feature selection alone thoroughly, we utilize seven imbalanced and high-dimensional datasets, three feature selection techniques, and six classifiers. Our results show that for a majority of scenarios, random under along with either 35:65 or 50:50 is the best data approach. Statistical analysis shows that there is significant difference between the data approaches. However, despite this, we still recommend using random under along with 35:65 as the final class ratio. This is because of the frequency of both random under and 35:65 being the most frequent top performing data technique and class ratio respectively. Additionally, 35:65 will have fewer negative impacts than 50:50 (less data loss or over fitting, which makes it a better choice if all other factors are equal) and random under is more computationally efficient than any other form of sampling, including no sampling (both by not requiring any internal calculations and by producing a reduced, easier-to-work-with dataset). To our knowledge, this is the most comprehensive work which focuses on the choice of the inclusion and implementation of data with different final class ratios on bioinformatics datasets which exhibit such large levels of class imbalance.

iLDA: A new dimensional reduction method for non-Gaussian and small sample size datasets

Tracking recurrence of correlation structure in neuronal recordings

High-dimensional data sets are often available in genome-enabled predictions. Such data sets include nonlinear relationships with complex dependence structures. For such situations, vine copula-based (quantile) regression is an important tool. However, the current vine copula-based regression approaches do not scale up to high and ultra-high dimensions. To perform high-dimensional sparse vine copula-based regression, we propose 2 methods. First, we show their superiority regarding computational complexity over the existing methods. Second, we define relevant, irrelevant, and redundant explanatory variables for quantile regression. Then, we show our method's power in selecting relevant variables and prediction accuracy in high-dimensional sparse data sets via simulation studies. Next, we apply the proposed methods to the high-dimensional real data, aiming at the genomic prediction of maize traits. Some data processing and feature extraction steps for the real data are further discussed. Finally, we show the advantage of our methods over linear models and quantile regression forests in simulation studies and real data applications.

With the rapid development of artificial intelligence in recent years, the research on image processing, text mining, and genome informatics has gradually deepened, and the mining of large-scale databases has begun to receive more and more attention. The objects of data mining have also become more complex, and the data dimensions of mining objects have become higher and higher. Compared with the ultra-high data dimensions, the number of samples available for analysis is too small, resulting in the production of high-dimensional small sample data. High-dimensional small sample data will bring serious dimensional disasters to the mining process. Through feature selection, redundancy and noise features in high-dimensional small sample data can be effectively eliminated, avoiding dimensional disasters and improving the actual efficiency of mining algorithms. However, the existing feature selection methods emphasize the classification or clustering performance of the feature selection results and ignore the stability of the feature selection results, which will lead to unstable feature selection results, and it is difficult to obtain real and understandable features. Based on the traditional feature selection method, this paper proposes an ensemble feature selection method, Random Bits Forest Recursive Clustering Eliminate (RBF-RCE) feature selection method, combined with multiple sets of basic classifiers to carry out parallel learning and screen out the best feature classification results, optimizes the classification performance of traditional feature selection methods, and can also improve the stability of feature selection. Then, this paper analyzes the reasons for the instability of feature selection and introduces a feature selection stability measurement method, the Intersection Measurement (IM), to evaluate whether the feature selection process is stable. The effectiveness of the proposed method is verified by experiments on several groups of high-dimensional small sample data sets.

Recently, anomaly detection has acquired a realistic response from data mining scientists as a graph of its reputation has increased smoothly in various practical domains like product marketing, fraud detection, medical diagnosis, fault detection and so many other fields. High dimensional data subjected to outlier detection poses exceptional challenges for data mining experts and it is because of natural problems of the curse of dimensionality and resemblance of distant and adjoining points. Traditional algorithms and techniques were experimented on full feature space regarding outlier detection. Customary methodologies concentrate largely on low dimensional data and hence show ineffectiveness while discovering anomalies in a data set comprised of a high number of dimensions. It becomes a very difficult and tiresome job to dig out anomalies present in high dimensional data set when all subsets of projections need to be explored. All data points in high dimensional data behave like similar observations because of its intrinsic feature i.e., the distance between observations approaches to zero as the number of dimensions extends towards infinity. This research work proposes a novel technique that explores deviation among all data points and embeds its findings inside well established density-based techniques. This is a state of art technique as it gives a new breadth of research towards resolving inherent problems of high dimensional data where outliers reside within clusters having different densities. A high dimensional dataset from UCI Machine Learning Repository is chosen to test the proposed technique and then its results are compared with that of density-based techniques to evaluate its efficiency.