Face Image Feature Extraction and Feature Selection

With the rapid development of artificial intelligence in recent years, the research on image processing, text mining, and genome informatics has gradually deepened, and the mining of large-scale databases has begun to receive more and more attention. The objects of data mining have also become more complex, and the data dimensions of mining objects have become higher and higher. Compared with the ultra-high data dimensions, the number of samples available for analysis is too small, resulting in the production of high-dimensional small sample data. High-dimensional small sample data will bring serious dimensional disasters to the mining process. Through feature selection, redundancy and noise features in high-dimensional small sample data can be effectively eliminated, avoiding dimensional disasters and improving the actual efficiency of mining algorithms. However, the existing feature selection methods emphasize the classification or clustering performance of the feature selection results and ignore the stability of the feature selection results, which will lead to unstable feature selection results, and it is difficult to obtain real and understandable features. Based on the traditional feature selection method, this paper proposes an ensemble feature selection method, Random Bits Forest Recursive Clustering Eliminate (RBF-RCE) feature selection method, combined with multiple sets of basic classifiers to carry out parallel learning and screen out the best feature classification results, optimizes the classification performance of traditional feature selection methods, and can also improve the stability of feature selection. Then, this paper analyzes the reasons for the instability of feature selection and introduces a feature selection stability measurement method, the Intersection Measurement (IM), to evaluate whether the feature selection process is stable. The effectiveness of the proposed method is verified by experiments on several groups of high-dimensional small sample data sets.

The paper proposes a hybrid approach to reduce dimension in text classification problems, to overcome the issue of Curse of Dimensionality. This hybrid approach is a combination of Feature Selection (FS) and Feature Extraction (FE) methods, considering different aspects of feature relevance, to effectively reduce the dimension in large text datasets. It prevents feature selection biased in favor of a particular FS method. Many FS methods like Term Variance, Document Frequency, Information Gain, Shannons Entropy measure, Mean-Median and Mean Absolute Difference, were implemented and a comparative study was made on their performance when implemented in a hybrid system. The features selected by the individual FS methods are merged using three approaches, namely, Union, Intersection and Modified Union. The sub lists of features further undergo Feature Extraction by PCA, and the reduced feature sub list is clustered with k-means. Finally, the sentiment-score of the individual clusters are calculated using SentiWordNet database which gives the polarity of the data. The experiments were conducted on the benchmark datasets namely Reuters-21,578 and Classic4. The performance evaluation of the system made using the measures like precision, recall, f-score and accuracy shows that the proposed method has improved performance compared to its competitive methods.

QSAR analysis of diaryl COX-2 inhibitors: Comparison of feature selection and train-test data selection methods

In the field of fault diagnosis, machine learning is highly valued for its broad applicability and efficiency. Feature extraction and feature selection are key steps in the application of machine learning, and the performance of fault diagnosis methods relies heavily on the effective execution of these two steps. For this reason, this paper aims to enhance the performance of fault diagnosis methods by improving these two aspects. Firstly, to address the non-linearity and non-stationarity of rotating machinery vibration signals under variable operation conditions, this paper proposes an improved rapid refined composite multiscale sample entropy (IR2CMSE) feature extraction method. In addition, this paper decomposes the vibration signals with improved complete ensemble empirical mode decomposition with adaptive noise (ICEEMDAN) and extracts the sensitive intrinsic modal functions’ (IMFs') IR2CMSE values (SI-IR2CMSE) as the initial feature vector, which more accurately reveals the intrinsic time-scale characteristics of the vibration signals. Secondly, to address the problem of over-reliance on sample labels in most feature selection methods, this paper proposes a semi-supervised Gaussian mixing model with sparse regularization (SSGMM-SR) feature selection model. The model does not require complete fault labels and can automatically identify important features. Finally, validation with two rotating machinery fault datasets shows that the method proposed in this study exhibits high diagnostic accuracy and stability across multiple classifiers.

Stock market prediction is a very noisy problem and the use of any additional information to increase accuracy is necessary. In this paper, for the stock daily return prediction problem, the set of features is expanded to include indicators not only for the stock to be predicted itself but also a set of other stocks and currencies. Afterwards, different feature selection and classification methods are utilized for prediction. The daily close returns of the 3 most traded stocks (GARAN, THYAO, and ISCTR) in Borsa Istanbul (BIST) are predicted using indicators computed on those stocks, indicators for all the other stocks listed in the BIST100 index, and indicators on the dollar-gold prices. Twenty-five different indicators on daily stock prices are computed to form feature vectors for each trading day. These feature vectors are assigned class labels according to the daily close returns. Expanding the feature space with BIST100 stocks features results in a high dimensional feature space, with possibly noisy or irrelevant features. Therefore, feature selection methods are utilized to select the most informative features. In order to determine relevance scores of features, fast filter-based methods, gain ratio and relief, are used. Experiments are performed based on individual stock features, dollar-gold features (DG), BIST100 stock features (BIST100), and a combination of BIST100 and DG with and without feature selection. Using the gain ratio feature selection with a gradient boosting machine (GBM), the movements of GARAN stock were predicted with an accuracy of 0.599 and an F-measure of 0.614. For THYAO, the relief feature selection with the GBM gave an accuracy of 0.558, and for ISCTR, the gain ratio feature selection with logistic regression achieved an accuracy of 0.581. It was found that using BIST100 stock features boosts classification results for all stocks in terms of accuracy.

Software defect prediction models are used for predicting high risk software components. Feature selection has significant impact on the prediction performance of the software defect prediction models since redundant and unimportant features make the prediction model more difficult to learn. Ensemble feature selection has recently emerged as a new methodology for enhancing feature selection performance. This paper proposes a new multi-criteria-decision-making (MCDM) based ensemble feature selection (EFS) method. This new method is termed as MCDM-EFS. The proposed method, MCDM-EFS, first generates the decision matrix signifying the feature’s importance score with respect to various existing feature selection methods. Next, the decision matrix is used as the input to well-known MCDM method TOPSIS for assigning a final rank to each feature. The proposed approach is validated by an experimental study for predicting software defects using two classifiers K-nearest neighbor (KNN) and naïve bayes (NB) over five open-source datasets. The predictive performance of the proposed approach is compared with existing feature selection algorithms. Two evaluation metrics – nMCC and G-measure are used to compare predictive performance. The experimental results show that the MCDM-EFS significantly improves the predictive performance of software defect prediction models against other feature selection methods in terms of nMCC as well as G-measure.

This work presents a comparative study of methods for building predictive models from data in high dimensionality spaces, i.e. where the number of features describing items to be classified is high as compared with the available number of items used to build the model and test its predictive performance. Application of such methods may be quite diverse, ranging from data analysis in life sciences (e.g., analysis of data from experiments generating thousands of feature-numbers per tested case, such a microarray or RT-PCR techniques), to analysis of monitoring data from a complex, highly reliable technical system, where the relationship is being sought between the monitoring data and the relatively infrequent occurrences of some event (such as a faulty or somewhat untypical state of the system). This latter case may be of special interest in early prediction of abnormal conditions in systems focused on dependability. The multidimensional data analysis challenges and generic methods are described in this paper using a very problem specific language of life sciences, namely classification of samples based on gene expression profiles obtained using DNA microarrays. We concentrate on feature selection methods (which in the context are gene selection methods). We also propose a method to evaluate performance of feature (gene) selection methods by looking at predictive power of classifiers based on selected features.

In many real-world unsupervised learning applications, given data with balanced distribution, that is, there are an approximately equal number of instances in each class, we often need to construct a model to reveal such balance. However, in many data, especially the high-dimensional ones, the data in the original feature space often do not present such balance due to the redundant and noisy features. To tackle this problem, we apply an unsupervised spectral feature selection method to select some informative features, which can better reveal the balanced structure of data. Although spectral feature selection is one of the most popular unsupervised feature selection methods and has been widely studied, none of the existing spectral feature selection methods consider the balance property of data. To address this issue, in this article, we propose a novel balanced spectral feature selection (BSFS) method, which not only selects the discriminative features but also picks those to reveal the balanced structure of data. To the best of our knowledge, this is the first spectral feature selection method considering balance structure of data. By introducing a balanced regularization term, we integrate the balanced spectral clustering and feature selection into a unified framework seamlessly. At last, the experiments on benchmark datasets show that the proposed one outperforms the conventional feature selection methods in both clustering performance and balance, which demonstrates the effectiveness and efficiency of the proposed method.

Radiomics is a technique that extracts quantitative features from medical images using data-characterization algorithms. Radiomic features can be used to identify tissue characteristics and radiologic phenotyping that is not observable by clinicians. A typical workflow for a radiomics study includes cohort selection, radiomic feature extraction, feature and predictive model selection, and model training and validation. While there has been increasing attention given to radiomic feature extraction, standardization, and reproducibility, currently, there is a lack of rigorous evaluation of feature selection methods and predictive models. Herein, we review the published radiomics investigations in CT lung cancer and provide an overview of the commonly used radiomic feature selection methods and predictive models. We also compare limitations of various methods in clinical applications and present sources of uncertainty associated with those methods. This review is expected to help raise awareness of the impact of radiomic feature and model selection methods on the integrity of radiomics studies.

With the development of technology, large databases become more accessible thanks to automation systems that automatically keep data and allow the use of large databases in many areas. Machine learning approaches, a sub-branch of artificial intelligence, are used in making decisions about the process by analyzing the data stored in databases and converting them into information. In this paper, the body production process of the surgical (medical) mask is analyzed. As it is known, surgical masks have become a part of our lives by becoming widespread all over the world with the COVID-19 pandemic. In the surgical mask body production process, using the real data of the production factors, first of all, filtering feature selection methods and analyzes were made and the feature selection method to be used was determined. With the specified feature selection method, the factors affecting the product quality are determined. Secondly, machine learning methods were used to determine the values and value ranges of factors (features) in the production of defect-free products. The performances of the machine learning models established in the second stage were increased by feature selection analysis. In the study, together with the parameter optimizations made to machine learning algorithms, it was seen that the best algorithm to estimate the defective product rate was the Ibk algorithm with 92.3% accuracy, 91.9% F measurement and 93% AUC value. Finally, in line with the decision rules revealed in the study, it was observed that the fabric types used for the upper/middle/lower layers that make up the body part in the mask body production process greatly affect the rates of defective or defect-free products. If the rod apparatus around the nose belongs to class k, it has been determined that many masks are defective. Improvement suggestions were presented according to the application results.

This paper proposes a method for texture feature extraction by integrating Gabor filters and independent component analysis (ICA) for Uyghur handwriting based writer identification. That is, the texture image is firstly filtered by a given bank of Gabor filters, and then higher dimensional feature vectors are constructed from the filtered texture images. Next, the dimensionality of these vectors is reduced by means of principal component analysis (PCA). Finally, the independent components in the resulting vectors with dimensionality reduced are analyzed and extracted by us. Experiments were performed using KNN-5 classifier to Uyghur handwriting samples from 55 different people and promising results of 92.5% correct identification rate were achieved.

In the era of digital information, online platforms play a crucial role in shaping public opinion. However, the extensive spread of misinformation and fake news poses a significant challenge, largely fueled by non-credible users. Detecting user credibility is vital for ensuring the reliability of information on these platforms. This study employs supervised machine learning algorithms, leveraging key user features to enhance credibility detection. Feature selection methods, specifically SelectKBest and correlation-based algorithms, are explored for their impact on X-Platform user credibility detection. Utilizing various classifiers, including support vector machine, logistic regression, and XGBoost, experiments are conducted on the ArPFN dataset, which is a labeled, balanced, publicly available dataset. The evaluation includes measures like accuracy, precision, recall, and F1-score to assess efficiency. This research considers feature categories and selection methods with SML to detect their impact on the accuracy of X-Platform user credibility detection, making this research a reference for researchers and practitioners working in the field of SML, feature engineering, and social media analysis. We aim to advance the field’s understanding of effective strategies for mitigating the spread of fake news. The novelty of this study lies in the comprehensive exploration of feature selection methods and their influence on credibility detection, contributing valuable insights for future research in this domain.

Day by day the number of text documents in digital form is increasing. Text classification is used to organize these text documents. However, text classification has the problem of high dimensionality of feature space. This high dimensionality of feature space is solved by feature selection and feature extraction methods and improves the performance of text categorization. The feature selection and feature extraction techniques remove the irrelevant features from the text documents and reduce the dimensionality of feature space. This paper presents the various feature selection and feature extraction methods. This paper also discusses various classifiers to classify the documents.

As advances in information technology (IT) affect all areas in the world, cyber-attacks also continue to increase. Malware has been used for cyber attacks, and the number of new malware and variants tends to explode in these years, depending on its trendy types. In this study, we introduce semantic feature generation and new feature selection methods for improving the accuracy of malware detection based on API sequences to detect these new malware and variants. Therefore, one of the existing feature selection methods is chosen because it shows the best performance, and then it is improved to be suitable for malware detection. In addition, the improved feature selection method is verified by using the Reuter dataset. Finally, the actual API sequences are extracted from the given malware and benign, and the proposed feature generation and selection methods are used to generate a feature vector. The performance is verified through classification.

Sentiment classification has become one of the most popular text classification domains especially in recent years. As it is valid for all text classification problems, high dimensionality of the feature space is one of the most important concerns for sentiment classification due to accuracy considerations. This study analyses the performance of six recent text feature selection methods for document level sentiment classification using two widely-known classifiers namely Support Vector Machines (SVM) and naive Bayes (NB). Three datasets including different types of sentiment data were utilized in the experiments. These datasets are named as Cornell movie review, Sentiment140, and Nine public sentiment. For evaluation, two different success measures namely Micro-F1 and Macro-F1 were used. Also, 3-fold cross-validation is preferred for a fair system performance evaluation. Experiments indicated that distinguishing feature selector (DFS) and discriminative features selection (DFSS) methods are superior to the other four feature selection methods for sentiment classification. The highest classification performances with SVM classifier were obtained when it is combined with DFSS feature selection method in general. On the other hand, highest classification performances with NB classifier were obtained when it is combined with DFS feature selection method.