Risk Analysis in Electronic Payments and Settlement System Using Dimensionality Reduction Techniques

This report provides a comprehensive study focused on land cover classification and remote sensing image analysis using the PaviaU dataset. The report first introduces the basic characteristics and application background of the data set, and then discusses in detail the application of feature selection and dimensionality reduction techniques, especially linear discriminant analysis (LDA) and principal component analysis (PCA). In terms of model application, the report not only uses traditional machine learning models, such as support vector machines, Bayesian and KNN posterior probabilities, but also explores the performance of these models under different parameter settings. Through a series of experiments, the study found that optimizing feature selection and dimensionality reduction techniques can significantly improve the classification accuracy of the model. Finally, the report compares different models and suggests alternative improvements.

A Review of Dimensionality Reduction Techniques for Efficient Computation

Remote sensing Hyperspectral Image (HSI) comprises significant information about the earth’s surface which is actually acquired by hundred of narrow and adjacent spectral bands. The intended performance of classification accuracy does not attain due to the volume of the original HSI dataset and the enormous quantity of spectral bands. As such, dimensionality reduction approaches using feature extraction and selection are typically adopted to enhance classification performance. The unsupervised Principal Component Analysis (PCA), as well as the supervised Linear Discriminant Analysis (LDA), are commonly used as linear feature extraction methods for feature subspace detection. However, due to considering the effects of global variation, both PCA and LDA fail to extract local characteristics of HSI. In this paper, we propose a segmented LDA-based (SLDA) feature extraction where we apply the LDA in a segmented way to extract better local characteristics as well as global characteristics from the HSI. Per-pixel classification using a Support Vector Machine (SVM) is applied to our proposed SLDA method, PCA, Segmented-PCA (SPCA), and LDA on the Indian Pines agricultural HSI dataset. The experimental results show that the overall classification performance of SLDA (90.60%) remarkably outperforms all the other investigated methods: PCA (85.55%), SPCA (86.96%), LDA (86.45%), and the complete original dataset without employing any feature reduction method (83.10%). The proposed SLDA also requires the least amount of space complexity in different implementation scenarios.

Diabetes mellitus is a group of metabolic diseases in which blood sugar levels are too high. About 8.8% of the world was diabetic in 2017. It is projected that this will reach nearly 10% by 2045. The major challenge is that when machine learning-based classifiers are applied to such data sets for risk stratification, leads to lower performance. Thus, our objective is to develop an optimized and robust machine learning (ML) system under the assumption that missing values or outliers if replaced by a median configuration will yield higher risk stratification accuracy. This ML-based risk stratification is designed, optimized and evaluated, where: (i) the features are extracted and optimized from the six feature selection techniques (random forest, logistic regression, mutual information, principal component analysis, analysis of variance, and Fisher discriminant ratio) and combined with ten different types of classifiers (linear discriminant analysis, quadratic discriminant analysis, naïve Bayes, Gaussian process classification, support vector machine, artificial neural network, Adaboost, logistic regression, decision tree, and random forest) under the hypothesis that both missing values and outliers when replaced by computed medians will improve the risk stratification accuracy. Pima Indian diabetic dataset (768 patients: 268 diabetic and 500 controls) was used. Our results demonstrate that on replacing the missing values and outliers by group median and median values, respectively and further using the combination of random forest feature selection and random forest classification technique yields an accuracy, sensitivity, specificity, positive predictive value, negative predictive value and area under the curve as: 92.26%, 95.96%, 79.72%, 91.14%, 91.20%, and 0.93, respectively. This is an improvement of 10% over previously developed techniques published in literature. The system was validated for its stability and reliability. RF-based model showed the best performance when outliers are replaced by median values.

BackgroundAutomatic classification of arrhythmias based on electrocardiography (ECG) data faces several significant challenges, particularly due to the substantial volume of clinical data involved in ECG signal analysis. The volume of clinical data has increased considerably, especially with the emergence of new clinical symptoms and signs in various arrhythmia conditions. These symptoms and signs, which serve as distinguishing features, can number in the tens of thousands. However, the inclusion of irrelevant features can lead to inaccurate classification results.MethodTo identify the most relevant and optimal features for ECG arrhythmia classification, common feature extraction techniques have been applied to ECG signals, specifically shallow and deep feature extraction. Additionally, a feature selection technique based on a metaheuristic optimization algorithm is utilized following the ECG feature extraction process.ResultsOur findings indicate that shallow feature extraction based on the time-domain analysis, combined with feature selection using a metaheuristic optimization algorithm, outperformed other ECG feature extraction and selection techniques. Among eight features of time-domain anaylsis, the selected feature is one to three features from RR-interval assesment, achieving 100% accuracy, sensitivity, specificity, and precision for ECG arrhythmia classification.ConclusionThe proposed end-to-end architecture for ECG arrhythmia classification demonstrates simplicity in parameters and low complexity, making it highly effective for practical applications.

The current world is referred to as the "data world," as, according to Google, 328.77 million terabytes of data are generated every day and are continually increasing. One of the cause contributing to the data growth is the stock market. Thus, it is now necessary to reduce data by removing unnecessary data and extracting only the data that is important. The feature selection procedure is crucial for focusing on key data and reducing the dimensionality of the data. As per my knowledge there aren't many published articles that review the feature extraction and selection techniques utilized in stock market prediction at this time. The same motive will be covered in this paper, where we will analyze feature extraction and selection techniques utilized in the stock market. It includes embedded, filter, wrapper, supervised, unsupervised, semi-supervised, and hybrid methods.

Text mining is a special case of data mining which explore unstructured or semi-structured text documents, to establish valuable patterns and rules that indicate trends and significant features about specific topics. Text mining has been in pattern recognition, predictive studies, sentiment analysis and statistical theories in many areas of research, medicine, financial analysis, social life analysis, and business intelligence. Text mining uses concept of natural language processing and machine learning. Machine learning algorithms have been used and reported to give great results, but their performance of machine learning algorithms is affected by factors such as dataset domain, number of classes, length of the corpus, and feature selection techniques used. Redundant attribute affects the performance of the classification algorithm, but this can be reduced by using different feature selection techniques and dimensionality reduction techniques. Feature selection is a data preprocessing step that chooses a subset of input variable while eliminating features with little or no predictive information. Feature selection techniques are Information gain, Term Frequency, Term Frequency-Inverse document frequency, Mutual Information, and Chi-Square, which can use a filters, wrappers, or embedded approaches. To get the most value from machine learning, pairing the best algorithms with the right tools and processes is necessary. Little research has been done on the effect of feature selection techniques on classification accuracy for pairing of these algorithms with the best feature selection techniques for optimal results. In this research, a text classification experiment was conducted using incident management dataset, where incidents were classified into their resolver groups. Support vector machine (SVM), K-Nearest Neighbors (KNN), Naïve Bayes (NB) and Decision tree (DT) machine learning algorithms were examined. Filtering approach was used on the feature selection techniques, with different ranking indices applied for optimal feature set and classification accuracy results analyzed. The classification accuracy results obtained using TF were, 88% for SVM, 70% for NB, 79% for Decision tree, and KNN had 55%, while Boolean registered 90%, 83%, 82% and 75%, for SVM, NB, DT, and KNN respectively. TF-IDF, had 91%, 83%, 76%, and 56% for SVM, NB, DT, and KNN respectively. The results showed that algorithm performance is affected by feature selection technique applied. SVM performed best, followed by DT, KNN and finally NB. In conclusion, presence of noisy data leads to poor learning performance and increases the computational time. The classifiers performed differently depending on the feature selection technique applied. For optimal results, the classifier that performed best together with the feature selection technique with the best feature subset should be applied for all types of data for accurate classification performance. Keywords: Text Classification, Supervised Machine Learning, Feature Selection DOI : 10.7176/JIEA/9-3-06 Publication date :May 31 st 2019

The remarkable development in technology has led to the increase of massive big data. Machine learning processes provide a way for investigators to examine and particularly classify big data. Besides, several machine learning models rely on powerful feature extraction and feature selection techniques for their success. In this paper, a big data classification approach is developed using an optimized deep learning classifier integrated with hybrid feature extraction and feature selection approaches. The proposed technique uses local linear embedding-based kernel principal component analysis and perturbation theory, respectively, to extract more representative data and select the appropriate features from the big data environment. In addition, the feature selection task is fine-tuned by using perturbation theory through heuristic search based on their output accuracy. This feature selection heuristic search method is analysed with five recent heuristic optimization algorithms for deciding the final feature subset. Finally, the data are categorized through an attention-based bidirectional long short-term memory classifier that is optimized with a golden eagle-inspired algorithm. The performance of the proposed model is experimentally verified on publicly accessible datasets. From the experimental outcomes, it is demonstrated that the proposed framework is capable of classifying large datasets with more than 90% accuracy.

Dimensionality reduction techniques are widely used in machine learning to reduce the computational complexity of the model and improve its performance by identifying the most relevant features. In this research paper, we compare various dimensionality reduction techniques, including Principal Component Analysis(PCA), Independent Component Analysis(ICA), Local Linear Embedding(LLE), Local Binary Patterns(LBP), and Simple Autoencoder, on the Olivetti dataset, which is a popular benchmark dataset in the field of face recognition. We evaluate the performance of these dimensionality reduction techniques using various classification algorithms, including Support Vector Classifier (SVC), Linear Discriminant Analysis (LDA), Logistic Regression (LR), K-Nearest Neighbors (KNN), and Support Vector Machine (SVM). The goal of this research is to determine which combination of dimensionality reduction technique and classification algorithm is the most effective for the Olivetti dataset. Our research provides insights into the performance of various dimensionality reduction techniques and classification algorithms on the Olivetti dataset. These results can be useful in improving the performance of face recognition systems and other applications that deal with high-dimensional data.

In many medical applications, feature selection is obvious; but in medical domains, selecting features and creating a feature vector may require more effort. The wavelet transform (WT) technique is used to identify the characteristic points of an electrocardiogram (ECG) signal with fairly good accuracy, even in the presence of severe high-frequency and low-frequency noise. Principal component analysis (PCA) is a suitable technique for ECG data analysis, feature extraction, and image processing — an important technique that is not based upon a probability model. The aim of the paper is to derive better diagnostic parameters for reducing the size of ECG data while preserving morphology, which can be done by PCA. In this analysis, PCA is used for decorrelation of ECG signals, noise, and artifacts from various raw ECG data sets. The aim of this paper is twofold: first, to describe an elegant algorithm that uses WT alone to identify the characteristic points of an ECG signal; and second, to use a composite WT-based PCA method for redundant data reduction and better feature extraction. PCA scatter plots can be observed as a good basis for feature selection to account for cardiac abnormalities. The study is analyzed with higher-order statistics, in contrast to the conventional methods that use only geometric characteristics of feature waves and lower-order statistics. A new algorithm — viz. PCA variance estimator — is developed for this analysis, and the results are also obtained for different combinations of leads to find correlations for feature classification and useful diagnostic information. PCA scatter plots of various chest and augmented ECG leads are obtained to examine the varying orientations of the ECG data in different quadrants, indicating the cardiac events and abnormalities. The efficacy of the PCA algorithm is tested on different leads of 12-channel ECG data; file no. 01 of the Common Standards for Electrocardiography (CSE) database is used for this study. Better feature extraction is obtained for some specific combinations of leads, and significant improvement in signal quality is achieved by identifying the noise and artifact components. The quadrant analysis discussed in this paper highlights the filtering requirements for further ECG processing after performing PCA, as a primary step for decorrelation and dimensionality reduction. The values of the parameters obtained from the results of PCA are also compared with those of wavelet methods.

This paper aims to improve recognition accuracies of single sources of partial discharge (PD) by employing image-oriented feature extraction and selection algorithms. 419 diversified samples of PD data acquired from typically artificial defect models, where the defect size, applied voltage and insulation aging are taken into account, are employed for algorithms validation. In the proposed method, the PD gray images are decomposed into various vectors by two-dimensional principal component analysis (2DPCA) on horizontal and vertical directions, respectively, in which 9 representative parameters are extracted from each image vector. Based on the 419 PD samples, the proposed image features are confirmed to achieve better PD recognition performances than typical image compression methods relevant to 2DPCA. To further improve the recognition performance and reduce the feature dimension, the feature selection technique based on non-dominated sorting genetic algorithm II (NSGA-II) is adopted to search the optimal 2DPCA features. By using the proposed image-oriented feature extraction and selection algorithm, fuzzy k-nearest neighbor classifier (FkNNC), back-propagation neural network (BPNN) and support vector machine (SVM) with 16-dimensional 2DPCA features can now achieve 96.4%, 94.4% and 97.5% accuracies, respectively. The increment of average recognition accuracies of 5%-7% are obtained by different classifiers compared with phase-resolved partial discharge (PRPD) statistical features in previous studies. The investigations on the recognition results relevant to image distortion, cavity discharge change with time and PD contaminated by random noises with different level confirm the robustness of the 16-dimensional 2DPCA features, which signify a great potential in applying the proposed method to field PD pattern recognition in the future.

In brain–computer interfaces (BCIs), it is crucial to process brain signals to improve the accuracy of the classification of motor movements. Machine learning (ML) algorithms such as artificial neural networks (ANNs), linear discriminant analysis (LDA), decision tree (D.T.), K-nearest neighbor (KNN), naive Bayes (N.B.), and support vector machine (SVM) have made significant progress in classification issues. This paper aims to present a signal processing analysis of electroencephalographic (EEG) signals among different feature extraction techniques to train selected classification algorithms to classify signals related to motor movements. The motor movements considered are related to the left hand, right hand, both fists, feet, and relaxation, making this a multiclass problem. In this study, nine ML algorithms were trained with a dataset created by the feature extraction of EEG signals.The EEG signals of 30 Physionet subjects were used to create a dataset related to movement. We used electrodes C3, C1, CZ, C2, and C4 according to the standard 10-10 placement. Then, we extracted the epochs of the EEG signals and applied tone, amplitude levels, and statistical techniques to obtain the set of features. LabVIEW™2015 version custom applications were used for reading the EEG signals; for channel selection, noise filtering, band selection, and feature extraction operations; and for creating the dataset. MATLAB 2021a was used for training, testing, and evaluating the performance metrics of the ML algorithms. In this study, the model of Medium-ANN achieved the best performance, with an AUC average of 0.9998, Cohen’s Kappa coefficient of 0.9552, a Matthews correlation coefficient of 0.9819, and a loss of 0.0147. These findings suggest the applicability of our approach to different scenarios, such as implementing robotic prostheses, where the use of superficial features is an acceptable option when resources are limited, as in embedded systems or edge computing devices.

Today, video data, as a powerful multimedia component, is accompanied by some problems with increasing usage in communication, health, education, and social media in particular. Classification and detection of concepts in video data by automatic methods are some of these challenging problems. In this study, we propose a video classification system, which incorporates deep convolutional neural networks (CNNs) by leveraging feature selection and data fusion techniques to improve the accuracy of the classification. Principal Component Analysis (PCA) as a feature selection method and Discriminant Correlation Analysis (DCA) technique, which incorporates class associations into the correlation analysis of feature sets for data fusion, are applied to the problem at the feature level. Support Vector Machines (SVMs) have been trained with new feature vectors obtained from different deep convolutional neural networks by feature selection and data fusion methods. The proposed method is tested for 38 concepts on TRECVID 2013 SIN video task dataset and the results are evaluated. Our results show that the classification accuracy is improved by 4% with an accuracy of 50.27% when the proposed data fusion and feature selection techniques are used.

Identification of voice disorders has been a vital role in our life nowadays. Acoustic analysis can be useful tool to diagnose voice disorders as a complementary technique to other medicine methods such as Laryngoscopy and Stroboscopy. In this paper, we scrutinized feature reduction techniques such as principal component analysis (PCA) and linear discriminant analysis (LDA) as feature subset extraction methods and individual feature selection (IFS), forward feature selection (FFS), backward feature selection (BFS) and branch and bound feature selection (BBFS) as feature subset selection procedures. Performance of each method is evaluated by different classifiers. Between feature selection methods, individual feature selection followed by SVM classifier shows the best recognition rate of 91.55% and AUC of 95.80% among these methods. The experimental results demonstrated that highest performance could be achieved by recognition rate of 94.26% and AUC of 97.94% using linear discriminant analysis along with support vector machine as a classifier. Also this mixture has lowest order of computational complexity in comparison with other architectures.

Network Intrusion Detection Systems (NIDS) monitor internet traffic to detect malicious activities. Unfortunately, the amount of data that must be analyzed by NIDS is too large. Several feature selection and feature extraction techniques have been proposed to reduce the size of data. Few are focused on finding exactly by how much the dataset should be reduced. The purpose of this paper is to contribute to the finding of that finite amount of data required for successful intrusion detection. A new hybrid algorithm MID-PCA combining PCA (Principal Component Analysis) and mRMR (minimum Redundancy Maximum Relevance — MID evaluation criteria) is proposed. PCA is first applied to the original dataset. Then, mRMR-MID is applied to the intermediary output to further reduce redundancy and maximize relevancy. An exhaustive evaluation of the MID-PCA algorithm is conducted with the KDD Cup'99, a used widely dataset in the network security community. MID-PCA performance was compared to that of PCA and mRMR using two classifiers namely J48 (C4.5) and BayesNet. Experimental results assert the effectiveness of the newly proposed algorithm MID-PCA for NIDS feature extraction compared with PCA and Mutual Information. The newly proposed MID-PCA shows better performance and classification accuracies with reduced datasets of only 4 dimensions for BayesNet (99.77%) and 6 dimensions for J48 (99.94%). This is an improvement over PCA which achieves similar classification accuracy with 12 principal components (twelve dimensions). An extension of this paper will conduct broader experiments using other datasets, then compare results to that of several well known feature reduction algorithms to confirm the superiority of MID-PCA.