A comparative dimensionality reduction study in telecom customer segmentation using deep learning and PCA

Despite the fact that technology is progressing, people continue to consume deadly wild mushrooms due to their inability in classifying the different mushroom categories. Currently, no specific traits are defined to accurately predict the edibility of mushrooms based on their attributes. To overcome this challenge, Machine Learning [ML] can be used for identifying the poisonous mushroom based on the feature appearance. By considering the above, the Mushroom dataset extracted from UCI data warehouse are used for predicting the mushroom edibility level. The division of mushroom edible classes are achieved in four different ways. Firstly, the dataset is preprocessed with feature scaling and missing values. Secondly, raw data set is fitted to all the classifier with and without the presence of feature scaling. Thirdly, raw data is applied with the principal component analysis with 8, 10 and 12 components and PCA reduced dataset is fitted to all the classifier with and without the presence of feature scaling. Fourth, the raw data is applied with the Linear discriminant analysis and LDA reduced dataset is fitted to all the classifier with and without the presence of feature scaling. Fifth, raw data is applied with the Factor analysis with 8, 10 and 12 components and FA reduced dataset is fitted to all the classifier with and without the presence of feature scaling. Sixth, the performance of raw data set, PCA reduced data set, LDA reduced dataset and FA reduced dataset are compared by analyzing the performance metrics like Precision, Recall, Accuracy and F-score. The implementation is done by using python language under Spyder platform with Anaconda Navigator. Experimental results show that, the kernel SVM, KNN and Adaboost classifier for the FA reduced dataset tends to retain the accuracy with 100% before and after feature scaling. The KNN classifier with LDA reduced dataset tends to retain the 96% accuracy before and after feature scaling. Kernel SVM, KNN and Adaboost classifier for the FA reduced dataset tends to retain the accuracy with 99% before and after feature scaling. From the above analysis, KNN classifier ismore efficient based on its accuracy with all PCA, LDA and FA reduced dataset.KeywordsMachine learningClassificationPrecisionAccuracyPCALDA and factor analysis

Principal component analysis (PCA) is known to be a powerful linear technique for data set dimensionality reduction. This paper focuses on revealing the essence of PCA to interpret the data, which is to identify the internal structure of the random process from a large experimental data set. We give an explanation of the PCA procedure performed on a generated data set to demonstrate the exact meaning of the dimensionality reduction. Especially, a method is proposed to precisely determine the number of significant principal components for a random process. Then, the internal structure of the random process can be modeled by analyzing the relation between the PCA results and the original data set. This is vital in the efficient random process modeling, which is finally applied to an application in HRTF Modeling.

<span>The curse of dimensionality and the empty space phenomenon emerged as a critical problem in text classification. One way of dealing with this problem is applying a feature selection technique before performing a classification model. This technique helps to reduce the time complexity and sometimes increase the classification accuracy. This study introduces a feature selection technique using K-Means clustering to overcome the weaknesses of traditional feature selection technique such as principal component analysis (PCA) that require a lot of time to transform all the inputs data. This proposed technique decides on features to retain based on the significance value of each feature in a cluster. This study found that k-means clustering helps to increase the efficiency of KNN model for a large data set while KNN model without feature selection technique is suitable for a small data set. A comparison between K-Means clustering and PCA as a feature selection technique shows that proposed technique is better than PCA especially in term of computation time. Hence, k-means clustering is found to be helpful in reducing the data dimensionality with less time complexity compared to PCA without affecting the accuracy of KNN model for a high frequency data.</span>

The aim of data transformation is to transform the original feature space of data into another space with better properties. This is typically combined with dimensionality reduction, so that the dimensionality of the transformed space is smaller. A widely used method for data transformation and dimensionality reduction is Principal Component Analysis (PCA). PCA finds a subspace that explains most of the data variance. While the new PCA feature space has interesting properties, such as removing linear correlation, PCA is an unsupervised method. Therefore, there is no guarantee that the PCA feature space will be the most appropriate for supervised tasks, such as classification or regression. On the other hand, 3-layer Multi Layer Perceptrons (MLP), which are supervised methods, can also be understood as a data transformation carried out by the hidden layer, followed by a classification/regression operation performed by the output layer. Given that the hidden layer is obtained after a supervised training process, it can be considered that it is performing a supervised data transformation. And if the number of hidden neurons is smaller than the input, also dimensionality reduction. Despite this kind of transformation being widely available (any neural network package that allows access to the hidden layer weights can be used), no extensive experimentation on the quality of 3-layer MLP data transformation has been carried out. The aim of this article is to carry out this research for classification problems. Results show that, overall, this transformation offers better results than the PCA unsupervised transformation method.

Dimensionality reduction through convolutional autoencoders for fracture patterns prediction

This paper carries on research surrounding the influences produced by dimensionality reduction on machine learning classification effect. Firstly, paper constructs the analysis architecture of data dimension reduction classification, combines the two different unsupervised dimension reduction methods, locally linear embedding (LLE) and principal component analysis (PCA) with the five machine learning classification methods: Gradient Boosting Decision Tree (GBDT), Random Forest, Support Vector Machine (SVM), K-Nearest Neighbor (KNN) and Logistic Regression. And then uses the handwritten digital identification dataset to analyze the classification performance of these five classification methods on different dimension datasets by different dimensionality reduction methods. The analysis shows that using the appropriate dimensionality reduction method for dimensionality reduction classification can effectively improve the classification accuracy; the dimensionality reduction classification effect of non-linear dimensionality reduction method is generally better than the linear dimensionality reduction method; different machine learning classification algorithms have significant differences in the sensitivity of dimensions.

Data dimension reduction (DDR) is all about mapping data from high dimensions to low dimensions, various techniques of DDR are being used for image dimension reduction like Random Projections, Principal Component Analysis (PCA), the Variance approach, LSA-Transform, the Combined and Direct approaches, and the New Random Approach. Auto-encoders (AE) are used to learn end-to-end mapping. In this paper, we demonstrate that pre-processing not only speeds up the algorithms but also improves accuracy in both supervised and unsupervised learning. In pre-processing of DDR, first PCA based DDR is used for supervised learning, then we explore AE based DDR for unsupervised learning. In PCA based DDR, we first compare supervised learning algorithms accuracy and time before and after applying PCA. Similarly, in AE based DDR, we compare unsupervised learning algorithm accuracy and time before and after AE representation learning. Supervised learning algorithms including support-vector machines (SVM), Decision Tree with GINI index, Decision Tree with entropy and Stochastic Gradient Descent classifier (SGDC) and unsupervised learning algorithm including K-means clustering, are used for classification purpose. We used two datasets MNIST and FashionMNIST Our experiment shows that there is massive improvement in accuracy and time reduction after pre-processing in both supervised and unsupervised learning.

The performance of gas-sensor array systems is greatly influenced by the pattern recognition scheme applied on the instrument's measurement data. The traditional method of choice is principal component analysis (PCA), aiming for reduction in dimensionality and visualization of multivariate measurement data. PCA, as a second-order statistical tool, performs well in many cases, but lacks the ability to give meaningful representations for non-Gaussian data, which often is a property of gas-sensor array measurement data. If, instead, higher order statistical methods are considered for data analysis, more useful information can be extracted from the data. This paper introduces the higher order statistical method called independent component analysis (ICA) as a novel tool for analysis of gas-sensor array measurement data. A comparison between the performances of PCA and ICA is illustrated both in theory and for two sets of practical measurement data. The described experiments show that ICA is capable of handling sensor drift combined with improved discrimination, dimensionality reduction, and more adequate data representation when compared to PCA.

The use of magnetic resonance imaging (MRI) in the diagnosis of dementia is significant because it reveals morphological differences induced by brain decline. The major obstacles faced by compuer assisted systems are: High dimensionality and unbalanced data. Numerous deep learning methods have been used on an MR image dataset for disease classification and prediction, but most of them have refused to endorse data dimensionality reduction, which resulted in skewed outcomes. The UCI machine learning repository provided the MR image of brain dataset utilised in this study. In this article, we suggest an productive dimensional reduction method to ameliorate the attainment of the dementia detection system.. The hybrid Linear Discriminant Analysis –Firefly algorithm (LDA-FA) is used for dimensionality reduction. On the reduced data set we apply an optimized convolutional neural network. The method we have proposed is less erroneous and more accurate than other recent work and also produced 95.21% accuracy. KeywordsAlzheimer’s Disease (AD)Convolutional Neural Network (CNN)DementiaDimensionality reductionFireflyLinear Discriminant Analysis (LDA)

Objectives This study aimed to use artificial intelligence (AI) computing techniques to determine if they can validate the findings of a previously published thematic analysis article focusing on disaster medicine experts’ open-ended feedback about Middle East and North African countries (MENA) for chemical, biological, radiological, and nuclear (CBRN) threats. Methods Automated text analytics techniques were employed to explore and visualize the semantic essence of the experts’ feedback through word vector transformation and Principal Component Analysis (PCA) for dimensionality reduction. The t-distributed Stochastic Neighbor Embedding (t-SNE) is another more advanced dimensionality reduction technique that enhanced the capturing of the determined components. Results Two prominent clusters emerged from the full textual data set representing word similarities groups in the original data set, denoting a thematic group of ideas that experts have emphasized in their responses. Upon deep reading the text feedback, the themes linked preparedness with different training types, such as tabletop exercises and policies/legislation. The findings are in line with currently adopted practices. Conclusions While AI methods demonstrated their valuable application in disaster medicine and helped validate the experts’ recommendations objectively, they should be approached cautiously, as they can be complex and challenging to comprehend fully.

In the ceramic production process, the content of Si, Al, Mg, Fe, Ti and other elements in the ceramic raw materials has an important impact on the quality of the ceramic products. Exploring a method that can quickly and accurately analyze the content of key elements in ceramic raw materials is of great significance to improve the quality of ceramic products. In this work, laser-induced breakdown spectroscopy (LIBS) is used for rapid analysis of ceramic raw materials. The chemical element composition and content of ceramic raw materials are quite different, which leads to serious matrix effects. Building an artificial neural network model is an effective way to solve the complex matrix effects, but model training can easily lead to overfitting due to the high number of spectral features and the limited number of samples. In order to solve this problem, we propose a feature extraction method that combines the linear regression (LR) and the sparse and under-complete autoencoder (SUAC) neural network. This LR + SUAC method performs nonlinear feature extraction and dimension reduction on high-dimensional spectral data. The spectral data dimension is reduced from 8188 to 100 through the LR layer, and further reduced to 32 through the SUAC encoding layer. Further, a quantitative analysis model for the elemental composition of ceramic raw materials is established by the combination of LR + SUAC and Back Propagation Neural Network (BPNN). Since the input data dimension and redundant information are greatly reduced by LR + SUAC, the overfitting problem of BPNN is greatly reduced. Experiment results showed that the LR + SUAC + BPNN method obtained the best quantitative analysis performance compared with several other methods in the cross-validation process.

This paper proposes a method for concentrating multivariate data (dimensionality reduction) using a neural network model. Specifically, a pulse-input pattern-output network (PPN), i.e., a multilayer network in which the input is presented as a pulsed input (a single true input with the others being held to zero Here, by pulsed input, we mean that only one component of the input vector is presented with a value of 1 and has no connection with pulsed signals in the time domain.), is employed and the N-dimensional training pattern is presented at the outputs. Using backpropagation learning in a PPN, it is demonstrated mathematically that subject to a certain set of conditions, the dimension of the sample space can be reduced arbitrarily while preserving optimality. That capability is compared with principal component analysis (K-L expansion), and it is demonstrated that the nonlinear dimensionality reduction capabilities of the model by means of a component analysis experiment employ the iris data—a data set which is well known in the field of multivariate analysis.

BackgroundUnsupervised clustering and outlier detection are important in medical research to understand the distributional composition of a collective of patients. A number of clustering methods exist, also for high-dimensional data after dimension reduction. Clustering and outlier detection may, however, become less robust or contradictory if multiple high-dimensional data sets per patient exist. Such a scenario is given when the focus is on 3-D data of multiple organs per patient, and a high-dimensional feature matrix per organ is extracted.MethodsWe use principal component analysis (PCA), t-distributed stochastic neighbor embedding (t-SNE) and multiple co-inertia analysis (MCIA) combined with bagplots to study the distribution of multi-organ 3-D data taken by computed tomography scans. After point-set registration of multiple organs from two public data sets, multiple hundred shape features are extracted per organ. While PCA and t-SNE can only be applied to each organ individually, MCIA can project the data of all organs into the same low-dimensional space.ResultsMCIA is the only approach, here, with which data of all organs can be projected into the same low-dimensional space. We studied how frequently (i.e., by how many organs) a patient was classified to belong to the inner or outer 50% of the population, or as an outlier. Outliers could only be detected with MCIA and PCA. MCIA and t-SNE were more robust in judging the distributional location of a patient in contrast to PCA.ConclusionsMCIA is more appropriate and robust in judging the distributional location of a patient in the case of multiple high-dimensional data sets per patient. It is still recommendable to apply PCA or t-SNE in parallel to MCIA to study the location of individual organs.

Cloud computing offers the possibility to store and process massive amounts of remotely sensed hyperspectral data in a distributed way. Dimensionality reduction is an important task in hyperspectral imaging, as hyperspectral data often contains redundancy that can be removed prior to analysis of the data in repositories. In this regard, the development of dimensionality reduction techniques in cloud computing environments can provide both efficient storage and preprocessing of the data. In this paper, we develop a parallel and distributed implementation of a widely used technique for hyperspectral dimensionality reduction: principal component analysis (PCA), based on cloud computing architectures. Our implementation utilizes Hadoop's distributed file system (HDFS) to realize distributed storage, uses Apache Spark as the computing engine, and is developed based on the map-reduce parallel model, taking full advantage of the high throughput access and high performance distributed computing capabilities of cloud computing environments. We first optimized the traditional PCA algorithm to be well suited for parallel and distributed computing, and then we implemented it on a real cloud computing architecture. Our experimental results, conducted using several hyperspectral datasets, reveal very high performance for the proposed distributed parallel method.

The non-stationary nature of energy data is a serious challenge for energy forecasting methods. Frequent model updates are necessary to adapt to distribution shifts and avoid performance degradation. However, retraining regression models with lookback windows large enough to capture energy patterns is computationally expensive, as increasing the number of features leads to longer training times. To address this problem, we propose an approach that guarantees fast convergence through dimensionality reduction. Using a synthetic neighborhood dataset, we first validate three deep learning models—an artificial neural network (ANN), a 1D convolutional neural network (1D-CNN), and a long short-term memory (LSTM) network. Then, in order to mitigate the long training time, we apply principal component analysis (PCA) and a variational autoencoder (VAE) for feature reduction. As a way to ensure the suitability of the proposed models for a residential context, we also explore the trade-off between low error and training speed by considering three test scenarios: a global model, a local model for each building, and a global model that is fine-tuned for each building. Our results demonstrate that by selecting the optimal dimensionality reduction method and model architecture, it is possible to decrease the mean squared error (MSE) by up to 63% and accelerate training by up to 80%.