A comparative study of different ICA algorithms for hyperspectral image analysis

Independent component analysis (ICA) is a technique that extracts independent source signals by searching for a linear or nonlinear transformation which minimizes the statistical dependence between components. ICA has been used in a variety of signal processing applications including dimensionality reduction in hyperspectral image (HSI) analysis. Due to the computation complexities and convergence rates, ICA is very time-consuming for high volume or dimension data set like hyperspectral images. Hardware implementation provides not only an optimal parallelism environment, but also a potential faster and real-time solution. This work synthesizes a parallel ICA (pICA) algorithm on field programmable gate array (FPGA). In the proposed implementation method, the pICA is partitioned into three temporally independent functional modules, and each of which is synthesized individually with several ICA-related reconfigurable components (RCs) that are developed for reuse and retargeting purpose. All modules are then integrated into a design and development environment for performing many subtasks such as FPGA synthesis, optimization, placement and routing. In a case study, we synthesize the pICA algorithm for hyperspectral image dimensionality reduction on the pilchard reconfigurahle computing platform embedded with Xilinx: VIRTEX V1000E. The FPGA executes at the maximum frequency of 20.161 MHz, and the pilchard board transfers data directly with CPU on the 64-bit memory bus at the maximum frequency of 133MHz. The performance comparisons between the proposed and another two ICA-related FPGA implementations show that the proposed FPGA implementation of pICA has potential in performing complicated algorithms on large volume data sets.

In the present state of affairs, the motive behind every educational organization is to uplift the academic achievement of students. Educational data mining (EDM) is an upward field of research, and it is very helpful for academic institutions to predict the academic performance of the students. Educational datasets are the basis of various predictive models. Quality of these models can be improved by using feature selection (FS). To get the required benefits from the available data, there must be some tools for analysis and prediction. In lieu of the above, machine learning/data mining are most suitable. In educational data mining, for better accuracy of prediction models’ and quality of various educational datasets, feature selection (FS) plays a vital role. Feature selection (FS) algorithms abolish inappropriate information from the repositories of educational background so that performance of classifier in terms of accuracy could be increased and the same could be used for better decision. In lieu of the above, a best feature selection algorithm must be selected. In this paper, two filter selection approaches namely correlation feature selection (CFS) and wrapper-based feature selection have been used to demonstrate the importance of selection of a feature subset for a classification problem. The present paper aims to find the detailed investigation of filter feature selection algorithms along with the classification algorithms on a given dataset. We found result with numerous numbers of features from various Feature selection algorithms and classifiers which will help the researcher to discover the most excellent mixture of filter feature selection algorithms and its associated classifiers. The result indicates that SMO and J48 have the highest accuracy measures with the correlation feature selection algorithms, while Naïve Bayes has the highest accuracy measures with the wrapper subset feature selection algorithms for predicting high, medium and low grade for the students.

Abnormality and presence of neurological brain disorder such as epileptic seizure is diagnosed by analyzing electroencephalography signals accurately. The acquired brain signals are analyzed in time–frequency domains by using wavelet for accurate diagnosis. The online standard EEG database signal is preprocessed to remove power noise and most important eye blink artifact using independent component analysis. Daubechies wavelet is implemented, and decomposition of frequency is carried out up to eight levels. The exact sub-band of frequencies are extracted from band of frequencies which are called as delta band, theta band, alpha band, beta band and gamma band from lower to higher. Suitable features such as Lacunarity, Fluctuation Index, Energy and Entropy, Kolmogorov Entropy, Kurtosis and Skewness are extracted and classified using K-Nearest Neighbor, Support Vector Machine and Probabilistic Neural Network. Performance analysis is carried out by measuring specificity, sensitivity, accuracy and true predictive value. Implementation of independent component analysis to the affected channels in preprocessing block to remove eye blink artifact, decomposition of brain signal using Daubechies wavelet up to eight levels provides accurate diagnosis of epileptic seizure. The proposed seizure detection system provides high metric of performance parameters.

This study evaluates the impact of three Feature Selection (FS) algorithms in an Object Based Image Analysis (OBIA) framework for Very-High-Resolution (VHR) Land Use-Land Cover (LULC) classification. The three selected FS algorithms, Correlation Based Selection (CFS), Mean Decrease in Accuracy (MDA) and Random Forest (RF) based Recursive Feature Elimination (RFE), were tested on Support Vector Machine (SVM), K-Nearest Neighbor, and Random Forest (RF) classifiers. The results demonstrate that the accuracy of SVM and KNN classifiers are the most sensitive to FS. The RF appeared to be more robust to high dimensionality, although a significant increase in accuracy was found by using the RFE method. In terms of classification accuracy, SVM performed the best using FS, followed by RF and KNN. Finally, only a small number of features is needed to achieve the highest performance using each classifier. This study emphasizes the benefits of rigorous FS for maximizing performance, as well as for minimizing model complexity and interpretation.

HiPerSAT, a C++ library and tools, processes EEG data sets with ICA (independent component analysis) methods. HiPerSAT uses BLAS, LAPACK, MPI and OpenMP to achieve a high performance solution that exploits parallel hardware. ICA is a class of methods for analyzing a large set of data samples and extracting independent components that explain the observed data. ICA is used in EEG research for data cleaning and separation of spatiotemporal patterns that may reflect different underlying neural processes. We present two ICA implementations (FastICA and Info-max) that exploit parallelism to provide an EEG component decomposition solution of higher performance and data capacity than current MATLAB-based implementations. Experimental results and the methodology used to obtain them are presented. Integrating HiPerSAT with EEGLAB (A. Delorme and S. Makeig, 2004) is described, as well as future plans for this research.

In this paper, a hidden conditional random field (HCRF) model with independent component analysis (ICA) mixture feature functions is developed for video event classification. Video content analysis problems can be modeled using graphical models. The hidden Markov model (HMM) is a commonly used graphical model, but the HMM has several limitations such as the assumption of observation independence, the form of observation distribution and the Markov chain interaction. Unlike the HMM, the HCRF is a discriminative model without conditional independence assumption of observations, and is more suitable for video content analysis. We formulate the video content analysis problem using a new HCRF framework based on the temporal interactions between video frames. In addition, according to the non-Gaussian property of video event features, a new feature function using the likelihoods of ICA mixture components is proposed for local observation to further enhance the HCRF model. The discriminative power of the HCRF and representation power of the ICA mixture for non-Gaussian distributions are combined in the new model. The new model is applied to the challenging bowling and golf event classifications as case studies. The simulation results support the analysis that the new ICA mixture HCRF (ICAMHCRF) outperforms the existing mixture HMM models in terms of classification accuracy.

One of the important practical applications of computer vision techniques is computer-aided diagnosis (CAD) in medical imaging. It has been shown that CAD can improve the accuracy of breast cancer detection and characterization by radiologists on mammograms. In this chapter, we discuss an important step — feature selection — in classifier design for CAD algorithms. Feature selection reduces the dimensionality of an available feature space and is therefore often used to prevent over-parameterization of a classifier. Many feature selection techniques have been proposed in the literature. We will illustrate the usefulness of genetic algorithms (GAs) for feature selection by comparing GA with a commonly used sequential selection method. A brief introduction to the GA is given and several examples using GA feature selection for the characterization of mammographic lesions are discussed. The examples illustrate the design of a fitness function for optimizing classification accuracy in terms of the receiver operating characteristics of the classifier, the dependence of GA performance on its evolution parameters, and the design of a fitness function tailored to a specific classification task.

Not alike to principal component analysis (PCA) based monitoring statistics (T2 and SPE), the control limits for independent component analysis (ICA) monitoring statistics (I2 , Ie 2 and SPE) cannot be determined directly from a particular approximation distribution due to latent variables do not follow Gaussian distribution. Lee et al. (2004) proposed to use kernel density estimation (KDE) to obtain the control limits. However, the KDE method is very sensitive to the choice of smoothing parameter. Therefore, this study utilizes the support vector machine (SVM) for process fault detection by taking information of ICA extracted statistics as inputs of SVM. The proposed method (named, ICA-SVM) will be implemented in the Tennessee Eastman Process. In which, several multivariate monitoring schemes such as PCA, ICA, modified ICA and ICA- PCA will be also compared to demonstrate the efficiency of proposed method.

ICA (Independent Component Analysis) technique can be used to detect difference in sequence of images. However, the algorithm is usually highly computation intensive. In this paper, FPGA (Field Programmable Gate Array) implementation to speed up the computation of a non-iterative algebraic ICA is described. System level approach to the design of FPGA prototype is explained. Although FPGA does not offer an optimized hardware implementation when compared to ASIC (Application Specific Integrated Circuit), it allows short development time and enables verification of algorithms in hardware at a low cost.

A new algorithm based on the fusion of edge and multispectral information is proposed for the pixel-wise classification of very high-resolution (VHR) remotely sensed imagery. It integrates the multispectral, spatial and structural information existing in the image. The edge feature is first extracted using an improved multispectral edge detection method, which takes into account the original multispectral bands, the linear NDVI, and the independent spectral components extracted by independent component analysis (ICA). Direction-lines are then defined using the edge and multispectral information. Two effective spatial measures are calculated based on the direction-lines in order to describe the contextual information and strengthen the multispectral feature space. Then, the support vector machine (SVM) is employed to classify the hybrid structural-multispectral feature set. In experiments, the proposed spatial measures were compared with the pixel shape index (PSI) and the gray level co-occurrence matrix (GLCM). The experimental results show that the proposed algorithm performs well in terms of classification accuracies and visual interpretation.

The Support Vector Machine (SVM) (Cortes and Vapnik, 1995; Vapnik, 1995; Burges, 1998) is intended to generate an optimal separating hyperplane by minimizing the generalization error without the assumption of class probabilities such as Bayesian classifier. The decision hyperplane of SVM is determined by the most informative data instances, called Support Vectors (SVs). In practice, these SVMs are a subset of the entire training data. By now, SVMs have been successfully applied in many applications, such as face detection, handwritten digit recognition, text classification, and data mining. Osuna et al. (1997) applied SVMs for face detection. Heisele et al. (2004) achieved high face detection rate by using 2nd degree SVM. They applied hierarchical classification and feature reduction methods to speed up face detection using SVMs. Feature extraction and reduction are two primary issues in feature selection that is essential in pattern classification. Whether it is for storage, searching, or classification, the way the data are represented can significantly influence performances. Feature extraction is a process of extracting more effective representation of objects from raw data to achieve high classification rates. For image data, many kinds of features have been used, such as raw pixel values, Principle Component Analysis (PCA), Independent Component Analysis (ICA), wavelet features, Gabor features, and gradient values. Feature reduction is a process of selecting a subset of features with preservation or improvement of classification rates. In general, it intends to speed up the classification process by keeping the most important class-relevant features.

In order to improve the generalization performance of support vector machine (SVM), a support vector machine ensembling method based on independent component analysis (ICA) and fuzzy kernel clustering (FKC) was proposed. The ICA emphasizes the independence between the data characteristics and can effectively obtain a series of independent features, the performance of single SVM can be improved when the SVM was trained on these independent features; The FKC method uses kernel function to expand the feature space, so that the clustering is more accurate and the diversity between sub-SVMs can also be guaranteed. Simulation results on UCI datasets show that the proposed ensembling method can improve the classification precision of SVM and make the ensemble SVM has better generalization property.

Feature selection techniques are enormously applied in a variety of data analysis tasks in order to reduce the dimensionality. According to the type of learning, feature selection algorithms are categorized to: supervised or unsupervised. In unsupervised learning scenarios, selecting features is a much harder problem, due to the lack of class labels that would facilitate the search for relevant features. The selecting feature difficulty is amplified when the data is corrupted by different noises. Almost all traditional unsupervised feature selection methods are not robust against the noise in samples. These approaches do not have any explicit mechanism for detaching and isolating the noise thus they can not produce an optimal feature subset. In this article, we propose an unsupervised approach for feature selection on noisy data, called Robust Independent Feature Selection (RIFS). Specifically, we choose feature subset that contains most of the underlying information, using the same criteria as the Independent component analysis (ICA). Simultaneously, the noise is separated as an independent component. The isolation of representative noise samples is achieved using factor oblique rotation whereas noise identification is performed using factor pattern loadings. Extensive experimental results over divers real-life data sets have showed the efficiency and advantage of the proposed algorithm.

New hybrid data mining model for credit scoring based on feature selection algorithm and ensemble classifiers

Support Vector Machines (SVM) and Independent Component Analysis (ICA) are two powerful and relatively recent techniques. SVMs are classifiers which have demonstrated high generalization capabilities in many different tasks, including the object recognition problem. ICA is a feature extraction technique which can be considered a generalization of Principal Component Analysis (PCA). ICA has been mainly used on the problem of blind signal separation. In this paper we combine these two techniques for the face recognition problem. Experiments were made on two different face databases, achieving very high recognition rates. As the results using the combination PCA/SVM were not very far from those obtained with ICA/SVM, our experiments suggest that SVMs are relatively insensitive to the representation space. Thus as the training time for ICA is much larger than that of PCA, this result indicates that the best practical combination is PCA with SVM.KeywordsSupport Vector MachineFace RecognitionFace ImageIndependent Component AnalysisFace DatabaseThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.