Integrating independent components and support vector machines for gender classification

Since the multi-channel coil of magnetic resonance imaging (MRI) became a mainstream, methods to reduce inhomogeneity in MRI without using the body coil for additional scan have developed. The interference of signals made the appearance of MR image badly. As a result it serious affects the classifications for MR images. In this study we would find the most adaptable method to improve inhomogeneity for MRI and get improvement for brain tissue classification by using independent component analysis (ICA) and support vector machine (SVM) method under multiple-channel phase-array coil. We chose three inhomogeneity correction methods: discrete wavelet transform (DWT), local entropy minimize with B-spline (LEM-BS) and local entropy minimize with cubic spline (LEM-CS). In our experiment, these three methods were used as the pre-processing method before applying ICA + SVM to correct the inhomogeneity of MR image. Web site images are first applied in the experiment and the Tanimoto index was used to measure the performance for the three methods. Real phantom MR images were also applied in this experiment. The results show that the LEM-BS is the best choice. Instead of using the average filter for LEM-BS, we use Gaussian filter to get better classification result. The LEM-BS method can be applied not only in single slice but also in multiple slices and sometimes it shows better result.

The Support Vector Machine (SVM) (Cortes and Vapnik, 1995; Vapnik, 1995; Burges, 1998) is intended to generate an optimal separating hyperplane by minimizing the generalization error without the assumption of class probabilities such as Bayesian classifier. The decision hyperplane of SVM is determined by the most informative data instances, called Support Vectors (SVs). In practice, these SVMs are a subset of the entire training data. By now, SVMs have been successfully applied in many applications, such as face detection, handwritten digit recognition, text classification, and data mining. Osuna et al. (1997) applied SVMs for face detection. Heisele et al. (2004) achieved high face detection rate by using 2nd degree SVM. They applied hierarchical classification and feature reduction methods to speed up face detection using SVMs. Feature extraction and reduction are two primary issues in feature selection that is essential in pattern classification. Whether it is for storage, searching, or classification, the way the data are represented can significantly influence performances. Feature extraction is a process of extracting more effective representation of objects from raw data to achieve high classification rates. For image data, many kinds of features have been used, such as raw pixel values, Principle Component Analysis (PCA), Independent Component Analysis (ICA), wavelet features, Gabor features, and gradient values. Feature reduction is a process of selecting a subset of features with preservation or improvement of classification rates. In general, it intends to speed up the classification process by keeping the most important class-relevant features.

Psychophysics with high dimensional stimuli (e.g. video fragments) can be challenging. Typically, a stimulus is presented with a noise mask but for high res stimulus displays, the number of free parameters increases exponentially inducing the curse of dimensionality. Another question is how such noise embeddings and stimulus manipulations affect the observers’ strategy [Macke &Wichmann 2010]. This research aims to investigate the errors we make in every day life in detecting emotional expressions. Opposed to noise based experiments, in every day life, faces are often fully visible and errors emerge rather from the misinterpretation of ambiguous but visible components of the face. Macke and Wichmann (2010) investigated a similar problem in gender classification. They presented static face images in a parameterized morph-space [Blanz & Vetter 1999]. We applied the same idea to classification of dynamic emotional expressions but we used Active Appearance Models [Cootes et al. 1998] to parameterize shape variations by low dimensional vectors of facial components. Via staircase procedures, we varied the amount of information in the different components. Participants detected dynamic emotional expressions ranging from no ambiguity at all, towards a static image with neutral expression. We determined separate thresholds for different facial components and found that participants were equally accurate in detecting emotional expressions as long as the eyes were informative.

Because of high dimensionality, correlation among covariates, and noise contained in data, dimension reduction (DR) techniques are often employed to the application of machine learning algorithms. Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), and their kernel variants (KPCA, KLDA) are among the most popular DR methods. Recently, Supervised Kernel Principal Component Analysis (SKPCA) has been shown as another successful alternative. In this paper, brief reviews of these popular techniques are presented first. We then conduct a comparative performance study based on three simulated datasets, after which the performance of the techniques are evaluated through application to a pattern recognition problem in face image analysis. The gender classification problem is considered on MORPH-II and FG-NET, two popular longitudinal face aging databases. Several feature extraction methods are used, including biologically-inspired features (BIF), local binary patterns (LBP), histogram of oriented gradients (HOG), and the Active Appearance Model (AAM). After applications of DR methods, a linear support vector machine (SVM) is deployed with gender classification accuracy rates exceeding 95% on MORPH-II, competitive with benchmark results. A parallel computational approach is also proposed, attaining faster processing speeds and similar recognition rates on MORPH-II. Our computational approach can be applied to practical gender classification systems and generalized to other face analysis tasks, such as race classification and age prediction.

Three-dimensional human face modeling is a very popular topic with many applications,such as facial animation, face recognition and model-based facial video communication. Therefore, how to model the facial geometry, texture intensity and illumination variation is important in computer vision and graphics. Previous works on 3D head modeling from a single face image utilized prior information on 3D head models. However, it is difficult to accurately reconstruct the 3D face model from a single face image with expression since the facial expression induces 3D face model deformation in a complex manner. The main challenge is the coupling of the neutral 3D face model and the 3D deformation due to expression, thus making the 3D model estimation from a single face image with expression very challenging. On the other hand, the illumination condition also makes the problem more difficult. In this thesis,we focus on developing a 3D face model reconstruction system including surface registration and training of 3D face models with expressional deformations as well as the estimation of the 3D neutral shape and the 3D expressional deformation from a single face image. The proposed reconstruction algorithm integrates the linear and non-linear subspace representations for a prior 3D neutral morphable model and the probabilistic manifold-based 3D expressional deformation. We incorporate the face geometry, expression deformation, texture and illumination information into the problem so that it is well constrained. The reconstructed 3D face models can also be further extended and applied to many real-world applications.

In this paper, we demonstrate the need for dimensionality reduction to mitigate model overfitting on the nontrivial problem of gender classification from digital images. In this study we explore four feature selection schemes using Genetic Algorithm, Memetic Algorithms, and Random Forest, which are fed to a nonlinear support vector machine (SVM) for final classification. The performance of the model (feature) selection approaches are evaluated against two distinct datasets of facial images: FG-NET which contains toddlers to seniors and the UIUC-PAL which contains faces of adults up to seniors. This work demonstrates that feature selection can, and does, improve performance of an SVM based gender classification system significantly.

By Using data mining techniques, much study has been undertaken to derive information from medical data, which may enhance the curing ability. This study has been compared four automated methods for blood diseases detection using a different method for attributes reduction methods: PCA (Principal Component Analysis), ICA (Independent Component Analysis), original data set, and four algorithms, which are Naive Bayes, K-Nearest Neighbor, Decision Tree, and SVM (Support Vector Machine) for classification. The results show that after applying the PCA technique, the accuracy is: 0.98, 1.0, 0.79, and 0.78 for Naive Bayes, Decision Tree, SVM, and KNN and after applying the ICA technique, the accuracy is 0.76,.76, 0.76, and 0.74 for Naive Bayes, Decision Tree, SVM, and KNN, while the accuracy after applying on the original data set is: 0.98, 1.0, 0.78, and 0.75 for Naive Bayes, Decision Tree, SVM, and KNN. The results appeared that when applying decision tree on the original data set and after applying the PCA method, have been gotten the highest accuracy which is 100 %.

Not alike to principal component analysis (PCA) based monitoring statistics (T2 and SPE), the control limits for independent component analysis (ICA) monitoring statistics (I2 , Ie 2 and SPE) cannot be determined directly from a particular approximation distribution due to latent variables do not follow Gaussian distribution. Lee et al. (2004) proposed to use kernel density estimation (KDE) to obtain the control limits. However, the KDE method is very sensitive to the choice of smoothing parameter. Therefore, this study utilizes the support vector machine (SVM) for process fault detection by taking information of ICA extracted statistics as inputs of SVM. The proposed method (named, ICA-SVM) will be implemented in the Tennessee Eastman Process. In which, several multivariate monitoring schemes such as PCA, ICA, modified ICA and ICA- PCA will be also compared to demonstrate the efficiency of proposed method.

Describes a pattern classification approach for detecting frontal-view faces via learning a decision boundary. The classification can be achieved either by explicit estimation of density functions of two classes, face and non-face or by direct learning of a classification function (decision boundary). The latter is a more effective approach, when the number of training available examples is small, compared to the dimensionality of image space. The proposed method consists of a implicit modeling of both face and near-face classes using Independent Component Analysis (ICA), and a subsequent classification stage based on the decision boundary estimation using Support Vector Machine (SVM). Multiple nonlinear SVMs are trained for local subspaces, considering the general non-Gaussian and multi-modal characteristic of face space. This parallelization of SVMs reduces computational cost of on-line classification, since the locally trained SVM has small number of support vectors compared to the SVM trained on entire data space. We showed that the proposed algorithm is superior to the simple combination of ICA and SVM, both in accuracy and computational burden.

We propose a new hybrid unsupervised/supervised learning scheme that integrates independent component analysis (ICA) with the support vector machine (SVM) approach and apply this new learning scheme to the face detection problem. In low-level feature extraction, ICA produces independent image bases that emphasize edge information in the image data. In high-level classification, SVM classifies the ICA features as a face or non-faces. Our experimental results show that by using ICA features we obtain a larger margin of separation and fewer support vectors than by training SVM directly on the image data. This indicates better generalization performance, which is verified in our experiments.

Cross-validation is a normal method for parameter selection of support vector machine (SVM) which is a novel machine learning method for hyperspectral data classification. Because of the high dimensionality of hyperspectral data, the process of cross-validation will cost more time. For reducing the time of cross-validation and improving classification accuracy, a new combination method of improving sequential minimal optimization (SMO), independent component analysis (ICA) and mixture kernels is proposed. It can be described as follows: first use the improving SMO method to optimize the model of SVM, and then use ICA method to do dimensionality reduction before cross-validation, at last use mixture kernels to do classification of unknown samples. By the experiments, it is proved that this method can guarantee the accuracy of unknown samples classification while reducing the time of cross-validation.

Independent Component Analysis (ICA) is an unsupervised source separation method that has been widely used in signal processing. In recent years ICA has been exploited also in remote sensing image processing, and particularly in hyperspectral image analysis to extract useful information for classification and unmixing problems. In this paper a comparative study between the three most widely used ICA implementations (FastICA, Infomax, JADE) is presented. ICA is exploited in terms of feature extraction technique in order to retrieve class discriminant information for the supervised classification. Two different experiments are proposed and analyzed. Firstly, the spectral dimensionality is reduced by employing a supervised feature selection (FS), followed by ICA in order to obtain a new feature space. In the second experiment ICA is applied on the whole dataset and the most informative features are selected from the FS algorithm. The final results are compared and presented in terms of classification accuracies, obtained by Support Vector Machine (SVM).

In order to improve the generalization performance of support vector machine (SVM), a support vector machine ensembling method based on independent component analysis (ICA) and fuzzy kernel clustering (FKC) was proposed. The ICA emphasizes the independence between the data characteristics and can effectively obtain a series of independent features, the performance of single SVM can be improved when the SVM was trained on these independent features; The FKC method uses kernel function to expand the feature space, so that the clustering is more accurate and the diversity between sub-SVMs can also be guaranteed. Simulation results on UCI datasets show that the proposed ensembling method can improve the classification precision of SVM and make the ensemble SVM has better generalization property.

Support Vector Machines (SVM) and Independent Component Analysis (ICA) are two powerful and relatively recent techniques. SVMs are classifiers which have demonstrated high generalization capabilities in many different tasks, including the object recognition problem. ICA is a feature extraction technique which can be considered a generalization of Principal Component Analysis (PCA). ICA has been mainly used on the problem of blind signal separation. In this paper we combine these two techniques for the face recognition problem. Experiments were made on two different face databases, achieving very high recognition rates. As the results using the combination PCA/SVM were not very far from those obtained with ICA/SVM, our experiments suggest that SVMs are relatively insensitive to the representation space. Thus as the training time for ICA is much larger than that of PCA, this result indicates that the best practical combination is PCA with SVM.KeywordsSupport Vector MachineFace RecognitionFace ImageIndependent Component AnalysisFace DatabaseThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Indonesian social media platforms, particularly X (formerly Twitter), generate short and highly informal texts that contain linguistic cues useful for demographic inference. Given the scarcity of controlled comparative studies in Indonesian gender prediction, especially with modest datasets, this research evaluates the performance of Multinomial Naïve Bayes, Linear Support Vector Machine (SVM), and Bidirectional Long Short-Term Memory (BiLSTM) using a balanced corpus of 478 manually labeled Indonesian-language tweets. These three models were selected to represent classical probabilistic learning, margin-based linear classification, and neural sequence modeling, thereby enabling a methodologically coherent comparison across distinct algorithmic paradigms. The study implemented a unified workflow consisting of manual labeling, structured preprocessing with Sastrawi stemming, RandomOverSampler for class balancing, TF-IDF features for classical models, and sequence-based tokenization for BiLSTM. All models were trained and evaluated using a stratified 80:20 split. Experimental results show that Linear SVM achieved the strongest performance, reaching 0.833 accuracy and 0.832 macro-F1, surpassing Naïve Bayes (0.771 accuracy) and BiLSTM (0.740 accuracy). SVM also demonstrated the most stable confusion-matrix distribution and superior AUC characteristics, while BiLSTM exhibited fluctuating validation curves, indicating sensitivity to the limited dataset size. These findings reinforce that classical models—particularly Linear SVM remain highly competitive for Indonesian short-text gender classification in low-resource environments and offer practical advantages where computational constraints and data scarcity are prominent. Although the dataset is topically narrow and limited in scale, the results highlight the need for larger corpora or transformer-based Indonesian models to further enhance generalizability and downstream demographic inference.