Optimal feature selection for SVM based weed classification via visual analysis

Optimal feature selection for classification of the power quality events using wavelet transform and least squares support vector machines

Since computer-aided diagnosis (CAD) schemes of medical images usually computes large number of image features, which creates a challenge of how to identify a small and optimal feature vector to build robust machine learning models, the objective of this study is to investigate feasibility of applying a random projection algorithm (RPA) to build an optimal feature vector from the initially CAD-generated large feature pool and improve performance of machine learning model. We assemble a retrospective dataset involving 1,487 cases of mammograms in which 644 cases have confirmed malignant mass lesions and 843 have benign lesions. A CAD scheme is first applied to segment mass regions and initially compute 181 features. Then, support vector machine (SVM) models embedded with several feature dimensionality reduction methods are built to predict likelihood of lesions being malignant. All SVM models are trained and tested using a leave-one-case-out cross-validation method. SVM generates a likelihood score of each segmented mass region depicting on one-view mammogram. By fusion of two scores of the same mass depicting on two-view mammograms, a case-based likelihood score is also evaluated. Comparing with the principle component analyses, nonnegative matrix factorization, and Chi-squared methods, SVM embedded with RPA yielded a significantly higher case-based lesion classification performance with the area under ROC curve of 0.84 ± 0.01 (p<0.02). The study demonstrates that RPA is a promising method to generate optimal feature vectors and improve SVM performance. This study presents a new method to develop CAD schemes with significantly higher and robust performance.

In this paper, we present a Face Recognition method based on the combination of the Gabor Wavelets (GW) and the Phase Congruency (PC) method [1]. While the GW method allows an efficient space-frequency analysis to code the Face Image points, it requires significant down-sampling, in order to end up with a reasonably small-size feature vector. Now, the Phase Congruency method can be used as an efficient features detector, to uncover the salient local features of the face and can be computed using the GW Transform. An additional advantage of the Phase Congruency method is its insensitivity to variations in image illumination and contrast. The method proposed is to use the PC method in determining the salient points of the Face Image, thus replacing the random down-sampling method. The first step in our proposed method is to obtain the local frequency information of a face image using a multi-scale, multi-orientation set of Gabor filters, from which the Phase Congruency image is computed. We then use this method to select a limited number of maximum-value points, whose values are taken from the GW- transformed face images, forming a feature vector of about 5000 components. Preliminary results using the ORL face dataset show 98% recognition rate even without the use of the PCA method. Upon using the PCA method we can further reduce the number of components to 45 while still retaining the recognition rate of 98%. This as compared to a recognition rate of 96% using GW and PCA with a random 64:1 down-sampling on the GW coefficients. The proposed combined method, allows in addition to an efficient reduction in the number of image points used, the derivation of both the PC image and the GW feature vector representation, using a single GW transform.

Objective: To identify optimum features from the initial feature set to classify ship ISAR images with good accuracy. The optimum feature selection brings down the computational complexity and also removes the redundant features in the feature vector. Methodology: Colour moments computed from R, G, B, RG, GB, RB component images are taken as one feature vector. The average wave energy values of horizontal, vertical and diagonal details of images decomposed up to five levels using wavelet transform are considered as second feature vector. The ship ISAR images measured with high resolution Radar are used as data set to test the classification performance of the two feature sets, which is explored uniquely in this work. Genetic algorithm and particle swarm optimization techniques are used to find the optimum feature sets. Probabilistic neural network is used to compare the classification accuracy achieved with full length feature vectors and optimum feature vectors of the two considered feature sets. Findings: Data set of two ships is taken. The data of each ship is divided in to two parts training data and test data. The feature vectors computed from training set are used to train the classifier and that computed from test set are used to test the classification accuracy. The number of variables (number of features in the feature vector) of optimization algorithms Genetic Algorithm (GA) and Particle Swarm Optimization (PSO) is varied to find the optimum feature set to classify the ship ISAR image data. The function that computes the classification accuracy is used as the objective function of the optimization algorithms. From the initial feature vector of thirty colour moments a set of four features is identified that gives 100% accuracy for both the targets considered. From starting set of fifteen average energy levels, a set of four energy levels identified by PSO and the set of three energy levels are found to be more effective in classification of the ship ISAR images with classification accuracy of 66.6% for the first target and 75% for the second target. Both the optimization techniques performed in similar way. Other optimization algorithms can be explored to find feature combinations that classify ship ISAR images with better accuracy. Application: Classification based on ISAR images has application in automatic non-cooperative target recognition, development of monitoring systems for surveillance along transportation corridors etc. to avoid illegal immigration. Future Work: Different classifiers Self Organizing Maps, Support Vector Machines etc. can be used to study the possibility of obtaining optimum solution in smaller time. Keywords: Classification, Feature Selection, ISAR image, Optimum Feature Set, Optimization Techniques

Co-firing coal and biomass has been applied in existing coal-fired power stations recently. Online blend-type identification was investigated by support vector machine (SVM) using flame emission spectrum for combustion optimisation. A spectrometer was used to capture the flame emission spectrum during co-combustion in a 0.3 MW furnace. A total of 22 flame features were defined and extracted from the flame emission spectrum for blend-type identification. ReliefF was applied to calculate the important weights of the extracted fame features. Alkali metals atomic excitation spectral intensities and the means of spectral signals show obviously higher important weights than the other flame features. Ultraviolet signal is more important than visible and infrared signals for blend-type identification. SVM was adopted to identify the blend types. The method of ‘ReliefF + SVM’ was proposed to obtain the optimum feature vector. The number of optimum features can be reduced from 22 to 17 if only the prediction accuracy is considered. The optimum sampling number is 12. At the optimum feature vector (17 features) and the optimum sampling number (12), the average prediction accuracy of the five fuels is 99.67%. The results demonstrate that combining SVM and flame emission spectrum is suitable for online blend-type identification during co-combustion.

Background: Heat shock proteins (HSPs) ubiquitously expressed in both prokaryotes and eukaryotes. According to their molecular mass and function, HSPs are classified into different families which are structurally different and play distinct functions in biological processes. Although some efforts have been made for identifying the types of HSPs, there is no method available that can be used to identify the types of HSPs in plants. Method: The amino acid distributions in the different types of HSPs are anazlyed. HSPs are encoded using the reduced amino acid alphabet (RAAA). By comparing the predictive capability of models based on the composition of RAAA with different sizes, the optimal feature vector was obtained. A support vector machine based model was developed to identify the types of HSPs by using the optimal feature vector. Results: The amino acid distributions are different among the different families of HSPs. In the rigorous jackknife test, the proposed method obtained an accuracy of 93.65% for identifying the five families of HSPs in plant. Conclusions: We hope the proposed method will become a useful tool to identify the types of HSPs in plants. Keywords: Biological processes, heat shock protein, reduced amino acid, support vector machine, n-peptide, jackknife test.

In this study, a feature vector optimization based method has been proposed for classification of the heartbeat types. Electrocardiogram (ECG) signals of five different heartbeat type were used for this aim. Firstly, wavelet transform (WT) method were applied on these ECG signals to generate all feature vectors. Optimizing these feature vectors is provided by performing particle swarm optimization (PSO), genetic search, best first, greedy stepwise and multi objective evoluationary algorithms on these vectors. These optimized feature vectors are later applied to the classifier inputs for performance evaluation. A comprehensive assessment was presented for the determination of optimized feature vectors for ECG signals and best-performing classifier for these optimized feature vectors was determined.

For classified and sensitive electronic documents within the scope of enterprises and organizations, in order to standardize and strengthen the confidentiality management of enterprises and meet the actual needs of secret text classification, a document automatic classification optimization method based on keyword retrieval and the kNN classification algorithm is proposed. The method supports keyword classification management, provides users with keywords of multiple risk levels, and then combines a matching scanning algorithm to label keywords of different levels. The text with labels is used as the training set of the kNN algorithm to classify the target text and realize the classification protection of text data. Aimed at solving the shortcomings of large feature vector dimension, low classification efficiency, and low accuracy in existing kNN text classification methods, an optimization method is proposed using a feature selection algorithm and a kNN algorithm based on an AVX instruction set to realize real-time classification of massive texts. By constructing a keyword dictionary and an optimized feature vector, parallel calculation of the feature vector weight and distance vector is realized, and the accuracy and efficiency of text classification are improved. The experimental results show that the multi-classification effect of the feature selection algorithm used in this paper, tf-DE, is better than that of the traditional tf-idf algorithm, and the classification effect of kNN is comparable to that of the support vector machine (SVM) algorithm. With the increase in feature vector dimensions, the classification effect of the text classification algorithm is improved and the classification time also increases linearly. The AVX-256 acceleration method takes about 55% of the time of the original version, thus verifying the effect of multi-classification of massive texts for document confidentiality management.

Diagnosis of incipient faults for analog circuits is very important, yet very difficult. A novel approach for incipient faults in analog circuits is proposed. Firstly, the statistical property feature vector, which is composed of range, mean, standard deviation, skewness, kurtosis, entropy and centroid, is used to reflect the global property of output response. Then, the least squares support vector machine (LSSVM) is used for diagnostics of the incipient faults in analog circuits. Traditionally, multi-fault diagnosis for analog circuits based on SVM usually uses a single feature vector to train all binary SVM classifier. However, in fact, each binary SVM classifier has different classification accuracy for different feature vectors. Therefore, the Mahalanobis distance (MD) based on particle swarm optimization (PSO) is proposed to select a near-optimal feature vector and decrease the dimensions of the feature vector for each binary classifier. The experiment results show as following: (1) The accuracy using the near-optimal feature vectors is better than the accuracy using a single vector; (2) The consuming time of the near-optimal feature vectors selected by MD based on PSO is reduced about 98% in comparison to the time of the optimal feature vectors selected by the exhaustive method.

BackgroundAntioxidant proteins are involved in several biological processes and can protect DNA and cells from the damage of free radicals. These proteins regulate the body's oxidative stress and perform a significant role in many antioxidant-based drugs. The current invitro-based medications are costly, time-consuming, and unable to efficiently screen and identify the targeted motif of antioxidant proteins.MethodsIn this model, we proposed an accurate prediction method to discriminate antioxidant proteins namely StackedEnC-AOP. The training sequences are formulation encoded via incorporating a discrete wavelet transform (DWT) into the evolutionary matrix to decompose the PSSM-based images via two levels of DWT to form a Pseudo position-specific scoring matrix (PsePSSM-DWT) based embedded vector. Additionally, the Evolutionary difference formula and composite physiochemical properties methods are also employed to collect the structural and sequential descriptors. Then the combined vector of sequential features, evolutionary descriptors, and physiochemical properties is produced to cover the flaws of individual encoding schemes. To reduce the computational cost of the combined features vector, the optimal features are chosen using Minimum redundancy and maximum relevance (mRMR). The optimal feature vector is trained using a stacking-based ensemble meta-model.ResultsOur developed StackedEnC-AOP method reported a prediction accuracy of 98.40% and an AUC of 0.99 via training sequences. To evaluate model validation, the StackedEnC-AOP training model using an independent set achieved an accuracy of 96.92% and an AUC of 0.98.ConclusionOur proposed StackedEnC-AOP strategy performed significantly better than current computational models with a ~ 5% and ~ 3% improved accuracy via training and independent sets, respectively. The efficacy and consistency of our proposed StackedEnC-AOP make it a valuable tool for data scientists and can execute a key role in research academia and drug design.

A new scene classification method is proposed based on the combination of local Gabor features with a spatial pyramid matching model. First, new local Gabor feature descriptors are extracted from dense sampling patches of scene images. These local feature descriptors are embedded into a bag-of-visual-words (BOVW) model, which is combined with a spatial pyramid matching framework. The new local Gabor feature descriptors have sufficient discrimination abilities for dense regions of scene images. Then the efficient feature vectors of scene images can be obtained byK-means clustering method and visual word statistics. Second, in order to decrease classification time and improve accuracy, an improved kernel principal component analysis (KPCA) method is applied to reduce the dimensionality of pyramid histogram of visual words (PHOW). The principal components with the bigger interclass separability are retained in feature vectors, which are used for scene classification by the linear support vector machine (SVM) method. The proposed method is evaluated on three commonly used scene datasets. Experimental results demonstrate the effectiveness of the method.

In the field of computer vision and image processing CBIR (Content based image retrieval) system become an active research area. The CBIR technique involves feature extraction based on image features (like color, shape and texture) and defining rules for comparing image. A classifier is also used by authors for image classification. For this purpose an image database taken from James Z. Wang research group website is used by many authors. Due to enormous increase in the size of database traditional methods of image retrieval by using keywords does not provide accurate result. Thus the concept of CBIR system is emerged. In last couple of years many method of feature extraction is developed but since now researcher are not able to develop a system which has gained high accuracy of human visual perception in distinguishing pictures. Therefore earlier researchers have concentrated to increase the retrieval accuracy of CBIR system by using different combination of features. The objective of this paper is to present review on a CBIR technique which combined four kinds of features set including every kind of feature for developing feature vector. Color based features (histogram), statistical features (mean, median and standard deviation), spatial features (Gabor feature and wavelet features) and texture features (GLCM) is used for forming feature vector. A SVM classifier is used with CBIR system to increase retrieval accuracy. SVM (support vector machine) is a machine learning classifier which provides good classification accuracy. Keywords: CBIR, SVM, NN, GLCM, HSV Color space, Euclidean distance matrix.

In this study, a hybrid security system is proposed. The proposed system is composed of two subsystems namely iris recognition system (IRS) and speaker recognition system (SRS). Pre-processing, feature extraction and feature matching are the main steps of these systems. In IRS subsystem, Gaussian filter, Canny edge detector, Hough transform, and histogram equalization is performed for pre-processing, respectively. After that, by applying 4-level Discrete Wavelet Transform (DWT) to pure iris image, the iris image is decomposed into four sub-bands (LL4, LH4, HL4 and HH4). In order to extract the feature vector from iris pattern, the LH4, HL4 and HH4 sub-bands (matrices) are merged into one matrix. Finally the matrix is transformed in vector to obtain the feature vector of iris image. For SRS subsystem, the pre-processing step includes spectral arrangement, silence part removing and band limitation operations. After pre-processing, frame blocking and windowing are applied to the long-term speech samples and then Fast Fourier Transform (FFT) is performed for the each short-term speech segments (frames). Finally, the Mel Frequency Cepstral Coefficients (MFCC) technique is performed in order to obtain feature vector of the speech. The feature matching step of both IRS and SRS is implemented with Dynamic Time Warping (DTW) which is an efficient algorithm to measure the distance between two vectors. According to the DTW results, the false acceptance rate (FAR) is zero and false rejecting rate (FRR) is about 4 % for the proposed hybrid system.

Machine learning and pattern recognition recently become a hot topic in computing world. This is due to the fast-growing of resources as well as techniques that make it easier to solve machine learning and pattern recognition problems. Problems that require machine learning solutions may be very simple for humans but actually can be very complex for machines to solve them. Face recognition is amongst those problems. Almost all human can easily recognize others without require specific knowledge to do it, different from machines which require its. This paper discussed face recognition task using machine learning strategies which involved Kernel Principal Component Analysis (KPCA) and Support Vector Machine (SVM) to identify person. KPCA extracted features from 2D image input and produced the important features of an image input. The extracted face features are recognized by SVM by classifying human face according to their stored identity in a database. SVM, which was basically a binary classifier worked by using one-against-one strategy to compare the face feature vector of a single test image to the stored face image in a face image database. Experiment results on grayscale images with size 92x112 pixels gave 96.25% of accuracy rate. Hence, KPCA and SVM for face recognition is a robust machine learning method.

Prediction subcellular localization of Gram-negative bacterial proteins by support vector machine using wavelet denoising and Chou's pseudo amino acid composition