FS/DS: A Theoretical Framework for the Dual Analysis of Feature Space and Data Space.

Intuitionistic Fuzzy C-means (IFCM) is a robust clustering method which is based upon intuitionistic fuzzy set theory. It uses Euclidean distance as a distance metric, hence can only cluster hyper spherically distributed data-sets in data space or in feature space. FCM and KFCM with a new distance measure (FCM-σ and KFCM-σ) can detect non-hyperspherical clusters in data space and feature space but they are sensitive to noise and produce inefficient results in the presence of noise. This paper present a robust Intuitionistic Fuzzy c-means(IFCM-σ) and a robust kernel Intutitionistic Fuzzy C-Means(KIFCM-σ) with a new distance metric that incorporates the distance variation in a cluster to regularize the distance between data point and the cluster centroid. Propose algorithms are the hybridization of IFCM, kernel function, and new distance metric in the data space and in the feature space which avoid various problems of IFCM and FCM-σ. Experiments are done using two-dimensional synthetic data-sets and noisy digital images, and results are compared with IFCM, KIFCM, FCM-σ and KFCM-σ. The results show that our proposed algorithms, especially KIFCM-σ are more effective.

Bayesian principal geodesic analysis for estimating intrinsic diffeomorphic image variability.

Stability analysis and control design of LTI discrete-time systems by the direct use of time series data

While some image textures can be changed with scale, others cannot. We focus on a multi-scale features of determing the sensitivity of the texture intensity to change. This paper presents a new method of texture structure classification and depth estimation using multi-scale features extracted from a higher order of the local autocorrelation functions. Multi-scale features consist of the meansand variances of distributions, which are extracted from theautocorrelation feature vectors according to multi-level scale. In order to reduce dimensional feature vectors, we employ the Principal Component Analysis (PCA) in the autocorrelation feature space. Each training image texture makes its own multi-scale model in a reduced PCA feature space, and the test of the texture image is projected in the homogeneous PCA space of the training data. The experimental results show that the proposed multi-scale feature can be utilized notonly for texture classification, but also depth estimation in two dimensional images with texture gradients.

A novel feature extraction method that utilizes nonlinear mapping from the original data space to the feature space is presented in this dissertation. Feature extraction methods aim to find compact representations of data that are easy to classify. Measurements with similar values are grouped to same category, while those with differing values are deemed to be of separate categories. For most practical systems, the meaningful features of a pattern class lie in a low dimensional nonlinear constraint region (manifold) within the high dimensional data space. A learning algorithm to model this nonlinear region and to project patterns to this feature space is developed. Least squares estimation approach that utilizes interdependency between points in training patterns is used to form the nonlinear region. The proposed feature extraction strategy is employed to improve face recognition accuracy under varying illumination conditions and facial expressions. Though the face features show variations under these conditions, the features of one individual tend to cluster together and can be considered as a neighborhood. Low dimensional representations of face patterns in the feature space may lie in a nonlinear constraint region, which when modeled leads to efficient pattern classification. A feature space encompassing multiple pattern classes can be trained by modeling a separate constraint region for each pattern class and obtaining a mean constraint region by averaging all the individual regions. Unlike most other nonlinear techniques, the proposed method provides an easy intuitive way to place new points onto a nonlinear region in the feature space. The proposed feature extraction and classification method results in improved accuracy when compared to the classical linear representations. Face recognition accuracy is further improved by introducing the concepts of modularity, discriminant analysis and phase congruency into the proposed method. In the modular approach, feature components are extracted from different sub-modules of the images and concatenated to make a single vector to represent a face region. By doing this we are able to extract features that are more representative of the local features of the face. When projected onto an arbitrary line, samples from well formed clusters could produce a confused mixture of samples from all the classes leading to poor recognition. Discriminant analysis aims to find an optimal line orientation for which the data classes are well separated. Experiments performed on various databases to evaluate the performance of the proposed face recognition technique have shown improvement in recognition accuracy, especially under varying illumination conditions and facial expressions. This shows that the integration of multiple subspaces, each representing a part of a higher order nonlinear function, could represent a pattern with variability. Research work is progressing to investigate the effectiveness of subspace projection methodology for building manifolds with other nonlinear functions and to identify the optimum nonlinear function from an object classification perspective.

A new generic nonlinear feature extraction map f is presented based on concepts from approximation theory. Let f map an input data vector x/spl isin//spl Rfr//sup n/, where n is high, to an appropriate feature vector y/spl isin//spl Rfr//sup m/, where m is sufficiently low. Also let X={X/sub 1/,...,X/sub N/} denote an available training set in /spl Rfr//sup n/. In this paper f is derived by requiring that the geometric structure (metric space attributes) of the points f(X)={f(X/sub 1/),...,f(X/sub N/)} in the feature space /spl Rfr//sup m/ be as similar as possible to the structure of the points X in the data space /spl Rfr//sup n/. This is accomplished by selecting first an appropriate dimension m for the feature space /spl Rfr//sup m/ according to the size N of the available training set X, subject to bounds on the distortion of the data structure caused by f and on the error in the estimation of the underlying likelihood functions in the feature space. The map f(i) is designed by a multi-dimensional scaling (MDS) approach that minimizes the Sammon's cost function. This approach uses graph-theoretic (minimal spanning tree) and genetic algorithmic concepts to search efficiently the optimal structure-preserving point-to-point mapping of the training samples X/sub 1/,...,X/sub N/ to their images in /spl Rfr//sup m/,Y/sub 1/,...,Y/sub N/. Finally, an optimal interpolating (OI) artificial neural network is used to recover the entire function f:/spl Rfr//spl rarr//spl Rfr//sup m/ by interpolating the values y/sub i/=1,...,N at X/sub i/, i=1,...,N. Preliminary simulation results based on this approach are also given.

There is a trend in recent machine learning community to construct a nonlinear version of a linear algorithm using the 'kernel method', e.g. Support Vector Machines (SVMs), kernel principal component analysis, kernel fisher discriminant analysis and the recent kernel clustering algorithms. In unsupervised clustering algorithms using kernel method, typically, a nonlinear mapping is used first to map the data into a potentially much higher feature space, where clustering is then performed. A drawback of these kernel clustering algorithms is thatthe clustering prototypes lie in high dimensional feature space and hence lack clear and intuitive descriptions unless using additional projection approximation from the feature to the data space as done in the existing literatures. In this paper, a novel clustering algorithm using the 'kernel method' based on the classical fuzzy clustering algorithm (FCM) is proposed and called as kernel fuzzy c-means algorithm (KFCM). KFCM adopts a new kernel-induced metric in the data space to replace the original Euclidean norm metric in FCM and the clustered prototypes still lie in the data space so that the clustering results can be reformulated and interpreted in the original space. Our analysis shows that KFCM is robust to noise and outliers and also tolerates unequal sized clusters. And finally this property is utilized to cluster incomplete data. Experiments on two artificial and one real datasets show that KFCM hasbetter clustering performance and more robust than several modifications of FCM for incomplete data clustering.

Feature selection based on label distribution and fuzzy mutual information

Scholars have developed a number of theories to explain economic regionalism in the postwar years. While the existing theories provide some intellectual insights on economic regionalism in one respect or the other, they could hardly explain a unique process and pattern of regional economic integration in East Asia that has developed since the mid-1980s. The analytical framework that is developed in this chapter is intended to explore economic regionalism in the world economy in general and regional economic integration in East Asia in particular. While the overall structure of this analytical framework is seminal, the approach is built on the basis of many insightful thoughts of contemporary international political economy theories with respect to economic regionalism, incorporating global (structural), regional and national factors into a unified analytical framework.1 As such, a review of major theoretic perspectives on economic regionalism is useful.

Social networking sites such as Facebook or Twitter attract millions of users, who everyday post an enormous amount of content in the form of tweets, comments and posts. Since social network texts are usually short, learning tasks have to deal with a very high dimensional and sparse feature space, in which most features have low frequencies. As a result, extracting useful knowledge from such noisy data is a challenging task, that converts large-scale short-text learning tasks in social environments into one of the most relevant problems in machine learning and data mining. Feature selection is one of the most known and commonly used techniques for reducing the impact of the high dimensional feature space in text learning. A wide variety of feature selection techniques can be found in the literature applied to traditional, long-texts and document collections. However, short-texts coming from the social Web pose new challenges to this well-studied problem as texts’ shortness offers a limited context to extract enough statistical evidence about words relations (e.g. correlation), and instances usually arrive in continuous streams (e.g. Twitter timeline), so that the number of features and instances is unknown, among other problems. This paper surveys feature selection techniques for dealing with short texts in both offline and online settings. Then, open issues and research opportunities for performing online feature selection over social media data are discussed.

Feature selection is one of the key processes in pattern classification. For the task of feature selection, an important problem is the evaluation of the classification capability or clarity of different feature spaces, which is related to their uncertainty. In the framework of Dempster-Shafer evidence theory (DST), the Basic Belief Assignment (BBA) can well describe the uncertainty. In this paper, Generative Adversarial Network (GAN) is used to generate BBAs in each feature space, and a novel feature selection approach based on ambiguity distribution is proposed, which uses the skewness and variance of the BBAs’ ambiguity values on each feature space. Experimental results of pattern classification on various UCI datasets show that our new proposed method is rational and can effectively improve the accuracy of pattern classification.

Nowadays, pre-training big models on large-scale datasets has achieved great success and dominated many downstream tasks in natural language processing and 2D vision, while pre-training in 3D vision is still under development. In this paper, we provide a new perspective of transferring the pre-trained knowledge from 2D domain to 3D domain with Point-to-Pixel Prompting in data space and Pixel-to-Point distillation in feature space, exploiting shared knowledge in images and point clouds that display the same visual world. Following the principle of prompting engineering, Point-to-Pixel Prompting transforms point clouds into colorful images with geometry-preserved projection and geometry-aware coloring. Then the pre-trained image models can be directly implemented for point cloud tasks without structural changes or weight modifications. With projection correspondence in feature space, Pixel-to-Point distillation further regards pre-trained image models as the teacher model and distills pre-trained 2D knowledge to student point cloud models, remarkably enhancing inference efficiency and model capacity for point cloud analysis. We conduct extensive experiments in both object classification and scene segmentation under various settings to demonstrate the superiority of our method. In object classification, we reveal the important scale-up trend of Point-to-Pixel Prompting and attain 90.3% accuracy on ScanObjectNN dataset, surpassing previous literature by a large margin. In scene-level semantic segmentation, our method outperforms traditional 3D analysis approaches and shows competitive capacity in dense prediction tasks.

BackgroundRecently, successful applications of the discrete wavelet transform have been reported in brain interface (BI) systems with one or two EEG channels. For a multi-channel BI system, however, the high dimensionality of the generated wavelet features space poses a challenging problem.MethodsIn this paper, a feature selection method that effectively reduces the dimensionality of the feature space of a multi-channel, self-paced BI system is proposed. The proposed method uses a two-stage feature selection scheme to select the most suitable movement-related potential features from the feature space. The first stage employs mutual information to filter out the least discriminant features, resulting in a reduced feature space. Then a genetic algorithm is applied to the reduced feature space to further reduce its dimensionality and select the best set of features.ResultsAn offline analysis of the EEG signals (18 bipolar EEG channels) of four able-bodied subjects showed that the proposed method acquires low false positive rates at a reasonably high true positive rate. The results also show that features selected from different channels varied considerably from one subject to another.ConclusionThe proposed hybrid method effectively reduces the high dimensionality of the feature space. The variability in features among subjects indicates that a user-customized BI system needs to be developed for individual users.

The problem of volume averaging in quantitating CSF, gray-matter, and white-matter fractions in the brain is solved using a three-compartment model and a simple graphical analysis of a multispectral MR feature space. Compartmentalization is achieved without the ambiguities of thresholding techniques or the need to assume that the underlying pixel probability distributions have a particular form. A 2D feature space is formed by double SE (proton density- and T2-weighted) MR data with image nonuniformity removed by a novel technique in which the brain itself serves as a uniformity reference. Compartments other than the basic three were rejected by the tailoring of limits in feature space. Phantom scans substantiate this approach, and the importance of the careful selection and standardization of pure tissue reference signals is demonstrated. Compartmental profiles from standardized subvolumes of three normal brains, based on a 3D (Talairach) coordinate system, demonstrate slice-by-slice detail; longitudinal studies confirm reproducibility. Compartmentalization may be described graphically and algebraically, complementing data displays in feature space and images of compartmentalized brain scans. These studies anticipate the application of our compartmentalization technique to patients with neurological disorders.

One of the important practical applications of computer vision techniques is computer-aided diagnosis (CAD) in medical imaging. It has been shown that CAD can improve the accuracy of breast cancer detection and characterization by radiologists on mammograms. In this chapter, we discuss an important step — feature selection — in classifier design for CAD algorithms. Feature selection reduces the dimensionality of an available feature space and is therefore often used to prevent over-parameterization of a classifier. Many feature selection techniques have been proposed in the literature. We will illustrate the usefulness of genetic algorithms (GAs) for feature selection by comparing GA with a commonly used sequential selection method. A brief introduction to the GA is given and several examples using GA feature selection for the characterization of mammographic lesions are discussed. The examples illustrate the design of a fitness function for optimizing classification accuracy in terms of the receiver operating characteristics of the classifier, the dependence of GA performance on its evolution parameters, and the design of a fitness function tailored to a specific classification task.