Joint L1/2-Norm Constraint and Graph-Laplacian PCA Method for Feature Extraction

The standard traffic assignment problem (TAP) is often augmented with additional constraints to address non-standard applications. These models are called TAP with side constraints (TAPSC). Despite the rising significance of TAPSC models, the ability to efficiently solve them to satisfactory precision remains limited in real-world applications. The purpose of this paper is to fill this gap by integrating a recently developed high performance TAP solver, known as the path-based Greedy algorithm, with the augmented Lagrangian multiplier (ALM) method. This paper examines how precisely the subproblems in the ALM method should be solved to optimize the overall convergence performance. It is found that insufficiently converged subproblem solutions sometimes lead to catastrophic failures, although pursuing extremely high precision could also be counterproductive. Accordingly, it is proposed to adjust the precision required to solve the subproblems based on an approximate gap measured by the augmented Lagrangian. Results of numerical experiments show that adaptively adjusting the subproblem precision limit produces a 25% speed-up compared with the algorithm with a fixed limit.

Imaging mass spectrometry is an innovative technique that combines high-resolution microscopic imaging tools with analytical capabilities of spectrometry. It is a powerful tool to determine the spatial distribution of chemical compounds on complex surfaces, for example, for microscale analysis of cells and tissue in biological samples. The result is a large spectral datacube: a three-dimensional (3D) dataset in which surface position and mass spectral distribution are represented. Analysts try to discover ‘features’: correlations in spectral profiles with a recognizable spatial distribution. Techniques for feature extraction and visualization are developed to improve the exploratory analysis of spectral datacubes. The topic of this work is the design and implementation of feature extraction and visualization techniques in high-resolution imaging spectrometry data. Principal Component Analysis (PCA) is interactively used as a governing approach for feature detection. A wide range of visualization techniques are implemented based on extracted features. The thesis is organized as follows. In Chapter 2 (Spectral analysis: a survey), we provide a brief background survey on spectral analysis. The analysis in the proposed approach is divided into three stages: data acquisition, feature extraction and feature visualization. For each stage, a detailed description of currently applied methods is given. The methods most appropriate for this qualitative approach of analysis are chosen as a specific subset. PCA in combination with a binning function is most suited for extracting features from imaging mass spectral data. Both methods increase the signal-to-noise ratio and reduce the amount of data from imaging mass spectrometry. Chapter 3 (PCA-based feature extraction) compares the quality of three different PCA-based methods for detecting and extracting features from spectral datacubes. We discuss preprocessing of mass spectral data, PCA, additional rotational optimization by VARIMAX, and the PARAFAC method for factor regression. The results are compared quantitatively and qualitatively, together with some performance characteristics. For the quantitative comparison, we used a RMSE metric to compare the methods with ground truth spectra under various noise conditions. For the qualitative comparison, we used three criteria to judge the quality of features in the resulting visualizations. These criteria were applied to interpret the visualizations of features. In Chapter 4 (Feature-based registration), a robust method for automatic featurebased registration is developed. The reduction of uncorrelated noise provided by PCA allows high-resolution imaging mass spectrometry datasets to be automatically aligned and combined for high-resolution analysis of large areas. The results clearly show that the entropy-weighted, mean squared error landscape of chemically matched component images can be used to automatically align high-resolution imaging mass spectrometry datasets. Several spectral datacubes are combined to provide better detection and extraction of features. In Chapter 5 (Feature visualization), a visualization technique is described that utilizes principal components to create transfer functions for volume rendering of a spectral datacube. Two types of spectral datacubes are visualized in 3D by direct volume rendering with these transfer functions to control opacity and highlight extracted features. This enables us to visualize the link between the spectral and spatial characteristics of a feature within the spectral datacube. Applications demonstrate the additional value of these visualizations. Chapter 6 (Feature zooming) presents a technique for spectral and/or spatial zooming of extracted features. This technique is especially useful for spatially extended datasets. The combined spectral datasets are too large in size to be explored and visualized using commonly feature extraction and visualization techniques. Analysts are able to select important features or deselect unimportant features for further analysis on different levels of detail. Moreover, features with unwanted artifacts can be removed to reduce noise. Chapter 7 (High-resolution feature visualization) provides an approach to parametrically visualize features in 3D and at the highest resolution possible. Three parameters control the spectral contribution, the level of detail and the level of density on which an extracted feature is represented. This visualization has feature shapes with well-defined borders and provides more insight into the influences of noise on a mass spectral measurement. It is possible to distinguish different peaks according to their difference in density and spatial position, which would not be possible in a separate spectral or spatial view. An application shows how resulting features are visualized and interpreted. The developed tools generate new possibilities to handle, explore, and visualize the large imaging mass spectrometry datasets. A sensitive, selective, and robust approach for feature extraction enables detection and classification of features in different proteomics applications. Multiple feature shapes with high-resolution characteristics can be compared and examined on different levels of detail. These visualizations can provide more detailed molecular insight in the biochemistry of surfaces and improve classification of peptides and proteins.

In this paper, we propose a time-frequency analysis method to obtain instantaneous frequencies and the corresponding decomposition by solving an optimization problem. In this optimization problem, the basis that is used to decompose the signal is not known a priori. Instead, it is adapted to the signal and is determined as part of the optimization problem. In this sense, this optimization problem can be seen as a dictionary adaptation problem, in which the dictionary is adaptive to one signal rather than a training set in dictionary learning. This dictionary adaptation problem is solved by using the augmented Lagrangian multiplier (ALM) method iteratively. We further accelerate the ALM method in each iteration by using the fast wavelet transform. We apply our method to decompose several signals, including signals with poor scale separation, signals with outliers and polluted by noise and a real signal. The results show that this method can give accurate recovery of both the instantaneous frequencies and the intrinsic mode functions.

Robust principal component analysis via ES-algorithm

Nonlinear principal component analysis (NLPCA) is known as a nonlinear generalization of the standard principal component analysis (PCA). Since nonlinear PCA is a non-unique concept, it is discussed, how nonlinear PCA can be defined as a nonlinear feature extraction technique most similar to linear PCA. The nonlinear reduction of a data set from its original dimension to the intrinsic dimension of the data is one aspect, but we also request that the nonlinear features spanning this intrinsic data space are hierarchically arranged similar to the linear features of PCA. Thus, such nonlinear PCA is a powerful pre-processing step. It can be used as nonlinear sphering (whitening) or it can be considered as a smoothing method which removes nonlinear correlations between variables. Nonlinear PCA can be performed by minimizing a hierarchical error function. This error function is applied to the autoencoder which is a multi-layer perceptron used in auto-associative mode to perform the identity mapping.

Change point detection (CPD) for multi-agent systems helps one to evaluate the state and better control the system. Multivariate CPD methods solve the d × T time series well; however, the multi-agent systems often produce the N × d × T dimensional data, where d is the dimension of multivariate observations, T is the total observation time, and N is the number of agents. In this paper, we propose two valid approaches based on higher-order features, namely, the Betti number feature extraction and the Persistence feature extraction, to compress the d-dimensional features into one dimension so that general CPD methods can be applied to higher-dimensional data. First, a topological structure based on the Vietoris-Rips complex is constructed on each time-slice snapshot. Then, the Betti number and persistence of the topological structures are obtained to separately constitute two feature matrices for change point estimates. Higher-order features primarily describe the data distribution on each snapshot and are, therefore, independent of the node correspondence cross snapshots, which gives our methods unique advantages in processing missing data. Experiments in multi-agent systems demonstrate the significant performance of our methods. We believe that our methods not only provide a new tool for dimensionality reduction and missing data in multi-agent systems but also have the potential to be applied to a wider range of fields, such as complex networks.

Speckle noise removal via nonlocal low-rank regularization

Spatial spectral total variation has been widely used in the hyperspectral image (HSI) denoising algorithms. However, the total variation norm only penalizes the gradient magnitudes of HSIs, which may influence the recovery of denoised image edges. In this paper, we proposed a new l <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> gradient regularized low-rank tensor model for removing HSI mixed noises. The low-rank tensor Tucker decomposition is applied for recovering the clean HSI part by using the HSI global correlation. The l0 gradient regularization controls the number of non-zero gradients directly and is designed to explore the spatial-spectral property of hyperspectral images and preserve important image features. The optimization problem is solved by the Augmented Lagrange Multiplier (ALM) method efficiently. The real-world data experiment demonstrates the better performance of our method.

This paper presents an effective design approach for reducing the weight of a flexible rotor in ball bearings with rotational speed and load dependent stiffness characteristics under constraints on the system eigenvalues and bearing fatigue life. Design variables are chosen to be the inner radii of shaft elements, the positions of ball bearings, and the preloads on the bearings. The stiffness characteristics of high speed ball bearings are completely described as functions of applied loads and spin speed, and applied to the dynamic behavior analysis of a rotor-bearing system. A transfer matrix method is used to obtain eigenvalues of the system and an augmented Lagrange multiplier (ALM) method is employed as an optimization technique. A multi-stepped rotor supported by two angular contact ball bearings is analyzed and designed to show the speed and load dependent stiffness effect on the system dynamic behavior and to demonstrate the effectiveness of the proposed optimum design approach. The results show that the effect of the stiffness on the system dynamic behavior is noticeable and that the suggested design approach is effective. [S0739-3717(00)00803-5]

Background Subtraction (BS) is one of the key steps in video analysis. Many background models have been proposed and achieved promising performance on public data sets. However, due to challenges such as illumination change, dynamic background etc. the resulted foreground segmentation often consists of holes as well as background noise. In this regard, we consider generalized fused lasso regularization to quest for intact structured foregrounds. Together with certain assumptions about the background, such as the low-rank assumption or the sparse-composition assumption (depending on whether pure background frames are provided), we formulate BS as a matrix decomposition problem using regularization terms for both the foreground and background matrices. Moreover, under the proposed formulation, the two generally distinctive background assumptions can be solved in a unified manner. The optimization was carried out via applying the augmented Lagrange multiplier (ALM) method in such a way that a fast parametric-flow algorithm is used for updating the foreground matrix. Experimental results on several popular BS data sets demonstrate the advantage of the proposed model compared to state-of-the-arts.

Tensorized multi-dimensional multi-view clustering based on nonnegative matrix factorization.

Partial least squares discriminant analysis (PLS-DA) is gaining popularity as a discriminant feature extraction tool. It is often viewed as a “supervised” version of the principle component analysis (PCA) where dimensionality reduction is achieved with full awareness of the class labels. It is well known that as a dimensionality reduction tool, PCA only captures the global geometric structure of a data set and this characteristics of PCA is inherited in PLS-DA. In this paper we propose a locality preserving PLS-DA (LPPLS-DA) in an effort to enhance the discriminant capabilities of PLS-DA. Our proposed LPPLS-DA incorporates neighborhood information of the data by means of a similarity matrix. We investigate the use of LPPLS-DA in spoken letter recognition. Finding relevant features for the classification of spoken letters is affected by high dimensionality of the data and the sound (phonetic) similarities between the letters. To address this problem, we used LPPLS-DA to extract relevant features of the data and discrimination is automatically achieved using the LPPLS-DA method. The effectiveness of LPPLS-DA is demonstrated by experimental results on the ISOLET (Isolated Letter Speech Recognition) data set. The classification accuracies in the experiments are measured using the nearest neighbor classifier. The LPPLS-DA method is shown to give higher accuracies than PLS-DA as well as several other feature extraction methods.

In many medical applications, feature selection is obvious; but in medical domains, selecting features and creating a feature vector may require more effort. The wavelet transform (WT) technique is used to identify the characteristic points of an electrocardiogram (ECG) signal with fairly good accuracy, even in the presence of severe high-frequency and low-frequency noise. Principal component analysis (PCA) is a suitable technique for ECG data analysis, feature extraction, and image processing — an important technique that is not based upon a probability model. The aim of the paper is to derive better diagnostic parameters for reducing the size of ECG data while preserving morphology, which can be done by PCA. In this analysis, PCA is used for decorrelation of ECG signals, noise, and artifacts from various raw ECG data sets. The aim of this paper is twofold: first, to describe an elegant algorithm that uses WT alone to identify the characteristic points of an ECG signal; and second, to use a composite WT-based PCA method for redundant data reduction and better feature extraction. PCA scatter plots can be observed as a good basis for feature selection to account for cardiac abnormalities. The study is analyzed with higher-order statistics, in contrast to the conventional methods that use only geometric characteristics of feature waves and lower-order statistics. A new algorithm — viz. PCA variance estimator — is developed for this analysis, and the results are also obtained for different combinations of leads to find correlations for feature classification and useful diagnostic information. PCA scatter plots of various chest and augmented ECG leads are obtained to examine the varying orientations of the ECG data in different quadrants, indicating the cardiac events and abnormalities. The efficacy of the PCA algorithm is tested on different leads of 12-channel ECG data; file no. 01 of the Common Standards for Electrocardiography (CSE) database is used for this study. Better feature extraction is obtained for some specific combinations of leads, and significant improvement in signal quality is achieved by identifying the noise and artifact components. The quadrant analysis discussed in this paper highlights the filtering requirements for further ECG processing after performing PCA, as a primary step for decorrelation and dimensionality reduction. The values of the parameters obtained from the results of PCA are also compared with those of wavelet methods.

Principal components analysis (PCA) is commonly used as a tool for feature space dimensionality reduction for various automatic target recognition (ATR) systems. Recently, PCA has also been employed in conjunction with linear discriminant analysis (LDA) to recondition ill posed LDA formulations. The key idea behind this approach is to use a PCA transformation to discard the null space of rank deficient scatter matrices so that LDA can be applied on this reconditioned space. This approach, called subspace LDA, has been employed with some level of success for face recognition algorithms. In this paper, we present a theoretical analysis of the effects of PCA on discrimination power of the projected subspace. We also provide experimental evidence of the ineffectiveness of a PCA projection for hyperspectral ATR applications using three different hyperspectral invasive species datasets.

Summary 4D seismic history matching (4D SHM) uses 4D seismic data to calibrate reservoir models to reduce production forecast uncertainty and improve reservoir surveillance. 4D seismic becomes very valuable in field developments with sparse well configurations and high areal uncertainty, such as offshore and carbon sequestration projects. Unlike the production data, the conventional uncertainty and sensitivity analysis (SA) with 4D seismic data might return misleading results. Due to the smooth nature of 4D seismic data, it is highly likely that low-frequency signals dampen the impact of model parameters on high-frequency signals. Consequently, some significant parameters are wrongly excluded from the 4D SHM process. Our work aims to address this issue by localizing the SA of 4D seismic data. The idea is first to identify specific seismic signals on the seismic maps and then perform the SA only at the individual locations rather than the entire map. This way we overcome the dampening effects of low-frequency signals. Several approaches to localize the SA are utilized. In one approach, we defined sliding windows to scan the seismic maps and then executed an SA inside the windows at each location. Other localization approaches employ dimensionality reduction and feature extraction tools. We used principal component analysis (PCA) and advanced machine learning methods such as autoencoders (AEs) and variational autoencoders (VAEs) to transform the 4D seismic maps into a latent space. The information content (the 4D seismic signals) in the high-dimensionality 4D seismic maps is represented by a few features in the latent space. Implementing an SA for each feature in the latent space is equivalent to performing SA with the seismic signals in the original map. The localized SA scheme is coupled with the ensemble smoother with multiple data assimilation (ESMDA) algorithm to carry out 4D SHM. Three 4D SHM scenarios were defined as full parameterization with no SA, conventional SA using the entire map, and localized SA. We ran these scenarios for a complex synthetic reservoir model based on a real field in the North Sea to match 4D P-wave seismic impedance. The results confirmed the superiority of the localized SA scenario which returned the final ensemble with the lowest error and the best match among the three scenarios. It also turned out that the PCA, for this specific case, is the most suitable methodology to localize the SA.