Newton-Type Greedy Selection Methods for $\ell _0$ -Constrained Minimization.

Effective common spatial pattern (CSP) feature extraction for motor-imagery (MI) EEG recordings usually depends on the filter band selection to a large extent. However, the most proper filter band can hardly be determined manually due to its subject-specific property. This study introduces a sparse support vector machine (SSVM) approach to implement simultaneous CSP feature selection and classification for MI-based brain–computer interface (BCI). In SSVM, CSP features are first extracted on multiple signals that are filtered from raw EEG data at a set of overlapping subbands. SVM with l 1-norm regularization is then proposed to classify MI tasks with automatic selection of filter bands giving the significant CSP features. The effectiveness of SSVM for MI classification is demonstrated on the BCI Competition III dataset IVa, in comparison with several other competing methods. Experimental results indicate that the proposed SSVM method is promising for development of an improved MI-based BCI.

Capturing CO2 from the flue gas of a power plant based on chemical absorption method is an effective way to cut down the emission of CO2. However, the principal components in the solvent are very difficult to be measured online. Therefore an online measurement and control system was designed for the process of capturing CO2 by MEA chemical absorption. The feature selection was done according to Bayesian networks. Furthermore based on the principle of a new sparse support vector machine (SLS-SVM), the measurement of ionic species distribution in the absorption solution was predicted, and the results agreed well with the data obtained by Nuclear Magnetic Resonance (NMR) measurement. The mean square error is 3.8645×10-7. The measurement of solution species distribution was carried out online by using infrared sensors and NMR technology. Meanwhile the sampling system and NMR unit were linked for the online analysis based on variable-temperature and variable-pressure NMR technology. Finally a measurement and control platform was constructed based on VI technology. It shows that the performance of traditional equipment can be improved by the novel system, which will provide a more effective method for CO2 capture process.

In this paper we study the formulation of inverse problems as constrained minimization problems and their iterative solution by gradient or Newton type methods. We carry out a convergence analysis in the sense of regularization methods and discuss applicability to the problem of identifying the spatially varying diffusivity in an elliptic PDE from different sets of observations. Among these is a novel hybrid imaging technology known as impedance acoustic tomography, for which we provide numerical experiments.

Sparse pinball twin support vector machines

Twin support vector machine (TSVM) has been widely applied to classification problems. But TSVM is sensitive to outliers and is not efficient enough to realize feature selection. To overcome the shortcomings of TSVM, we propose a novel sparse twin support vector machine with the correntropy-induced loss (C-STSVM), which is inspired by the robustness of the correntropy-induced loss and the sparsity of the \(\ell _1\)-norm regularization. The objective function of C-STSVM includes the correntropy-induced loss that replaces the hinge loss, and the \(\ell _1\)-norm regularization that can make the decision model sparse to realize feature selection. Experiments on real-world datasets with label noise and noise features demonstrate the effectiveness of C-STSVM in classification accuracy and confirm the above conclusion further.

Melt index is one of the most important quality variables to determine product specifications in the polymer processing, which cannot be real-time measured by ordinary method. In order to estimate the melt index, a soft sensor method using sparse least square support vector machine based on genetic algorithm is proposed. In the proposed method, genetic algorithm is used to realize the sparseness of least square support vector machine and the optimization of its parameters. Simulation results of industrial data show that the proposed method can reduce the model scale which has better generalization capability than that of the non-sparse model.

New algorithms for the numerical solution of optimization problems involving the $$l_{0}$$ pseudo-norm are proposed. They are designed to use a recently proposed computational methodology that is able to deal numerically with finite, infinite and infinitesimal numbers. This new methodology introduces an infinite unit of measure expressed by the numeral $$\textcircled {1}$$ (grossone) and indicating the number of elements of the set $${\text {I}}\!{\text {N}}$$ , of natural numbers. We show how the numerical system built upon $$\textcircled {1}$$ and the proposed approximation of the $$l_0$$ pseudo-norm in terms of $$\textcircled {1}$$ can be successfully used in the solution of elastic net regularization problems and sparse support vector machines classification problems.

A unified way of analyzing some greedy algorithms

In this paper, we propose a robust sequential quadratic programming (SQP) method for nonlinear programming without using any explicit penalty function and filter. The method embeds the modified QP subproblem proposed by Burke and Han (Math Program 43:277---303, 1989) for the search direction, which overcomes the common difficulty in the traditional SQP methods, namely the inconsistency of the quadratic programming subproblems. A non-monotonic technique is employed further in a framework in which the trial point is accepted whenever there is a sufficient relaxed reduction of the objective function or the constraint violation function. A forcing sequence possibly tending to zero is introduced to control the constraint violation dynamically, which is able to prevent the constraint violation from over-relaxing and plays a crucial role in global convergence and the local fast convergence as well. We prove that the method converges globally without the Mangasarian---Fromovitz constraint qualification (MFCQ). In particular, we show that any feasible limit point that satisfies the relaxed constant positive linear dependence constraint qualification is also a Karush---Kuhn---Tucker point. Under the strict MFCQ and the second order sufficient condition, furthermore, we establish the superlinear convergence. Preliminary numerical results show the efficiency of our method.

The mutation status of KRAS is a significant biomarker in the prognosis of rectal cancer. This study investigated the feasibility of MRI-based radiomics in predicting the mutation status of KRAS with a composite index which could be an important criterion for KRAS mutation in clinical practice. In this retrospective study, a total of 127 patients with rectal cancer were enrolled. The 3D Slicer was used to extract the radiomics features from the MRI images, and sparse support vector machine (SVM) with linear kernel was applied for feature reduction. The radiomics classifier for predicting the KRAS status was then constructed by Linear Discriminant Analysis (LDA) and its performance was evaluated. The composite index was determined with LDA model. Out of 127 rectal cancer subjects, there were 44 KRAS mutation cases and 83 wild cases. A total of 104 radiomics features were extracted, 54 features were filtered by linear SVM with L1-norm regularization and 6 features that had no significant correlations within them were finally selected. The radiomics classifier constructed using the 6 features featured an AUC value of 0.669 (specificity: 0.506; sensitivity: 0.773) with LDA. Furthermore, the composite index (Radscore) had statistically significant difference between the KRAS mutation and wild groups. It is suggested that the MRI-based radiomics has the potential in predicting the KRAS status in patients with rectal cancer, which may enhance the diagnostic value of MRI in rectal cancer.

In this paper, we propose a new class of adaptive trust region methods for unconstrained optimization problems and develop some convergence properties. In the new algorithms, we use the current iterative information to define a suitable initial trust region radius at each iteration. The initial trust region radius is more reasonable in the sense that the trust region model and the objective function are more consistent at the current iterate. The global convergence, super-linear and quadratic convergence rate are analyzed under some mild conditions. Numerical results show that some special adaptive trust region methods are available and efficient in practical computation.

One of the downsides of the massive multiple-input-multiple-output (M-MIMO) system is its computational complexity. Considering that techniques and different algorithms proposed in the literature applied to conventional MIMO may not be well suited or readily applicable to M-MIMO systems, in this paper, the application of different formulations inside the convex optimization framework is investigated. This paper is divided into two parts. In the first part, linear programming, quadratic programming (QP), and semidefinite programming are explored in an M-MIMO environment with high-order modulation and under realistic channel conditions, i.e., considering spatial correlation, error in the channel estimation, as well as different system loading. The bit error rate is evaluated numerically through Monte Carlo simulations. In the second part, algorithms to solve the QP formulation are explored, and computational complexity in terms of floating-point operations (flops) is compared with linear detectors. Those algorithms have interesting aspects when applied to our specific problem (M-MIMO detection formulated as QP), such as the exploitation of the structure of the problem ( simple constraints ) and the improvement of the rate of convergence due to the well-conditioned Gram matrix (channel hardening). The number of iterations is higher when the number of users $K$ becomes similar to the number of base station antennas $M$ (i.e., $K\approx M$ ) than the case $K\ll M$ ; the number of iterations increases slowly as the number of users $K$ and base station antennas $M$ increases while keeping a low system loading. The QP with projected algorithms presented better performance than minimum mean square error detector when $K\approx M$ and promising computational complexity for scenarios with increasing $K$ and low system loading.

Mobile cloud computing is an emerging field of research which aims to provide a platform on which intelligent and feature-rich applications are delivered to the user at any time and at anywhere. When such a cloud-assisted mobile application workflow requires the cooperation of many devices, solving the task allocation problem becomes a critical step in ensuring the energy efficiency of the mobile cloud platform. In this paper, we construct a quadratic binary program to model the task allocation problem in such scenarios. In order to overcome the poor scalability of generic quadratic program solvers, we present an implementation of the simulated annealing algorithm and a greedy autonomous offload algorithm to approximate the optimal solution. Both heuristics are tailored to solve our task allocation problem efficiently. We verify and compare our algorithms against a commercial quadratic program solver in a series of simulations. Results show that both heuristics produce good solutions to the task allocation problem. Solutions provided by our greedy algorithms is consistently close to optimal and can be obtained in a more time efficient manor than our implementation of the simulated annealing algorithm.

The sparsity driven classification technologies have attracted much attention in recent years, due to their capability of providing more compressive representations and clear interpretation. Two most popular classification approaches are support vector machines (SVMs) and kernel logistic regression (KLR), each having its own advantages. The sparsification of SVM has been well studied, and many sparse versions of 2-norm SVM, such as 1-norm SVM (1-SVM), have been developed. But, the sparsification of KLR has been less studied. The existing sparsification of KLR is mainly based on L1 norm and L2 norm penalties, which leads to the sparse versions that yield solutions not so sparse as it should be. A very recent study on L1/2 regularization theory in compressive sensing shows that L1/2 sparse modeling can yield solutions more sparse than those of 1 norm and 2 norm, and, furthermore, the model can be efficiently solved by a simple iterative thresholding procedure. The objective function dealt with in L1/2 regularization theory is, however, of square form, the gradient of which is linear in its variables (such an objective function is the so-called linear gradient function). In this paper, through extending the linear gradient function of L1/2 regularization framework to the logistic function, we propose a novel sparse version of KLR, the 1/2 quasi-norm kernel logistic regression (1/2-KLR). The version integrates advantages of KLR and L1/2 regularization, and defines an efficient implementation scheme of sparse KLR. We suggest a fast iterative thresholding algorithm for 1/2-KLR and prove its convergence. We provide a series of simulations to demonstrate that 1/2-KLR can often obtain more sparse solutions than the existing sparsity driven versions of KLR, at the same or better accuracy level. The conclusion is also true even in comparison with sparse SVMs (1-SVM and 2-SVM). We show an exclusive advantage of 1/2-KLR that the regularization parameter in the algorithm can be adaptively set whenever the sparsity (correspondingly, the number of support vectors) is given, which suggests a methodology of comparing sparsity promotion capability of different sparsity driven classifiers. As an illustration of benefits of 1/2-KLR, we give two applications of 1/2-KLR in semi-supervised learning, showing that 1/2-KLR can be successfully applied to the classification tasks in which only a few data are labeled.

We consider a Newton-type method for the solution of semidefinite programs. This Newton-type method is based on a semismooth reformulation of the semidefinite program as a nonsmooth system of equations. We establish local quadratic convergence of this method under a linear independence assumption and a weak nondegeneracy condition that implies uniqueness of the optimal solution but does not imply strict complementarity.