A Parametric Kernel Function Yielding the Best Known Iteration Bound of Interior-Point Methods for Semidefinite Optimization

In this paper, we present a class of large- and small-update primal-dual interior-point methods forsemidefinite optimization based on a parametric kernel function with a trigonometric barrier term.For both versions of the kernel-based interior-point methods, the worst case iteration bounds are established, namely,$O(n^{\frac{2}{3}}\log\frac{n}{\varepsilon})$ and $O(\sqrt{n}\log\frac{n}{\varepsilon})$, respectively.These results match the ones obtained in the linear optimization case.

As indicated in the recent studies about primal-dual interior-point methods (IPMs) based on kernel functions, a kernel function not only serves to determine the search direction and measure the distance of the current iteration point to the $$\mu $$ -center, but also affects the iteration complexity and the practical computational efficiency of the algorithm. In this paper, we propose a new IPM for semidefinite optimization (SDO) based on a parameterized kernel function which is a generalization of the one presented by Bai et al. (Optim Methods Softw 17(6):985–1008, 2002). By using the good properties of the parameterized kernel function, we deduce that the iteration bound for large-update method is $$O(\sqrt{n}\log {n}\log {\frac{n}{\epsilon }})$$ for $$q=O(n)$$ , which is the best known complexity results for such methods. In our knowledge, this result is the first instance of primal-dual interior point method for SDO which involving the kernel function. Some numerical results have been provided.

Semidefinite optimization has an ever growing family of crucial applications, and large neighborhood interior point methods (IPMs) yield the method of choice to solve them. This chapter reviews the fundamental concepts and complexity results of Self-Regular (SR) IPMs for semidefinite optimizaion, that up to a log factor achieve the best polynomial complexity bound of small neighborhood IPMs. SR kernel functions are in the core of SR-IPMs. This chapter reviews several none SR kernel functions too. IPMs based on theses kernel functions enjoy similar iteration complexity bounds as SR-IPMs, though their complexity analysis requires additional tools.

Based on a new parametric kernel function, this paper presents a primal-dual large-update interior-point algorithm (IPM) for semi-definite optimization (SDO) problems. The new parametric function is neither self-regular function nor the usual logarithmic barrier function. It is strongly convex and possesses some novel analytic properties. We analyse this new parametric kernel function and show that the proposed algorithm has favorable complexity bound in terms of the analytic properties of the kernel function. Moreover, the complexity bound for our large-update IPM is shown to be O(\sqrt{n}(\log n)^2 \log\frac{n}{\epsilon}). Some numerical results are reported to illustrate the feasibility of the proposed algorithm.

According to the characteristics that the kernel function of extreme learning machine (ELM) and its performance have a strong correlation, a novel extreme learning machine based on a generalized triangle Hermitian kernel function was proposed in this paper. First, the generalized triangle Hermitian kernel function was constructed by using the product of triangular kernel and generalized Hermite Dirichlet kernel, and the proposed kernel function was proved as a valid kernel function of extreme learning machine. Then, the learning methodology of the extreme learning machine based on the proposed kernel function was presented. The biggest advantage of the proposed kernel is its kernel parameter values only chosen in the natural numbers, which thus can greatly shorten the computational time of parameter optimization and retain more of its sample data structure information. Experiments were performed on a number of binary classification, multiclassification, and regression datasets from the UCI benchmark repository. The experiment results demonstrated that the robustness and generalization performance of the proposed method are outperformed compared to other extreme learning machines with different kernels. Furthermore, the learning speed of proposed method is faster than support vector machine (SVM) methods.

In this paper a primal-dual interior-point algorithm for convex quadratic optimization based on a parametric kernel function, with parameters $p\in[0,1]$ and $q \geq 1$, is presented. The proposed parametric kernel function is of excellent properties which are used both for determining the search directions and measuring the distance between the given iterate and the central path for the algorithm. These properties enable us to derive the currently bestknown iteration bound for the algorithm with large-update method, namely, $O(\sqrt{n}\log {n }\log\frac{n}{\varepsilon})$, which reduces the gap between the practical behavior of the algorithm and its theoretical performance results.

Kernel functions serve the central goal of creating new search directions for the primal-dual interiorpoint algorithm to solve linear optimization problems. A significantly improved primal-dual interior-point algorithm for linear optimization is presented based on a novel kernel function. We show a primal-dual interior-point technique for linear optimization based on a class of kernel functions that are eligible. This research presents a new efficient kernel function-based primal-dual IPM algorithm for semidefinite programming problems based on the Nesterov-Todd (NT) direction. With a new and simple technique, we propose a new kernel function to obtain an optimal solution of the perturbed problem (SDP)µ. We obtain the best-known complexity results,for smalland large-update namely O(pp+1/2p √nlog tr(X0S0)/ε) and O((pn)p+1/2p log trX0S0/ε) large update To prove the effectiveness of our proposed kernel function, we compare our numerical results with some alternatives presented by Touil et al. (2017).

Recently the authors introduced the notions of self-regular functions and self-regular proximity functions and used them in the design and analysis of interior-point methods (IPMs) for linear and semidefinite optimization (LO and SDO). In this paper, we consider an extension of these concepts to second-order conic optimization (SOCO). This nontrivial extension requires the development of various new tools. Versatile properties of general analytical functions associated with the second-order cone are exploited. Based on the so-called self-regular proximity functions, new primal-dual Newton methods for solving SOCO problems are proposed. It will be shown that these new large-update IPMs for SOCO enjoy polynomial ${\cal{O}}({\max\{p,q\} N^{(q+1)/{2q}}\log \frac{N}{\e}})$ iteration bounds analogous to those of their LO and SDO cousins, where N is the number of constraining cones and p,q are constants, the so-called growth degree and barrier degree of the corresponding proximity. Our analysis allows us to choose not only a constant q but also a q as large as logN. In this case, our new algorithm has the best known ${\cal{O}}({N^{1/2}\log{N}\log\frac{N}{\e}})$ iteration bound for large-update IPMs.

Support Vector Machines (SVMs) versus Multilayer Perception (MLP) in data classification

General graph matching methods often suffer from the lack of mathematical structure in the space of graphs. Using kernel functions to evaluate structural graph similarity allows us to formulate the graph matching problem in an implicitly existing vector space and to apply well-known methods for pattern analysis. In this paper we propose a novel convolution graph kernel. Our kernel function differs from other graph kernels mainly in that it is closely related to error-tolerant graph edit distance and can therefore be applied to attributed graphs of various kinds. The proposed kernel function is evaluated on two graph datasets. It turns out that our method is generally more accurate than a standard edit distance based nearest-neighbor classifier, an edit distance based kernel variant, and a random walk graph kernel

This thesis presents a novel prior knowledge based Green's kernel for support vector regression (SVR) and provides an empirical investigation of SVM's (support vector machines) ability to model complex real world problems using a real dataset. After reviewing the theoretical background such as theory SVM, the correspondence between kernels functions used in SVM and regularization operators used in regularization networks as well as the use of Green's function of their corresponding regularization operators to construct kernel functions for SVM, a mathematical framework is presented to obtain the domain knowledge about the magnitude of the Fourier transform of the function to be predicted and design a prior knowledge based Green's kernel that exhibits optimal regularization properties by using the concept of matched filters. The matched filter behavior of the proposed kernel function provides the optimal regularization and also makes it suitable for signals corrupted with noise that includes many real world systems. Several experiments, mostly using benchmark datasets ranging from simple regression models to non-linear and high dimensional chaotic time series, have been conducted in order to compare the performance of the proposed technique with the results already published in the literature for other existing support vector kernels over a variety of settings including different noise levels, noise models, loss functions and SVM variations. The proposed kernel function improves the best known results by 18.6% and 24.4% on a benchmark dataset for two different experimental settings.

It is known that predictor-corrector methods in a large neighborhood of the central path are among the most efficient interior-point methods (IPMs) for linear optimization (LO) problems. The best iteration bound based on the classical logarithmic barrier function is O(nlog (n/∊)). In this paper, we propose a family of self-regular proximity-based predictor-corrector (SRPC) IPMs for LO in a large neighborhood of the central path. In the predictor step, we use either an affine scaling or a self-regular direction; in the corrector step, we use always a self-regular direction. Our new algorithms use a special proximity function with different search directions and thus allows us to improve the so far best theoretical iteration complexity for a family of SRPC IPMs. An O\((\sqrt{n}{\exp} ((1 - q + {\log} n)/2) {\log} n {\log} (n/\epsilon))\) worst-case iteration bound with quadratic convergence is established, where q is the barrier degree of the SR proximity function.

Exploring complexity of large update interior-point methods for [formula omitted] linear complementarity problem based on Kernel function

This paper proposes an alternative version of SVM using string vectors as its input data. A string vector is defined as a finite ordered set of words. Almost all of machine learning algorithms including traditional versions of SVM use numerical vectors as their input data. In order to apply such machine learning algorithms to classification or clustering, any type of raw data should be encoded into numerical vectors. In text categorization or text clustering, representing documents into numerical vectors leads to two main problems: huge dimensionality and sparse distribution. Although traditional versions of SVM are tolerable to huge dimensionality, they are not robust to sparse distribution in their training and classification. Therefore, in order to avoid this problem, this research proposes another version of SVM, which uses string vectors given as an alternative type of structured data to numerical vectors. For applying the proposed version of SVM to text categorization, documents are encoded into string vectors by defining conditions of words as their features and positioning words corresponding to the conditions into each string vector. A similarity matrix, word by word, which defines semantic similarities of words, should be built from a given training corpus, before using the proposed version of SVM for text categorization. Each semantic similarity given as an element of the matrix is computed based on collocations of two words within their same documents over a given corpus. In this paper, inner product of two string vectors as the proposed kernel function indicates an average semantic similarity over their elements. In order to validate the proposed version of SVM, it will be compared with a traditional version of SVM using linear kernel functions in text categorization on two test beds.

The purpose of this paper is to obtain new complexity results for a second-order cone optimization (SOCO) problem. We define a proximity function for the SOCO by a kernel function. Furthermore we formulate an algorithm for a large-update primal-dual interior-point method (IPM) for the SOCO by using the proximity function and give its complexity analysis, and then we show that the new worst-case iteration bound for the IPM is O(q√N(logN)q+1/q log N/e), where q?1.