Sparse Covariance Matrix Estimation by DCA-Based Algorithms.

DC approximation approaches for sparse optimization

It is undoubtedly that mathematical modelling and optimisation play a key role in the supply chain and the production management (SCPM). In this paper, we provide a survey on DC (Difference of Convex function) programming and DCA (DC Algorithm), a state-of-the-art optimisation approach for challenging problems in SCPM. DC programming and DCA constitute the backbone of non-convex programming and global optimisation. Whilst DC programming and DCA were widely and successfully investigated in many areas, it seems that they were not so much popular in the community of SCPM. There is therefore a need to further develop this efficient and scalable approach for SCPM applications, especially for large-scale problems in the context of Big data. For such purpose, this paper aims to present benchmark models and state-of-the-art DCA-based methods for solving challenging problems in SCPM systems. We prove that all the benchmark classes of optimisation models appeared in SCPM systems can be formulated/reformulated as a DC program and show how to solve these classes of problems by DCA-based algorithms. We offer the community of researchers in SCPM efficient algorithms in a unified DC programming framework to tackle various applications such as supply chain design, scheduling, multi-stage production/inventory system, vehicle routing, …

In this letter, we consider the nonnegative matrix factorization (NMF) problem and several NMF variants. Two approaches based on DC (difference of convex functions) programming and DCA (DC algorithm) are developed. The first approach follows the alternating framework that requires solving, at each iteration, two nonnegativity-constrained least squares subproblems for which DCA-based schemes are investigated. The convergence property of the proposed algorithm is carefully studied. We show that with suitable DC decompositions, our algorithm generates most of the standard methods for the NMF problem. The second approach directly applies DCA on the whole NMF problem. Two algorithms-one computing all variables and one deploying a variable selection strategy-are proposed. The proposed methods are then adapted to solve various NMF variants, including the nonnegative factorization, the smooth regularization NMF, the sparse regularization NMF, the multilayer NMF, the convex/convex-hull NMF, and the symmetric NMF. We also show that our algorithms include several existing methods for these NMF variants as special versions. The efficiency of the proposed approaches is empirically demonstrated on both real-world and synthetic data sets. It turns out that our algorithms compete favorably with five state-of-the-art alternating nonnegative least squares algorithms.

In this paper, we consider four optimization models for solving the Linear Complementarity (LCP) Problems. They are all formulated as DC (Difference of Convex functions) programs for which the unified DC programming and DCA (DC Algorithms) are applied. The resulting DCA are simple: they consist of solving either successive linear programs, or successive convex quadratic programs, or simply the projection of points on \(\mathbb{R}_{+}^{2n}\). Numerical experiments on several test problems illustrate the efficiency of the proposed approaches in terms of the quality of the obtained solutions, the speed of convergence, and so on. Moreover, the comparative results with Lemke algorithm, a well known method for the LCP, show that DCA outperforms the Lemke method.

In this paper, a version of K-median problem, one of the most popular and best studied clustering measures, is discussed. The model using squared Euclidean distances terms to which the K-means algorithm has been successfully applied is considered. A fast and robust algorithm based on DC (Difference of Convex functions) programming and DC Algorithms (DCA) is investigated. Preliminary numerical solutions on real-world databases show the efficiency and the superiority of the appropriate DCA with respect to the standard K-means algorithm.

Time-index formulation for the earliness tardiness scheduling problem has received a great attention from many researchers because lower bound obtained by linear relaxation is rather good. Much work is devoted to tackle its upper bound. In this paper, we consider this formulation by additionally proposing a deadline for each job. We also propose an approach based on DC (Difference of Convex functions) programming and DCA (DC Algorithm) to find upper bound efficiently for this problem. The results obtained are promising.

We investigate a nonconvex, nonsmooth optimization approach based on DC (Difference of Convex functions) programming and DCA (DC Algorithm) for the reduced-rank multitask linear regression problem with covariance estimation. The objective is to model the linear relationship between a multitask response and more explanatory variables by estimating a low-rank coefficient matrix and a covariance matrix. The problem is formulated as minimizing the constrained negative log-likelihood function of these two matrix variables. Then, we consider a reformulation of this problem which takes the form of a partial DC program i.e. it is a standard DC program for each variable when fixing the other variable. Next, an alternating version of a standard DCA scheme is developed. Numerical results on many synthetic multitask linear regression datasets and benchmark real datasets show the efficiency of our approach in comparison with the existing alternating/joint methods.

This paper addresses the problem of feature selection for Multi-class Support Vector Machines (MSVM). Basing on the l 0 and the l 2-l 0 regularization we consider two models for this problem. The l 0-norm is approximated by a suitable way such that the resulting optimization problems can be expressed as DC (Difference of Convex functions) programs for which DC programming and DC Algorithms (DCA) are investigated. The preliminary numerical experiments on real-world datasets show the efficiency and the superiority of our methods versus one of the best standard algorithms on booth feature selection and classification.

In this chapter, a so-called DCA method based on a DC (difference of convex functions) optimization approach for solving large-scale distance geometry problems is developed. Two main problems are considered: the exact and the general distance geometry problems. Different formulations of equivalent DC programs are introduced. Substantial subdifferential calculations permit to compute sequences of iterations in the DCA quite simply and allow exploiting sparsity in the large-scale setting. For improving the computational efficiency of the DCA schemes we investigate several techniques. A two-phase algorithm using shortest paths between all pairs of atoms to generate the complete dissimilarity matrix, a spanning trees procedure, and a smoothing technique are investigated in order to compute a good starting point (SP) for the DCAs. An important issue in the DC optimization approach is well exploited, say the nice effect of DC decompositions of the objective functions. For this purpose we propose several equivalent DC formulations based on the stability of Lagrangian duality and the regularization techniques. Finally, many numerical simulations of the molecular optimization problems with up to 12,567 variables are reported which prove the practical usefulness of the nonstandard nonsmooth reformulations, the globality of found solutions, the robustness, and the efficiency of our algorithms.

The weighted sum rate (WSR) maximizing linear precoding algorithm is studied in large correlated multiple-input single-output (MISO) interference broadcast channels (IBC). We consider an iterative WSR design via difference of convex functions (DC) programming as in [1], [2] and [3], focusing on the version in [3]. We propose an asymptotic approximation of the signal-to-interference plus noise ratio (SINR) at every iteration.

A sparse extreme learning machine framework by continuous optimization algorithms and its application in pattern recognition

Indefinite kernel support vector machine (IKSVM) has recently attracted increasing attentions in machine learning. Different from traditional SVMs, IKSVM essentially is a non-convex optimization problem. Some algorithms directly change the spectrum of the indefinite kernel matrix at the cost of losing some valuable information involved in the kernels so as to transform the non-convex problem into a convex one. Other algorithms aim to solve the dual form of IKSVM, but suffer from the dual gap between the primal and dual problems in the case of indefinite kernels. In this paper, we directly focus on the non-convex primal form of IKSVM and propose a novel algorithm termed as IKSVM-DC. According to the characteristics of the spectrum for the indefinite kernel matrix, IKSVM-DC decomposes the objective function into the subtraction of two convex functions and thus reformulates the primal problem as a difference of convex functions (DC) programming which can be optimized by the DC algorithm (DCA). In order to accelerate convergence rate, IKSVM-DC further combines the classical DCA with a line search step along the descent direction at each iteration. A theoretical analysis is then presented to validate that IKSVM-DC can converge to a local minimum. Systematical experiments on real-world datasets demonstrate the superiority of IKSVM-DC compared to state-of-the-art IKSVM related algorithms.

In this paper a novel optimization model for bilevel hierarchical clustering has been proposed. This is a hard nonconvex, nonsmooth optimization problem for which we investigate an efficient technique based on DC (Difference of Convex functions) programming and DCA (DC optimization Algorithm). Preliminary numerical results on some artificial and real-world databases show the efficiency and the superiority of this approach with respect to related existing methods.

Markov chain stochastic DCA and applications in deep learning with PDEs regularization

We consider the supervised pattern classification in the high-dimensional setting, in which the number of features is much larger than the number of observations. We present a novel approach to the sparse Fisher linear discriminant problem using the $$\ell _0$$ -norm. The resulting optimization problem is nonconvex, discontinuous and very hard to solve. We overcome the discontinuity by using appropriate approximations to the $$\ell _0$$ -norm such that the resulting problems can be formulated as difference of convex functions (DC) programs to which DC programming and DC Algorithms (DCA) are investigated. The experimental results on both simulated and real datasets demonstrate the efficiency of the proposed algorithms compared to some state-of-the-art methods.