DC programming and DCA for sparse Fisher linear discriminant analysis

In this paper, we consider four optimization models for solving the Linear Complementarity (LCP) Problems. They are all formulated as DC (Difference of Convex functions) programs for which the unified DC programming and DCA (DC Algorithms) are applied. The resulting DCA are simple: they consist of solving either successive linear programs, or successive convex quadratic programs, or simply the projection of points on \(\mathbb{R}_{+}^{2n}\). Numerical experiments on several test problems illustrate the efficiency of the proposed approaches in terms of the quality of the obtained solutions, the speed of convergence, and so on. Moreover, the comparative results with Lemke algorithm, a well known method for the LCP, show that DCA outperforms the Lemke method.

DC approximation approaches for sparse optimization

In this letter, we consider the nonnegative matrix factorization (NMF) problem and several NMF variants. Two approaches based on DC (difference of convex functions) programming and DCA (DC algorithm) are developed. The first approach follows the alternating framework that requires solving, at each iteration, two nonnegativity-constrained least squares subproblems for which DCA-based schemes are investigated. The convergence property of the proposed algorithm is carefully studied. We show that with suitable DC decompositions, our algorithm generates most of the standard methods for the NMF problem. The second approach directly applies DCA on the whole NMF problem. Two algorithms-one computing all variables and one deploying a variable selection strategy-are proposed. The proposed methods are then adapted to solve various NMF variants, including the nonnegative factorization, the smooth regularization NMF, the sparse regularization NMF, the multilayer NMF, the convex/convex-hull NMF, and the symmetric NMF. We also show that our algorithms include several existing methods for these NMF variants as special versions. The efficiency of the proposed approaches is empirically demonstrated on both real-world and synthetic data sets. It turns out that our algorithms compete favorably with five state-of-the-art alternating nonnegative least squares algorithms.

It is undoubtedly that mathematical modelling and optimisation play a key role in the supply chain and the production management (SCPM). In this paper, we provide a survey on DC (Difference of Convex function) programming and DCA (DC Algorithm), a state-of-the-art optimisation approach for challenging problems in SCPM. DC programming and DCA constitute the backbone of non-convex programming and global optimisation. Whilst DC programming and DCA were widely and successfully investigated in many areas, it seems that they were not so much popular in the community of SCPM. There is therefore a need to further develop this efficient and scalable approach for SCPM applications, especially for large-scale problems in the context of Big data. For such purpose, this paper aims to present benchmark models and state-of-the-art DCA-based methods for solving challenging problems in SCPM systems. We prove that all the benchmark classes of optimisation models appeared in SCPM systems can be formulated/reformulated as a DC program and show how to solve these classes of problems by DCA-based algorithms. We offer the community of researchers in SCPM efficient algorithms in a unified DC programming framework to tackle various applications such as supply chain design, scheduling, multi-stage production/inventory system, vehicle routing, …

This letter proposes a novel approach using the [Formula: see text]-norm regularization for the sparse covariance matrix estimation (SCME) problem. The objective function of SCME problem is composed of a nonconvex part and the [Formula: see text] term, which is discontinuous and difficult to tackle. Appropriate DC (difference of convex functions) approximations of [Formula: see text]-norm are used that result in approximation SCME problems that are still nonconvex. DC programming and DCA (DC algorithm), powerful tools in nonconvex programming framework, are investigated. Two DC formulations are proposed and corresponding DCA schemes developed. Two applications of the SCME problem that are considered are classification via sparse quadratic discriminant analysis and portfolio optimization. A careful empirical experiment is performed through simulated and real data sets to study the performance of the proposed algorithms. Numerical results showed their efficiency and their superiority compared with seven state-of-the-art methods.

In this paper, a version of K-median problem, one of the most popular and best studied clustering measures, is discussed. The model using squared Euclidean distances terms to which the K-means algorithm has been successfully applied is considered. A fast and robust algorithm based on DC (Difference of Convex functions) programming and DC Algorithms (DCA) is investigated. Preliminary numerical solutions on real-world databases show the efficiency and the superiority of the appropriate DCA with respect to the standard K-means algorithm.

As a development of ν-support vector machine (ν-SVM), parametric-margin ν-support vector machine (Par-ν-SVM) can be useful in many cases, especially heteroscedastic noise classification problems. The present article proposes a novel and fast method to solve the primal problem of Par-ν-SVM (named as DC-Par-ν-SVM), while Par-ν-SVM maximizes the parametric-margin by solving a dual quadratic programming problem. In fact, the primal non-convex problem is converted into an unconstrained problem to express the objective function as the difference of convex functions (DC). The DC-Algorithm (DCA) based on generalized Newton’s method is proposed to solve the unconstrained problem cited. Numerical experiments performed on several artificial, real-life, UCI and NDC data sets showed the superiority of the DC-Par-ν-SVM in terms of both accuracy and learning speed.

Time-index formulation for the earliness tardiness scheduling problem has received a great attention from many researchers because lower bound obtained by linear relaxation is rather good. Much work is devoted to tackle its upper bound. In this paper, we consider this formulation by additionally proposing a deadline for each job. We also propose an approach based on DC (Difference of Convex functions) programming and DCA (DC Algorithm) to find upper bound efficiently for this problem. The results obtained are promising.

We investigate a nonconvex, nonsmooth optimization approach based on DC (Difference of Convex functions) programming and DCA (DC Algorithm) for the reduced-rank multitask linear regression problem with covariance estimation. The objective is to model the linear relationship between a multitask response and more explanatory variables by estimating a low-rank coefficient matrix and a covariance matrix. The problem is formulated as minimizing the constrained negative log-likelihood function of these two matrix variables. Then, we consider a reformulation of this problem which takes the form of a partial DC program i.e. it is a standard DC program for each variable when fixing the other variable. Next, an alternating version of a standard DCA scheme is developed. Numerical results on many synthetic multitask linear regression datasets and benchmark real datasets show the efficiency of our approach in comparison with the existing alternating/joint methods.

This paper addresses the problem of feature selection for Multi-class Support Vector Machines (MSVM). Basing on the l 0 and the l 2-l 0 regularization we consider two models for this problem. The l 0-norm is approximated by a suitable way such that the resulting optimization problems can be expressed as DC (Difference of Convex functions) programs for which DC programming and DC Algorithms (DCA) are investigated. The preliminary numerical experiments on real-world datasets show the efficiency and the superiority of our methods versus one of the best standard algorithms on booth feature selection and classification.

We propose a local regularization of elliptic optimal control problems which involves the nonconvex $$L^q$$ quasi-norm penalization in the cost function. The proposed Huber type regularization allows us to formulate the PDE constrained optimization instance as a DC programming problem (difference of convex functions) that is useful to obtain necessary optimality conditions and tackle its numerical solution by applying the well known DC algorithm used in nonconvex optimization problems. By this procedure we approximate the original problem in terms of a consistent family of parameterized nonsmooth problems for which there are efficient numerical methods available. Finally, we present numerical experiments to illustrate our theory with different configurations associated to the parameters of the problem.

Optimal correction of an infeasible equations system as Ax + B|x|= b leads into a non-convex fractional problem. In this paper, a regularization method(ℓp-norm, 0 < p < 1), is presented to solve mentioned fractional problem. In this method, the obtained problem can be formulated as a non-convex and nonsmooth optimization problem which is not Lipschitz. The objective function of this problem can be decomposed as a difference of convex functions (DC). For this reason, we use a special smoothing technique based on DC programming. The numerical results obtained for generated problem show high performance and the effectiveness of the proposed method.

In this chapter, a so-called DCA method based on a DC (difference of convex functions) optimization approach for solving large-scale distance geometry problems is developed. Two main problems are considered: the exact and the general distance geometry problems. Different formulations of equivalent DC programs are introduced. Substantial subdifferential calculations permit to compute sequences of iterations in the DCA quite simply and allow exploiting sparsity in the large-scale setting. For improving the computational efficiency of the DCA schemes we investigate several techniques. A two-phase algorithm using shortest paths between all pairs of atoms to generate the complete dissimilarity matrix, a spanning trees procedure, and a smoothing technique are investigated in order to compute a good starting point (SP) for the DCAs. An important issue in the DC optimization approach is well exploited, say the nice effect of DC decompositions of the objective functions. For this purpose we propose several equivalent DC formulations based on the stability of Lagrangian duality and the regularization techniques. Finally, many numerical simulations of the molecular optimization problems with up to 12,567 variables are reported which prove the practical usefulness of the nonstandard nonsmooth reformulations, the globality of found solutions, the robustness, and the efficiency of our algorithms.

The multiple-input multiple-output (MIMO) system is widely used in wireless communications. For the problem of the discrete maximum-likelihood (ML) detection for the MIMO system, one can formulate it as binary quadratic programming (BQP). The general BQP problem is an NP-hard problem, which is a challenge for finding promising solutions. The variable complexity is a special considered issue. In this paper, inspired by the optimization of sparse constrains, we employ a regularization approach to deal with the binary constraints in the proposed formulation and then introduce the difference of convex functions (DC) programming to solve the formulated nonconvex cost function. A novel and robust DC algorithm is proposed. Numerical experiments show that the proposed algorithm, which is based on DC programming, can achieve accurate results with a higher convergence rate.

Massive connectivity has become a critical requirement for Internet-of-Things (IoT) networks, where a large number of devices need to connect to an access-point sporadically. Moreover, low-latency communication and sporadic device traffic are essential to support intelligent services in IoT networks. In this paper, to support low-latency communication for massive devices with sporadic traffic, we present a sparse blind demixing to simultaneously detect the active devices and decode multiple source signals without a priori channel state information in multi- in-multi-out (MIMO) networks. To address the unique challenges of bilinear measurements and sporadic device activity detection, we recast the estimation problem as a sparse and low-rank optimization problem via matrix lifting. We further propose a difference-of-convex-functions (DC) representation for the rank function to guarantee the exact rank constraint, followed by ignoring the non-convex group sparse function. This is achieved by exploiting the difference between nuclear norm and the convex Ky Fan k- norm for a rank function representation. We then develop an efficient DC algorithm to solve the resulting non-convex DC program without regularization parameter. Numerical results demonstrate that the proposed DC approach is able to exactly recover the ground truth signals with reduced sample sizes, as well as achieve better performance against noise compared with the existing convex methods.