TWO APPLICATIONS OF COMBINATORIAL OPTIMIZATION

Many of the currently best-known approximation algorithms for NP-hard optimization problems are based on Linear Programming (LP) and Semi-definite Programming (SDP) relaxations. Given its power, this class of algorithms seems to contain the most favourable candidates for outperforming the current state-of-the-art approximation guarantees for NP-hard problems, for which there still exists a gap between the inapproximability results and the approximation guarantees that we know how to achieve in polynomial time. In this thesis, we address both the power and the limitations of these relaxations, as well as the connection between the shortcomings of these relaxations and the inapproximability of the underlying problem. In the first part, we study the limitations of LP relaxations of well-known graph problems such as the Vertex Cover problem and the Independent Set problem. We prove that any small LP relaxation for the aforementioned problems, cannot have an integrality gap strictly better than $2$ and $\omega(1)$, respectively. Furthermore, our lower bound for the Independent Set problem also holds for any SDP relaxation. Prior to our work, it was only known that such LP relaxations cannot have an integrality gap better than $1.5$ for the Vertex Cover Problem, and better than $2$ for the Independent Set problem. In the second part, we study the so-called knapsack cover inequalities that are used in the current best relaxations for numerous combinatorial optimization problems of covering type. In spite of their widespread use, these inequalities yield LP relaxations of exponential size, over which it is not known how to optimize exactly in polynomial time. We address this issue and obtain LP relaxations of quasi-polynomial size that are at least as strong as that given by the knapsack cover inequalities. In the last part, we show a close connection between structural hardness for k-partite graphs and tight inapproximability results for scheduling problems with precedence constraints. This connection is inspired by a family of integrality gap instances of a certain LP relaxation. Assuming the hardness of an optimization problem on k-partite graphs, we obtain a hardness of $2-\varepsilon$ for the problem of minimizing the makespan for scheduling with preemption on identical parallel machines, and a super constant inapproximability for the problem of scheduling on related parallel machines. Prior to this result, it was only known that the first problem does not admit a PTAS, and the second problem is NP-hard to approximate within a factor strictly better than 2, assuming the Unique Games Conjecture.

Solving optimization problems with sparse or low-rank optimal solutions has been an important topic since the recent emergence of compressed sensing and its matrix extensions such as the matrix rank minimization and robust principal component analysis problems. Compressed sensing enables one to recover a signal or image with fewer observations than the “length” of the signal or image, and thus provides potential breakthroughs in applications where data acquisition is costly. However, the potential impact of compressed sensing cannot be realized without efficient optimization algorithms that can handle extremely large-scale and dense data from real applications. Although the convex relaxations of these problems can be reformulated as either linear programming, second-order cone programming or semidefinite programming problems, the standard methods for solving these relaxations are not applicable because the problems are usually of huge size and contain dense data. In this dissertation, we give efficient algorithms for solving these “sparse” optimization problems and analyze the convergence and iteration complexity properties of these algorithms. Chapter 2 presents algorithms for solving the linearly constrained matrix rank minimization problem. The tightest convex relaxation of this problem is the linearly constrained nuclear norm minimization. Although the latter can be cast and solved as a semidefinite programming problem, such an approach is computationally expensive when the matrices are large. In Chapter 2, we propose fixed-point and Bregman iterative algorithms for solving the nuclear norm minimization problem and prove convergence of the first of these algorithms. By using a homotopy approach together with an approximate singular value decomposition procedure, we get a very fast, robust and powerful algorithm, which we call FPCA (Fixed Point Continuation with Approximate SVD), that can solve very large matrix rank minimization problems. Our numerical results on randomly generated and real matrix completion problems demonstrate that this algorithm is much faster and provides much better recoverability than semidefinite programming solvers such as SDPT3. For example, our algorithm can recover 1000 × 1000 matrices of rank 50 with a relative error of 10−5 in about 3 minutes by sampling only 20 percent of the elements. We know of no other method that achieves as good recoverability. Numerical experiments on online recommendation, DNA microarray data set and image inpainting problems demonstrate the effectiveness of our algorithms. In Chapter 3, we study the convergence/recoverability properties of the fixed-point continuation algorithm and its variants for matrix rank minimization. Heuristics for determining the rank of the matrix when its true rank is not known are also proposed. Some of these algorithms are closely related to greedy algorithms in compressed sensing. Numerical results for these algorithms for solving linearly constrained matrix rank minimization problems are reported. Chapters 4 and 5 considers alternating direction type methods for solving composite convex optimization problems. We present in Chapter 4 alternating linearization algorithms that are based on an alternating direction augmented Lagrangian approach for minimizing the sum of two convex functions. Our basic methods require at most O(1/e) iterations to obtain an e-optimal solution, while our accelerated (i.e., fast) versions require at most O(1/ 3 ) iterations, with little change in the computational effort required at each iteration. For more general problem, i.e., minimizing the sum of K convex functions, we propose multiple-splitting algorithms for solving them. We propose both basic and accelerated algorithms with O(1/e) and O(1/ 3 ) iteration complexity bounds for obtaining an e-optimal solution. To the best of our knowledge, the complexity results presented in these two chapters are the first ones of this type that have been given for splitting and alternating direction type methods. Numerical results on various applications in sparse and low-rank optimization, including compressed sensing, matrix completion, image deblurring, robust principal component analysis, are reported to demonstrate the efficiency of our methods.

As a well‐structured and costly activity that pervades industries in both the public and private sector, vehicle fleet management would appear to be a splendid candidate for model‐based planning and optimization. And yet, until recently the combinatorial intricacies of vehicle routing and of vehicle scheduling have precluded the widespread use of optimization (exact) methods for this problem class. Our discussion in this paper identifies the extent and nature of these problem complexities and draws contrasts with other applications of combinatorial optimization. It also summarizes a number of successful uses of optimization for vehicle fleet planning and highlights potentially fruitful avenues for algorithmic development. In particular, we describe several alternative models and novel algorithms for the vehicle routing problem, show how various modeling approaches for this problem are intimately related, and illustrate the interplay between model formulations and the algorithms that they suggest. This discussion shows that prospects for applying exact methods, possibly in conjunction with heuristics, are far from fully realized and points to vehicle fleet planning as a tempting target of opportunity for further investigation.

Summary of Volume 3: Applications of Combinatorial Optimization

Finding the largest suborder of fixed width in a partially ordered set is an interesting combinatorial problem with applications in combinatorial optimization and scheduling. We present a polynomial time solution for this problem by transforming it into a minimum cost network flow problem in an appropriate auxiliary network.

This paper deals with algorithms for positive semidefinite matrix factorization (PSDMF). PSDMF is a recently-proposed extension of nonnegative matrix factorization with applications in combinatorial optimization, among others. In this paper, we focus on improving the local convergence of an alternating block gradient (ABC) method for PSDMF in a noise-free setting by replacing the quadratic objective function with the Poisson log-likelihood. This idea is based on truncated Wirtinger flow (TWF), a phase retrieval (PR) method that trims outliers in the gradient and thus regularizes it. Our motivation is a recent result linking PR with PSDMF. Our numerical experiments validate that the numerical benefits of TWF may carry over to PSDMF despite the more challenging setting, when initialized within its region of convergence. We then extend TWF from PR to affine rank minimization (ARM), and show that although the outliers are no longer an issue in the ARM setting, PSDMF with the new objective function may still achieves a smaller error for the same number of iterations. In a broader view, our results indicate that a proper choice of objective function may enhance convergence of matrix (or tensor) factorization methods.

PurposeThe purpose of this paper is to introduce a general equation for eigensolution. Eigenvalues and eigenvectors of graphs have many applications in combinatorial optimization and structural mechanics. Some important applications of graph products consist of nodal ordering and graph partitioning for structuring the structural matrices and finite element subdomaining, respectively.Design/methodology/approachIn the existing methods for the eigensolution of Laplacian matrices, members have been added to the model of a graph product such that for its Laplacian matrix an algebraic relation between blocks become possible. These methods are categorized as topological approaches. Here, using concepts of linear algebra a general algebraic method is developed.FindingsA new algebraic method is introduced for calculating the eigenvalues of Laplacian matrices in graph products.Originality/valueThe present method provides a simple tool for calculating the eigenvalues of the Laplacian matrices without using the configurational model and merely by using the Laplacian matrices. The developed formula for calculating the eigenvalues contains approximate terms which can be managed by the analyst.

The Schur complement trick appears sporadically in numerical optimization methods [Schur 1917; Cottle 1974]. The trick is especially useful for solving Lagrangian saddle point problems when minimizing quadratic energies subject to linear equality constraints [Gill et al. 1987]. Typically, to apply the trick, the energy's Hessian is assumed positive definite. I generalize this technique for positive semi-definite Hessians.

Multi-band RF front-ends with adaptive image rejection : a DECT/Bluetooth study case

This dissertation consists of two chapters: Chapter 1 develops nonparametric and semiparametric regression methodologies which relate the group testing responses to the individual covariates information. In this chapter, we extend the parametric regression model of Xie (2001) for binary group testing data to the nonparametric and semiparametric models. We fit nonparametric and semiparametric models and obtain estimators of the parameters by maximizing penalized likelihood function. For implementation, we apply EM algorithm considering the individual responses as complete data and the group testing responses as observed data. Simulation studies are performed to illustrate the methodologies and to evaluate the finite sample performance of our methods. In general, group testing involves a large number of subjects, hence, the computational aspect is also discussed. The results show that our estimation methods perform well for estimating both the individual probability of positive outcome and the prevalence rate in the population.Chapter 2 studies a partially linear regression model with missing response variable and develops semiparametric efficient inference for the parametric component of the model. The missingness considered here includes a broad range of missing patterns. For the estimation method, we use the concept of least favorable curve, least favorable direction and the generalized profile likelihood in Severini and Wong (1992). Asymptotic distributions for the estimators of the parametric components are obtained. It is shown that the estimators are asymptotically normally distributed under some conditions. Furthermore, we prove that the asymptotic covariance of the estimators achieves the semiparametric lower bound under the regularity conditions and additional conditions given in the appendix.We also propose an algorithm which runs iteratively between fitting parametric components and fitting nonparametric components while holding the other fixed. EM algorithms are used in estimating the parametric components by a semiparametric estimating equation and in estimating the nonparametric components by smoothing methods. It is proved that the estimators from this iterative algorithm equal to the conditional expectations (conditioned on observed data) of the semiparametric efficient estimators from complete data. The methodology is illustrated and evaluated by numerical examples.

The sensor network localization (SNL) problem in embedding dimension r consists of locating the positions of wireless sensors, given only the distances between sensors that are within radio range and the positions of a subset of the sensors (called anchors). Current solution techniques relax this problem to a weighted, nearest, (positive) semidefinite programming (SDP) completion problem by using the linear mapping between Euclidean distance matrices (EDM) and semidefinite matrices. The resulting SDP is solved using primal-dual interior point solvers, yielding an expensive and inexact solution. This relaxation is highly degenerate in the sense that the feasible set is restricted to a low dimensional face of the SDP cone, implying that the Slater constraint qualification fails. Cliques in the graph of the SNL problem give rise to this degeneracy in the SDP relaxation. In this paper, we take advantage of the absence of the Slater constraint qualification and derive a technique for the SNL problem, with exact data, that explicitly solves the corresponding rank restricted SDP problem. No SDP solvers are used. For randomly generated instances, we are able to efficiently solve many huge instances of this NP-hard problem to high accuracy by finding a representation of the minimal face of the SDP cone that contains the SDP matrix representation of the EDM. The main work of our algorithm consists in repeatedly finding the intersection of subspaces that represent the faces of the SDP cone that correspond to cliques of the SNL problem.

The comparative performance of Integer Programming (IP) and Constraint Logic Programming (CLP) is explored by examining a number of models for four different combinatorial optimization applications. Computational results show contrasting behavior for the two approaches, and an analysis of performance with respect to problem and model characteristics is presented. The analysis shows that tightness of formulation is of great benefit to CLP where effective search reduction results in problems that can be solved quickly. In IP, if the linear feasible region does not identify the corresponding integer polytope, the problem may be difficult to solve. The paper identifies other characteristics of model behavior and concludes by examining ways in which IP and CLP may be incorporated within hybrid solvers.

Entanglement is a key resource in the emerging field of Quantum Information. The strong correlations between systems described by an entangled state allow us to perform certain tasks more efficiently than it would be possible by using only classical resources. This is why the characterization of entanglement is one of the most important problems in Quantum Information. In this thesis, we analyze several aspects of entanglement. First, we introduce a new family of criteria to determine if a bipartite mixed state is entangled or not. This family consists of a sequence of tests that can be implemented efficiently, and has the property that all entangled states can be detected by some test in the sequence. Each test in the family can be stated as a semidefinite program, which is a class of convex optimization problems. The duality structure of these programs allows us to explicitly construct an entanglement witness that proves entanglement of a state, whenever the state fails one of the tests in the sequence. The entanglement witnesses constructed in this manner have well-defined algebraic properties that can be used to give a characterization of the interior of the set of all possible entanglement witnesses, as well as the set of strictly positive bihermitian forms and the set of strictly positive maps. We also study deterministic transformations of three-qubit pure state when only local operations and classical communication (LOCC) are allowed. We derive strong constraints that the operations and states involved must satisfy, and we apply these results to characterize the set of real states that can be obtained from the GHZ state by LOCC.

Following their divorce, Alice and Bob would like to split some of their possessions by flipping a coin. Unwilling to meet in person, and without a trusted third party, they must figure out a scheme to flip the coin over a telephone that guarantees that neither party can cheat. The preceding scenario is the traditional definition of two-party coin-flipping. In a classical setting, without limits on the available computational power, one player can always guarantee a coin-flipping victory by cheating. However, by employing quantum communication it is possible to guarantee, with only information-theoretic assumptions, that neither party can win by cheating, with a probability greater than two thirds. Along with the description of such a protocol, this thesis derives a tight lower bound on the bias for a large family of quantum weak coin-flipping protocols, proving such a protocol optimal within the family. The protocol described herein is an improvement and generalization of one examined by Spekkens and Rudolph. The key steps of the analysis involve Kitaev's description of quantum coin-flipping as a semidefinite program whose dual problem provides a certificate that upper bounds the amount of cheating for each party. In order for such quantum protocols to be viable, though, a number of practical obstacles involving the communication and processing of quantum information must be resolved. In the second half of this thesis, a scheme for processing quantum information is presented, which uses non-abelian anyons that are the magnetic and electric excitations of a discrete-group quantum gauge theory. In particular, the connections between group structure and computational power are examined, generalizing previous work by Kitaev, Ogburn and Preskill. Anyon based computation has the advantage of being topological, which exponentially suppresses the rate of decoherence and the errors associated with the elementary quantum gates. Though no physical systems with such excitations are currently known to exist, it remains an exciting open possibility that such particles could be either engineered or discovered in exotic two-dimensional systems.

Improving the linear relaxation of maximum k-cut with semidefinite-based constraints