Column Generation Algorithms for Nonlinear Optimization, I: Convergence Analysis

Column generation is a fundamental technique for solving large-scale combinatorial optimization problems such as unit commitment and vehicle routing, yet its performance is often limited by dual oscillation. This study explores the intrinsic cause of this phenomenon from the perspective of shadow price theory and demonstrates that dual oscillation arises from the lack of marginal interpretability of Lagrange multipliers when multiple dual solutions coexist. To address this issue, an improved column generation framework is proposed in which traditional multipliers are replaced with minimum-norm multipliers that possess clear economic meaning and act as directional shadow prices. A generalized pricing subproblem is formulated, and partial minimum-norm multipliers are obtained through convex quadratic optimization to guide column generation. Numerical experiments on a simplified single-period unit commitment case and large-scale cutting stock problems showed that the proposed approach eliminated invalid column generation and achieved speedy convergence to the optimal solution within only two iterations for the unit commitment case, and the classical column generation exhibited slow convergence with dual oscillation in large-scale scenarios while the improved algorithm achieved fast and stable convergence. The results indicate that the stabilization method enhances the consistency of dual variables and provides a more robust foundation for the theoretical and practical development of column generation algorithms.

The minimum sum-of-squares clustering problem (MSSC), also known as k-means clustering, refers to the problem of partitioning n data points into k clusters, with the objective of minimizing the total sum of squared Euclidean distances between each point and the center of its assigned cluster. We propose an efficient algorithm for solving large-scale MSSC instances, which combines column generation (CG) with dynamic constraint aggregation (DCA) to effectively reduce the number of constraints considered in the CG master problem. DCA was originally conceived to reduce degeneracy in set partitioning problems by utilizing an aggregated restricted master problem obtained from a partition of the set partitioning constraints into disjoint clusters. In this work, we explore the use of DCA within a CG algorithm for MSSC exact solution. Our method is fine-tuned by a series of ablation studies on DCA design choices, and is demonstrated to significantly outperform existing state-of-the-art exact approaches available in the literature. History: Accepted by Andrea Lodi, Design & Analysis of Algorithms–Discrete. Funding: This work was supported by Natural Sciences and Engineering Research Council of Canada [Grant 2023-04466]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2024.0938 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2024.0938 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

In this paper, we introduce the Electric Vehicle Routing Problem with Flexible Time Windows (EVRPFTW) in which vehicles are allowed to serve customers before and after the earliest and latest time window bounds, respectively. The objective of this problem is to assign electric vehicles to feasible routes and make schedules with minimum total cost that includes the traveling costs, the costs of using electric vehicles and the penalty costs incurred for earliness and lateness. The proposed mathematical model is solved by a column generation procedure. To generate an integer solution, we solve an integer programming problem using the routes constructed by the column generation algorithm. We further develop a linear programming model to compute the optimal times to start service at each customer for the selected routes. A number of well-known benchmark instances is solved by our solution procedure to evaluate the operational gains obtained by employing flexible time windows.

We present a new model and a column generation algorithm for the linear minimum cost multicommodity flow problem. The new model has two differences when compared to the classic formulation based on flows on paths: it has an additional set of variables and an additional set of inequalities. The extra variables are associated with flows on circuits where some arcs are reversed. The extra inequalities assure that an optimal solution to the extended model can be converted to an optimal solution to the path formulation. In order to obtain an optimal solution to the extended model, we use column generation. The extra variables are explicitly considered in the restricted master problem, from the beginning of the column generation process. The subproblem remains a set of shortest path problems. Our present work is related to the use of dual–optimal inequalities in the sense that we extend a model with additional primal variables/dual inequalities. However, our extended model also has additional primal inequalities/dual variables. We present computational results of our method in planar instances and compare them with standard column generation, a bundle method, and a general‐purpose solver. For the tested instances, there is an effective improvement in computational time of the column generation method when the extended model is used. © 2007 Wiley Periodicals, Inc. NETWORKS, Vol. 50(1), 3–16 2007

Designing networks with resiliency to edge failures using two-stage robust optimization

An effective resource allocation technique is required for each Ready Mixed Concrete (RMC). Finding the optimum solution for large scale RMC dispatching problems with available computing facilities is intractable. Two kinds of techniques have been implemented to deal with this problem: (i) evolutionary techniques and (ii) numerical techniques. For the purposes of this paper we selected a technique from each category and compared them under the same conditions. Robust Genetic Algorithm (Robust-GA) and Column Generation (CG) were selected and tested with different sizes of real RMC problems. The results show that on average CG solutions are obtained with a 20% reduced cost. However, Robust-GA converges 40% faster than CG, while the number of unassigned customers for both the techniques is almost the same.

One of the major tasks of telecommunications network planners is deciding where and how much spare capacity to build so that disrupted traffic may be rerouted in the event of a network failure. This paper presents an experimental study comparing two techniques for restoration capacity planning. The first, linear programming using column generation, produces optimal fractional solutions that can be integerized in practice for little extra cost. This approach is time-consuming, however, and for some networks of interest the linear programs are too large to be practically solvable. The second algorithm is a fast heuristic called LOCAL, which can be practically applied to much larger problem sizes than column generation. A fast linear-programming lower bound is used to measure the efficiency of the solutions. The purpose of the study was threefold: to determine how much benefit is obtained by using column generation, when the problem size is small enough that column generation is practical; to determine the quality of solutions produced by LOCAL on both small and large problems; and to investigate the utility of the lower-bound LP.We find that column generation produces networks whose restoration capacity cost is 10% to 16% less than those produced by LOCAL. Sometimes the total cost of the network is of primary interest, including both the cost of service and restoration capacity, and in this case the difference between column generation and LOCAL is 4% to 6%. Column generation is the method of choice for making final decisions about purchasing spare capacity. When millions are to be spent, running time is not a consideration, unless the computation simply will not complete in the available time. In such cases, the differential between column generation and LOCAL is small enough and consistent enough that LOCAL can safely be used. LOCAL is the method of choice for network design spreadsheet applications, where different architectures can be quickly compared, modified, and then compared again, and where a consistent approximation to the optimum is sufficient. The bound produced by the lower-bound LP is consistently extremely close to the true optimum, so the lower-bound LP can be effectively used to monitor the performance of both methods.

We survey the main results obtained by the author in his PhD dissertation supervised by Anass Nagih and Lucas Letocart. It was defended in December 2008 at The Computer Science lab of the Paris-Nord University (L.I.P.N.), Villetaneuse, France. The thesis is written in French and is available from http://www-lipn.univ-paris13.fr/~touati/TM_Thesis.pdf. Column generation algorithms are instrumental in many areas of applied optimization, where linear programs with an enormous number of variables need to be solved. Although successfully used in many applications, this method suffers from well-known “instability” issues that somewhat limit its efficiency. This work focuses on accelerating strategies in a column generation scheme; on the first iterations using diversification, on the last iterations using reoptimization and on the overall GC scheme using an improving technique to solve the pricing problems (dynamic programming with blocs). The effectiveness of these approaches is validated on the Vehicle Routing Problem with Time Windows.

Dantzig–Wolfe decomposition of the daily course pattern formulation for curriculum-based course timetabling

During the past few years, there has been a trend to enrich traditional revenue management models built upon the independent demand paradigm by accounting for customer choice behavior. This extension involves both modeling and computational challenges. One way to describe choice behavior is to assume that each customer belongs to a segment, which is characterized by a consideration set, i.e., a subset of the products provided by the firm that a customer views as options. Customers choose a particular product according to a multinomial-logit criterion, a model widely used in the marketing literature. In this paper, we consider the choice-based, deterministic, linear programming model (CDLP) of Gallego et al. (2004) [Gallego, G., G. Iyengar, R. Phillips, A. Dubey. 2004. Managing flexible products on a network. Technical Report CORC TR-2004-01, Department of Industrial Engineering and Operations Research, Columbia University, New York], and the follow-up dynamic programming decomposition heuristic of van Ryzin and Liu (2008) [van Ryzin, G. J., Q. Liu. 2008. On the choice-based linear programming model for network revenue management. Manufacturing Service Oper. Management 10(2) 288–310]. We focus on the more general version of these models, where customers belong to overlapping segments. To solve the CDLP for real-size networks, we need to develop a column generation algorithm. We prove that the associated column generation subproblem is indeed NP-hard and propose a simple, greedy heuristic to overcome the complexity of an exact algorithm. Our computational results show that the heuristic is quite effective and that the overall approach leads to high-quality, practical solutions.

This paper reports on our attempt to design an efficient exact algorithm based on column generation for the cutting stock problem. The main focus of the research is to study the extend to which standard branch-and-bound enhancement features such as variable fixing, the tightening of the formulation with cutting planes, early branching, and rounding heuristics can be usefully incorporated in a branch-and-price algorithm. We review and compare lower bounds for the cutting stock problem. We propose a pseudo-polynomial heuristic. We discuss the implementation of the important features of the integer programming column generation algorithm and, in particular, the implementation of the branching scheme. Our computational results demonstrate the efficiency of the resulting algorithm for various classes of bin packing and cutting stock problems.

An exact approach for maximizing the lifetime of sensor networks with adjustable sensing ranges

Magnetic resonance imaging (MRI)-guided Co-60 provides daily and intrafractional MRI soft tissue imaging for improved target and critical organ tracking. To increase delivery efficiency, the system uses three Co-60 sources at 120° apart, allowing up to 600 cGy combined dose rate at isocenter. Despite the potential tripling in output, creating a delivery plan that uses all three sources is considerably unintuitive. Here, the authors computerize the triplet orientation optimization using column generation, an approach that was demonstrated effective in integrated beam orientation and fluence optimization for noncoplanar therapies. To achieve a better plan quality without increasing the treatment time, the authors then solve a fluence map optimization (FMO) problem while regularizing the fluence maps to reduce the number of deliverable MLC segments. Three patients-one prostate, one lung, and one head and neck boost plan (H&NBoost)-were evaluated in this study. For each patient, the beamlet doses were calculated using Monte Carlo, under a 0.35 T magnetic field, for 180 equally spaced coplanar beams grouped into 60 triplets. The beamlet size is 1.05 × 0.5 cm determined by the MLC leaf thickness and step size. The triplets were selected using the column generation algorithm. The FMO problem was formulated using an L2-norm dose fidelity term and an L1-norm anisotropic total variation regularization term, which allows controlling the number of MLC segments, and hence the treatment time, with minimal degradation to the dose. The authors' Fluence Regularization and Optimized Selection of Triplets (FROST) plans were compared against the clinical treatment plans (CLNs) produced by an experienced dosimetrist. PTV homogeneity, max dose, mean dose, D95, D98, and D99 were evaluated. OAR max and mean doses, as well as R50, defined as the ratio of the 50% isodose volume over the planning target volume were investigated. The mean PTV D95, D98, and D99 differ by +0.04%, +0.07%, and +0.25% of the prescription dose between planning methods. The mean PTV homogeneity was virtually same with values at 0.8788 (FROST) and 0.8812 (CLN). R50 decreased by 0.67 comparing FROST to CLN. On average, FROST reduced Dmax and Dmean of OARs by 7.30% and 6.08% of the prescription dose, respectively. The manual CLN planning processes required numerous trial and error runs. The FROST plans on the other hand required minimal human intervention. Efficient delivery of MRI-guided Co-60 therapy needs the output of multiple sources yet suffers from unintuitive and laborious manual beam selection processes. Computerized triplet orientation optimization improves both planning efficiency and plan dosimetry. The novel fluence map regularization provides additional controls over the number of MLC segments and treatment time.

Given n points in ℝd and a maximum allowed tolerance ϵ > 0, the minimum hyperplanes clustering problem consists in finding a minimum number of hyperplanes such that the Euclidean distance between each point and the nearest hyperplane is at most ϵ. We present a column generation approach for this problem based on a mixed integer nonlinear formulation in which the master is a set covering problem and the pricing subproblem is a mixed integer program with a nonconvex normalization constraint. We propose different ways of generating the initial pool of columns and investigate their impact on the overall algorithm. Since the pricing subproblem is substantially complicated by the ℓ2-norm constraint, we consider approximate pricing subproblems involving different norms. Some strategies for refining the solution and speeding-up the overall method are also discussed. The performance of our column generation algorithm is assessed on realistic randomly generated instances as well as on real-world instances.

In this paper, we address the electric vehicle routing problem with time windows and propose two branch-and-price-and-cut methods based on a column generation algorithm. One is an exact algorithm whereas the other is a heuristic method. The pricing sub-problem of the column generation method is solved using a label correcting algorithm. The algorithms are strengthened with the state-of-the-art acceleration techniques and a set of valid inequalities. The acceleration techniques include: (i) an intermediate column pool to prevent solving the pricing sub-problem at each iteration, (ii) a label correcting method employing the ng-route algorithm adopted to our problem, (iii) a bidirectional search mechanism in which both forward and backward labels are created, (iv) a procedure for dynamically eliminating arcs that connect customers to remote stations from the network during the path generation, (v) a bounding procedure providing early elimination of sub-optimal routes, and (vi) an integer programming model that generates upper bounds. Numerical experiments are conducted using a benchmark data set to compare the performances of the algorithms. The results favour the heuristic algorithm in terms of both the computational time and the number of instances solved. Moreover, the heuristic algorithm is shown to be specifically effective for larger instances. Both algorithms introduce a number of new solutions to the literature.