Leveraging linear and mixed integer programming for SMT

In this paper we present a new progressive cooperating simplifier for deciding the satisfiability of a quantifier-free formula in the first-order theory of integers involving combinations of sublogics, referred to as Satisfiability Modulo Theories (SMT). Our approach, given an SMT problem, replaces each non-propositional theory atom with a Boolean indicator variable yielding a purely propositional formula to be decided by a SAT solver. Starting with the most abstract representation (the Boolean formula), the solver gradually integrates more complex theory solvers into the working decision procedure. Additionally, we propose a method to simplify “expensive” atoms into suitable conjunctions of “cheaper” theory atoms when conflicts occur. This process considerably increases the efficiency of the overall procedure by reducing the number of calls to the slower theory solvers. This is made possible by adopting our novel inter-logic implication framework, as proposed in this paper. We have implemented these methods in our Ario SMT solver by combining three different theory solvers within a DPLL-style SAT solver: a Unit-Two-Variable-Per-Inequality (UTVPI) solver, an integer linear programming (ILP) solver, and a solver for systems of equalities with uninterpreted functions. The efficiencies of our proposed algorithms are demonstrated and exhaustively investigated on a wide range of benchmarks in hardware and software verification domain. Empirical results are also presented showing the advantages/limitations of our methods over other modern techniques for solving these SMT problems.

Numerous computer models for river basin planning and management have been developed and used extensively since the mid‐1970s. Early developments have relied on the use of network flow algorithms (NFA), due mainly to higher execution speed than the standard Simplex solvers. However, subsequent efforts to include proper modeling of hydraulic and hydrologic constraints introduced iterative schemes into the NFA‐based models, which diminished the initial advantages in execution speed and which also caused concerns over the accuracy of the convergence schemes. Hence full‐blown commercial linear programming (LP) solvers were introduced as a replacement to the iterative solution strategy of the NFA approach. This paper demonstrates one possible failure to solve a simple allocation problem using the NFA‐based model and shows how this problem can be solved using the standard LP approach. It then identifies cases when even a full‐blown LP approach cannot properly model two critical aspects of river basin management, one related to reservoirs with multiple outflows and the other one related to modeling of hydrologic channel routing. For NFA‐based models the failures are the result of the inability to include relationships between flows on different model components directly into the search process. For the models based on LP solvers, the failures are caused by the fact that integrated reservoir outflow capacity between the starting and the ending storage levels is assumed over the entire length of the assumed time step, while the actual outflow can only take place during the portion of the time step when the storage level is above the invert of the outlet structure.

Advances in technology for the manufacturing of integrated circuits have resulted in extremely large, and time consuming, problems on how to lay out components for optimal circuit performance. These problems can be written as mixed integer programs which are easily relaxed to linear programs with a very high number of variables and constraints. The relaxed programs can often be solved by applying state-of-the-art linear programming software, however these solutions come at the expense of long solution time. In this paper we show that, by considering the structure inherent in VLSI problems, one can specialize classical preprocessing algorithms to take into account the standard form of the constraint matrix for VLSI problems, thereby achieving improved preprocessing results with relatively little effort. We provide analysis showing our preprocessing techniques are accurate and provide some numerical testing demonstrating the increased efficiency. The numerical tests also demonstrate that using our preprocessing in conjunction with internal preprocessing methods that come with many linear program solvers, can improve the overall performance of the linear program solver and its preprocessor.

In this work, we develop a new framework for dynamic network flow problems based on optimal transport theory. We show that the dynamic multicommodity minimum-cost network flow problem can be formulated as a multimarginal optimal transport problem, where the cost function and the constraints on the marginals are associated with a graph structure. By exploiting these structures and building on recent advances in optimal transport theory, we develop an efficient method for such entropy-regularized optimal transport problems. In particular, the graph structure is utilized to efficiently compute the projections needed in the corresponding Sinkhorn iterations, and we arrive at a scheme that is both highly computationally efficient and easy to implement. To illustrate the performance of our algorithm, we compare it with a state-of-the-art linear programming (LP) solver. We achieve good approximations to the solution at least one order of magnitude faster than the LP solver. Finally, we showcase the methodology on a traffic routing problem with a large number of commodities. Funding: This work was supported by KTH Digital Futures, Knut och Alice Wallenbergs Stiftelse [Grants KAW 2018.0349, KAW 2021.0274, the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation], Vetenskapsrådet [Grant 2020-03454], and the National Science Foundation [Grants 1942523 and 2206576].

The row pivoting method for linear programming

When developing linear programming models, issues such as budget limitations, customer requirements, or licensing may preclude the use of commercial linear programming solvers. In such cases, one option is to use an open-source linear programming solver. A survey of linear programming tools was conducted to identify potential open-source solvers. From this survey, four open-source solvers were tested using a collection of linear programming test problems and the results were compared to IBM ILOG CPLEX Optimizer (CPLEX) [1], an industry standard. The solvers considered were: COIN-OR Linear Programming (CLP) [2], [3], GNU Linear Programming Kit (GLPK) [4], lp_solve [5] and Modular In-core Nonlinear Optimization System (MINOS) [6]. As no open-source solver outperforms CPLEX, this study demonstrates the power of commercial linear programming software. CLP was found to be the top performing open-source solver considered in terms of capability and speed. GLPK also performed well but cannot match the speed of CLP or CPLEX. lp_solve and MINOS were considerably slower and encountered issues when solving several test problems.

In our previous work we have presented an approach to learn interpretable classification rules using a Boolean compressed sensing formulation. Our approach uses a linear programming (LP) relaxation and allows us to find interpretable (sparse) classification rules that achieve good generalization accuracy. However, the resulting LP representation for problems with either a large number of samples or large number of continuous features tends to become challenging for off-the-shelf LP solvers. We have explored a screening approach which allows us to dramatically reduce the number of active features without sacrificing optimality. In this work we explore reducing the number of samples in a sequential setting where we can certify reaching a near-optimal solution while only solving the LP on a small fraction of the available data points. In a batch setting this approach can dramatically reduce the computational complexity of the rule-learning LP formulation. In an online setting we derive stochastic upper and lower bounds on the the LP objective for unseen samples. This allows early stopping when we detect that the classifier will not change significantly with additional samples. The upper bounds are related to the learning curve literature in machine learning, and our lower bounds appear not to have been explored. Finally, we discuss a quick approach to compute the complete regularization path balancing rule interpretability versus accuracy.

COHO is a verification tool for systems modeled by nonlinear ordinary differential equations (ODEs). Verification is performed using reachability analysis. The reachable space is represented by projectagons which are the polyhedron described by their projection onto two dimensional subspace. COHO approximates nonlinear ODEs with linear differential inclusions to compute the reachable state. We re-engineer the COHO system to provide a robust implementation with a clear interface and well-organized structure. We demonstrate the soundness and robustness of our approach by applying it to several examples, including a three dimensional, non-linear systems for which previous version of COHO failed due to numerical stability problems. The correctness of COHO strongly depends on the accuracy, efficiency and robustness of the linear program solver that is used throughout the analysis. Our COHO linear program solver is implemented by integrating the Simplex algorithm with interval arithmetic based on the framework set up by Laza [Laz01]. For highly ill-conditioned problems, an approximation that is very close to the optimal result is computed by our algorithm. This results in a very small error bound that allows COHO to successfully verify interesting examples. This thesis also presents an algorithm to solve the problem of projecting the feasible region of a linear program onto two dimensional subspaces. This algorithm uses the COHO linear program solver for efficiency and accuracy. We derive an analytical upper bound for the error and present experimental results to show that the errors are negligible in practice.

As discrete jumps and continuous flows tangle in the behavior of linear hybrid automata (LHA), the bounded model checking (BMC) for reachability of LHA is a challenging problem. Current works try to handle this problem by encoding all the discrete and continuous behaviors in the bound into a set of SMT formulas which can then be solved by SMT solvers. However, when the system size is large, the object SMT problem could be huge and difficult to solve. Instead of encoding everything into one constraint set, this paper proposes a SAT---LP---IIS joint-directed solution to conduct the BMC for reachability of LHA in a layered way. First, the bounded graph structure of LHA is encoded into a propositional formula set, and solved by SAT solvers to find potential paths which can reach the target location on the graph. Then, the feasibility of certain path is encoded into a set of linear constraints which can then be solved by linear programming (LP) efficiently. If the path is not feasible, irreducible infeasible set (IIS) technique is deployed to locate an infeasible path segment which will be fed to the SAT solver to accelerate the enumerating process. Experiments show that by this SAT---LP---IIS joint-directed solution, the memory usage of the BMC of LHA is well-controlled and the performance outperforms the state-of-the-art SMT-style competitors significantly.

We present an approach for the verification of feed-forward neural networks in which all nodes have a piece-wise linear activation function. Such networks are often used in deep learning and have been shown to be hard to verify for modern satisfiability modulo theory (SMT) and integer linear programming (ILP) solvers.

The equitable dissemination of COVID-19 vaccinations in lower-income nations presents a crucial worldwide dilemma, necessitating smart allocation techniques to guarantee impartial and effective coverage. This paper introduces a mathematical approach that utilizes Linear Programming (LP) to efficiently distribute vaccination doses among lower-income countries. The model incorporates crucial variables such population magnitude, healthcare infrastructure capability, susceptibility to the virus, financial limitations, and worldwide vaccine availability. The linear programming (LP) model is constructed using decision variables that indicate the quantity of vaccination doses assigned to each country. The goal function seeks to optimize the fair allocation of resources by giving weights to countries according to certain criteria. Constraints are imposed to ensure compliance with global vaccine supply, population coverage targets, healthcare capacity, vulnerability concerns, and budget limits. The non-negativity constraints guarantee that the vaccination allocations will always be greater than or equal to zero. The model is designed to be flexible, taking into account dynamic factors such as emerging viral strains and fluctuating vaccination availability. It highlights the significance of timely data updates to facilitate decision-making. The LP solver, such as the simplex method or interior-point approaches, is used to determine the best solution that achieves a fair distribution while considering logistical limitations. The results obtained from the linear programming (LP) model offer practical and effective guidance for policymakers and health organizations. These insights help in distributing COVID-19 vaccinations in a way that maximizes their impact and reduces inequalities among lower-income countries. The approach recognizes the interdependence of global health and emphasizes the need for coordinated endeavors in addressing the pandemic. This mathematical model is a useful tool in the ongoing efforts to distribute COVID-19 vaccinations fairly and efficiently. It contributes to the worldwide aim of obtaining universal immunity and overcoming the problems posed by the epidemic in lower-icnome countries.

This paper proposes a linear programming approach for onboard optimization of rocket trajectories, motivated by the need for reliable and computationally efficient trajectory optimization methods in real-time applications. To reformulate rocket trajectory optimization problems within a linear programming framework, a polytopal approximation method is developed. The polytopal approximation method approximates the thrust magnitude constraint using a set of artificial variables and linear constraints. Compared with a state-of-the-art approximation method, the polytopal approximation method demonstrates a higher accuracy-to-size ratio, enabling rocket trajectory optimization problems to be cast into linear subproblems without significantly increasing problem sizes. As linear subproblems can be readily solved by linear programming solvers, the proposed trajectory optimization approach inherits the solver’s reliability and computational efficiency, making it potentially useful for critical onboard applications. An ascent trajectory optimization problem is provided as an illustrative example to demonstrate the effectiveness of the proposed linear programming approach.

For many of us, modern-day linear programming (LP) started with the work of George Dantzig in 1947. However, it must be said that many other scientists have also made seminal contributions to the subject, and some would argue that the origins of LP predate Dantzig’s contribution. It is matter open to debate [36]. However, what is not open to debate is Dantzig’s key contribution to LP computation. In contrast to the economists of his time, Dantzig viewed LP not just as a qualitative tool in the analysis of economic phenomena, but as a method that could be used to compute actual answers to specific real-world problems. Consistent with that view, he proposed an algorithm for solving LPs, the simplex algorithm [12]. To this day the simplex algorithm remains a primary computational tool in linear and mixed-integer programming (MIP). In [11] it is reported that the first application of Dantzig’s simplex algorithm to the solution of a non-trivial LP was Laderman’s solution of a 21 constraint, 77 variable instance of the classical Stigler Diet Problem [41]. It is reported that the total computation time was 120 man-days! The first computer implementation of an at-least modestly general version of the simplex algorithm is reported to have been on the SEAC computer at the then National Bureau of Standards [25]. (There were apparently some slightly earlier implementations for dealing with models that were “triangular”, that is, where all the linear systems could be solved by simple addition and subtraction.) Orchard-Hays [35] reports that several small instances having as many as 10 constraints and 20 variables were solved with this implementation. The first systematic development of computer codes for the simplex algorithm began very shortly thereafter at the RAND Corporation in Santa Monica, California. Dantzig’s initial LP work occurred at the Air Force following

In statistical relational learning, one is concerned with inferring the most likely explanation (or world) that satisfies a given set of weighted constraints. The weight of a constraint signifies our confidence in the constraint, and the most likely world that explains a set of constraints is simply a satisfying assignment that maximizes the weights of satisfied constraints. The relational learning community has developed specialized solvers (e.g., Alchemy and Tuffy) for such weighted constraints independently of the work on SMT solvers in the verification community. In this paper, we show how to leverage SMT solvers to significantly improve the performance of relational solvers. Constraints associated with a weight of 1 (or 0) are called axioms because they must be satisfied (or violated) by the final assignment. Axioms can create difficulties for relational solvers. We isolate the burden of axioms to SMT solvers and only lazily pass information back to the relational solver. This information can either be a subset of the axioms, or even generalized axioms (similar to predicate generalization in verification). We implemented our algorithm in a tool called Soft-Cegar that out-performs state-of-the-art relational solvers Tuffy and Alchemy over four real-world applications. We hope this work opens the door for further collaboration between relational learning and SMT solvers.

A Dantzig–Wolfe decomposition algorithm for linear economic model predictive control of dynamically decoupled subsystems