A Review on Bilevel Optimization: From Classical to Evolutionary Approaches and Applications

Combinatorial bi-level optimization remains a challenging topic, especially when the lower-level is an NP-hard problem. In this paper, we tackle large-scale and combinatorial bi-level problems using GP hyper-heuristics, i.e., an approach that permits to train heuristics like a machine learning model. Our contribution aims at targeting the intensive and complex lower-level optimizations that occur when solving a large-scale and combinatorial bi-level problem. For this purpose, we consider hyper-heuristics through heuristic generation. Using a GP hyper-heuristic approach, we train greedy heuristics in order to make them more reliable when encountering unseen lower-level instances that could be generated during bi-level optimization. To validate our approach referred to as GA+AGH, we tackle instances from the bi-level cloud pricing optimization problem (BCPOP) that model the trading interactions between a cloud service provider and cloud service customers. Numerical results demonstrate the abilities of the trained heuristics to cope with the inherent nested structure that makes bi-level optimization problems so hard. Furthermore, it has been shown that training heuristics for lower-level optimization permits to outperform human-based heuristics and metaheuristics which constitute an excellent outcome for bi-level optimization.

Presolving linear bilevel optimization problems

Uncertainties in variables and parameters cause optimization problems to move away from globally-optimal and uncertain solutions. Practitioners resort to finding robust and reliable solutions in such situations. Bilevel optimization problems involving a hierarchy of two nested optimization problems have received a growing attention in the recent past due to their relevance in practice. While a number of studies on bilevel solution methodologies and applications are available for a deterministic setup, but studies on uncertainties in bilevel optimization are rare. In this paper, we suggest methodologies for handling uncertainty in both lower and upper level variables that may occur from different practicalities. For the first time, we perform a systematic study demonstrating the effect of uncertainties in each level along with the definition of robustness and reliability in the context of bilevel optimization. The issues and complexities introduced due to such uncertainties are then studied through a number of test cases, for brevity, we only show results on three test cases. Finally, two real-world bilevel problems involving uncertainties in their variables are solved. The study provides foundations and demon- strates viable directions for further research in uncertainty-based bilevel optimization problems.

Linear bilevel programs with multiple objectives at the upper level

Bilevel problems are highly challenging optimization problems that appear in many applications of energy market design, critical infrastructure defense, transportation, pricing, and so on. Often these bilevel models are equipped with integer decisions, which makes the problems even harder to solve. Typically, in such a setting in mathematical optimization, one develops primal heuristics in order to obtain feasible points of good quality quickly or to enhance the search process of exact global methods. However, there are comparably few heuristics for bilevel problems. In this paper, we develop such a primal heuristic for bilevel problems with a mixed-integer linear or quadratic upper level and a linear or quadratic lower level. The heuristic is based on a penalty alternating direction method, which allows for a theoretical analysis. We derive a convergence theory stating that the method converges to a stationary point of an equivalent single-level reformulation of the bilevel problem and extensively test the method on a test set of more than 2,800 instances—which is one of the largest computational test sets ever used in bilevel programming. The study illustrates the very good performance of the proposed method in terms of both running times and solution quality. This renders the method a suitable subroutine in global bilevel solvers as well as a reasonable standalone approach.Summary of Contribution: Bilevel optimization problems form a very important class of optimization problems in the field of operations research, which is mainly due to their capability of modeling hierarchical decision processes. However, real-world bilevel problems are usually very hard to solve—especially in the case in which additional mixed-integer aspects are included in the modeling. Hence, the development of fast and reliable primal heuristics for this class of problems is very important. This paper presents such a method.

Bilevel optimization problems require every feasible upper-level solution to satisfy optimality of a lower-level optimization problem. These problems commonly appear in many practical problem solving tasks including optimal control, process optimization, game-playing strategy development, transportation problems, and others. In the context of a bilevel single objective problem, there exists a number of theoretical, numerical, and evolutionary optimization results. However, there does not exist too many studies in the context of having multiple objectives in each level of a bilevel optimization problem. In this paper, we address bilevel multi-objective optimization issues and propose a viable algorithm based on evolutionary multi-objective optimization (EMO) principles. Proof-of-principle simulation results bring out the challenges in solving such problems and demonstrate the viability of the proposed EMO technique for solving such problems. This paper scratches the surface of EMO-based solution methodologies for bilevel multi-objective optimization problems and should motivate other EMO researchers to engage more into this important optimization task of practical importance.

Multilevel optimization problems often arise in various applications (in economics, ecology, power engineering and other areas) when modeling complex systems with a hierarchical structure associated with independent actions of subsystems. The difficulty of analyzing such complex systems requires first of all the study of bilevel models, the management of which would be an integral part of the analysis of more complex systems. In solving bilevel programming problems, an important role is played by the property of partial calmness, the presence of which allows us to reduce the bilevel problem to the classical nonlinear programming problem with a nonsmooth objective function. It is known that linear bilevel programming problems are partially stable. The proof of this property for more complex problems meets difficulties. In particular, our article shows the inaccuracy of some results in this area. The goal of the paper is to obtain some new results in the partial calmness of bilevel programming. In particular, new sufficient conditions for bilevel problems are proved. The results are obtained on the base of Lipschitz-like properties for multivalued mappings. In the paper we propose new sufficient conditions for partial calmness which are based on some modification of the known constraint qualification RCPLD which have been proposed by the researches Andreani, Haeser, Schuverdt and Silva.

In this study, we consider solution approaches used to solve the proposed bilevel linear programming model for an Industrial Symbiosis network. We first solve this model with the well-known Karush-Kuch-Tucker (KKT) approach. However, transforming a bilevel programming model with the KKT approach increases the number of variables and constraints. For this reason, we use the fuzzy programming approach and the fuzzy goal programming approaches as alternatives to the KKT approach. Next, we compare the results of the KKT approach with these methods and examine the suitability of these approaches to solve our bilevel problem. Unlike previous studies, which claimed that fuzzy approaches mostly outperform the KKT approach, in our case, the best solution is obtained with the KKT approach. This is most probably because these approaches ignore the hierarchical nature of the problem. We believe that more research on fuzzy approaches is needed to evaluate the suitability of these approaches for solving bilevel programming problems.KeywordsIndustrial symbiosisBilevel programmingFuzzy programming approachFuzzy goal programming

This paper addresses a bilevel optimization problem with a guaranteed (pessimistic) solution in a rather general form. The problem is reduced to a family of bilevel problems with optimistic solutions and perturbed follower's problems. For the quadratic-linear bilevel optimization problem, a reduction of the bilevel problem to a series of non-convex one-level optimization problems is proposed using the penalty method. Additionally, a solution method based on Global Search Theory (GST) is developed for the obtained problem. The GST consists of two stages: special local search and Global Search Scheme (GSS), which is based on Global Optimality Conditions (GOCs). The computational testing of the developed approach on specially constructed bilevel problems of high dimensions was successfully carried out. Finally, the computational testing on specially constructed bilevel problems of rather high dimensions was successfully carried out, that substantiate the developed approach.

A multi-objective Bi-level leader-follower joint optimization for concurrent design of product family and assembly system

Image segmentation is mainly used as a preprocessing step in problems of image processing and computer vision. Its performance has a great influence on subsequent tasks. Evolutionary Computation (EC) techniques have been introduced to the area of image segmentation due to their high search capacity. However, there are rarely comprehensive surveys on EC based image segmentation methods, which can enable researchers to get a quick understanding of this area and compare the existing methods. Therefore, this paper provides an overview of EC based image segmentation methods, and discusses the remaining issues in this area. It is observed that among all EC techniques, four of them (genetic algorithms, genetic programming, differential equation and partial swarm optimization) are more frequently used and GAs are the most popular technique. It is noted that low generalization capacity and computational complexity are two common problems in EC techniques applied to image segmentation.KeywordsEvolutionary Computer VisionImage SegmentationEvolutionary ComputationGenetic AlgorithmsGenetic Programming

Challenging multi-modal optimization problems have been very successfully solved by evolutionary computation (EC) techniques. To date, many methods have been proposed on evolutionary optimization for both single and multiobjective large scale problems. In the age of Big Data, there is an urge to take evolutionary optimization techniques to the next level for solving problems with even larger scales: thousands and millions of variables. These problems arise in many domains ranging from bioinformatics, to neuroscience and social simulations. In this paper, we investigate the use of EC to solve Big electroencephalography (EEG) data optimization problems with thousands of variables. The optimization problem attempts to identify maximum information that should be kept from a signal while minimizing the artifact. The high level of epistasis inherent in a signal can slow down the evolution. Therefore, we investigate the advantages of optimizing the problem in the frequency domain with different thresholds as opposed to the time domain. We propose synthetic EEG data sets of various scale and noise level. These data sets were the basis for the Optimization of Big Data 2015 Competition (BigOpt), CEC 2015. Two state-of-art multiobjective evolutionary algorithms (MOEAs) were evaluated. The results of this work suggest that frequency representation of the signals facilitates dimensionality reduction for big scale optimization of time series data, and hence provides faster and better quality solutions for EEG data cleaning. Moreover, the results suggest that existing state-of-art multiobjective evolutionary computation methods are extremely slow. Methods that can optimize the problem faster and with high quality are needed.

Electronic countermeasure (ECM) has become one of the most significant factors in modern warfare, in the course of combat, the electronic jamming allocation tasks need to be flexibly adjusted with the change of combat stage, which puts forward higher requirements for the modeling and solution method of this kind of problems. To solve the ECM dynamic weapon target assignment (ECM-DWTA) problem, a hybrid multi-target bi-level programming model is established. The upper level takes the sum of the electronic jamming effects in the whole combat stage as an optimization objective, and locally optimizes the ECM weapon (ECM-WP) assignment scheme in each stage. The lower level takes the importance expectation value of the target subjected to interference and combat consumption as double optimization objectives to globally optimize the ECM-WP assignment scheme. Focus on solving this complex model, a hybrid multi-objective bi-level interactive fuzzy programming algorithm (HMOBIF) is proposed, in this method, exponential membership function is used to describe the satisfaction degree of each level. When solving the multi-objective optimization problem composed of membership functions in the upper and lower levels, we use the MOEA/D algorithm to obtain the Pareto Front (PF) solution set, and then each solution in PF is evaluated and selected by the TOPSIS multi-criteria evaluation method. This local and global interactive optimization process of bi-level model is actually the process of executing observation-orientation-decision-action loop in practical combat. According to the current example, we conduct numerical simulation on the parameters in the model and obtain the parameter values suitable for the model solution. The computational experiments on different scale ECM-DWTA problems show that HMOBIF method is superior to four bi-level programming algorithms in terms of performance index, and can better solve ECM-DWTA problems.

The paper deals with semivectorial bilevel optimization problems. The upper level is a scalar optimization problem to be solved by the leader, and the lower level is a multiobjective optimization problem to be solved by several followers acting in a cooperative way inside the greatest coalition, so choosing among Pareto solutions. In the so-called “optimistic problem”, the followers choose among their best responses (i.e. Pareto solutions) one which is the most favorable for the leader. The opposite is the “pessimistic problem”, when there is no cooperation between the leader and the followers, and the followers choice among their best responses may be the worst for the leader. The paper presents a general method which allows, under certain mild hypotheses, to transform a semivectorial bilevel problem into an ordinary bilevel optimization. Some applications are given.

This paper presents an optimal bidding strategy for a strategic wind power producer (WPP) in a distribution-level energy market (DLEM). The behavior of the WPP is modelled through a bi-level stochastic optimization problem where the upper-level problem maximizes the profit of the WPP and the lower-level problem describes the clearing processes of the DLEM while considering network constraints. The bi-level problem is a stochastic mathematical program with equilibrium constraints (MPEC) that is formulated as a mixed-integer linear programming (MILP) problem. The main focus of this study is investigating prosumers’ impact on the market power of the strategic WPP in a DLEM structure. In this model, the effect of flexible prosumers from the aspects of demand response (DR) participants and photovoltaic penetration level (PVPL) on the WPP’s offering strategy is investigated. Moreover, the impact of bilateral contract on the market power of the strategic WPP and the cleared prices of the network is addressed. The proposed model is implemented in an IEEE 33-bus and numerical results illustrate how behavior of flexible prosumers and PVPL index affect the decision making of the strategic WPP when network constraints are considered. Numerical results show that by active participation of prosumers in DR programs, the reliance of DLEM on the strategic WPP reduces. Moreover, if the WPP participates in bilateral contracts, its offering to the DELM decreases, and as the result, the cleared prices augment indicating market power of the WPP.