A set‐based approach for hierarchical optimization problem using Bayesian active learning

This thesis aims at developing efficient mathematical methods for solving hierarchical dynamic optimization problems. The main motivation is to model processes in nature, for which there is evidence to assume that they run optimally. We describe models of such processes by optimal control problems (called optimal control models (OCMs)). However, an OCM typically includes unknown parameters that cannot be derived entirely on a theoretical basis, which is in particular the case for the cost function. Therefore, we develop parameter estimation techniques to estimate the unknowns in an OCM from observation data of the process. Mathematically, this leads to a hierarchical dynamic optimization problem with a parameter estimation problem on the upper level and an optimal control problem on the lower level. We focus on multi-stage equality and inequality constrained optimal control problems based on nonlinear ordinary differential equations. The main goal of this thesis is to derive numerically efficient mathematical methods for solving hierarchical dynamic optimization problems, and to use these methods to estimate parameters in high-dimensional OCMs from real-world measurement data. We develop parameter-dependent OCMs for the gait of cerebral palsy patients and able-bodied subjects. The unknown parameters in the OCMs are then estimated from real-world motion capture data provided by the Heidelberg MotionLab of the Orthopedic University Clinic Heidelberg by using the mathematical methods developed within this work. The main novelties and contributions of this thesis to the field of hierarchical dynamic optimization are summarized herein. - We establish a novel mathematical method, a so-called direct all-at-once approach, for solving hierarchical dynamic optimization problems based on the direct multiple shooting method and first-order optimality conditions. - Furthermore, we propose an efficient numerical algorithm for large-scale hierarchical dynamic optimization problems, which fully exploits the structures inherited from both the hierarchical setting and the discretization. - Pontryagin's maximum principle is used to analyze solution properties of hierarchical dynamic optimization problems like second-order optimality conditions of the lower-level problem. - In addition, we propose and discuss alternative methods for hierarchical dynamic optimization that are based on derivative-free optimization and a bundle approach. These methods keep the hierarchical problem setting and do not reformulate the lower-level problem using first-order optimality conditions. - We establish a novel lifting method for regularizing mathematical programs with complementarity constraints, which is discussed and numerically investigated by means of a well-known collection of benchmark problems. - Proofs of regularity and convergence results for sequential quadratic programming methods applied to lifted mathematical programs with complementarity constraints are provided. - Efficient state-of-the-art implementations of all mathematical methods derived in this thesis, as well as a benchmark collection of hierarchical dynamic optimization problems are presented. - High-dimensional optimal control gait models for cerebral palsy patients and able-bodied subjects are developed. The mathematical methods derived in this thesis are used to estimate the unknown model parameters from real-world motion capture data provided by the Heidelberg MotionLab of the Orthopedic University Clinic Heidelberg. The theoretical and practical results presented in this thesis can be considered an initial motivating step towards answering open questions in current medical research in fields like treatment planning, classification of gaits, or the evaluation of surgeries by means of hierarchical dynamic optimization.

The Analytic Target Cascading (ATC) is an effective method for solving hierarchical Multidisciplinary Design Optimization (MDO) problems. At the same time, this method suffers from poor convergence and low accuracy, which is caused by the inconsistency of system constraints. In this paper, a novel ATC method based on dynamic relaxation factor is proposed. The dynamic relaxation factor of consistency constraint is added in the system level and is adjusted by the deviation of the linking variables between the levels to ensure the feasible region of the design space. The effectiveness and accuracy of this method are verified by a mathematical example. This method is used to solve the lightweight problem of the trussed front part of the vehicle body frame based on active and passive safety to achieve the collaborative optimization of lightweight trussed frame, crash safety, and aerodynamic characteristics. The important value of the novel ATC method based on dynamic relaxation factor in engineering applications is proven.

Analytical Target Cascading method has been widely developed to solve hierarchical design optimization problems. In the Analytical Target Cascading method, a weighted-sum formulation has been commonly used to coordinate the inconsistency between design points and assigned targets in each level while minimizing the cost function. However, the choice of the weighting coefficients is very problem dependent and improper selections of the weights will lead to incorrect solutions. To avoid the problems associated with the weights, single objective functions in the hierarchical design optimization are formulated by a new Bounded Target Cascading method. Instead of point targets assigned for design variables in the Analytical Target Cascading method, bounded targets are introduced in the new method. The target bounds are obtained from the optimal solutions in each level while the response bounds are updated back to the system level. If the common variables exist, they are coordinated based on their sensitivities with respect to design variables. Finally, comparisons of the results from the proposed method and the weighted-sum Analytical Target Cascading are presented and discussed.

†‡ § ** In traditional structural optimization, the geometric properties of the product are optimized for a specific set of material properties. This paper seeks to extend the domain of product optimization to also include the material modeling parameters by treating the product and material as a hierarchical multilevel system that can be decomposed and solved using the analytical target cascading (ATC) method. Two material models are considered here. The low-fidelity model is based on a simple elastic-plastic constitutive law with linear hardening whereas the high-fidelity model is based on an internal-state-variable representation that is influenced by the microstructural features of the material. The ATC method is applied to product-material design optimization of a multi-cell, thin-walled tube for impact energy absorption and durability. The integrated product-material optimization problem is solved using both single and multilevel approaches with results compared.

The area of Multiparametric Optimization (MPO) solves problems that contain unknown problem data represented by parameters. The solutions map parameter values to optimal design and objective function values. In this paper, for the first time, MPO techniques are applied to improve and advance Multidisciplinary Design Optimization (MDO) to solve engineering problems with parameters. A multiparametric subgradient algorithm is proposed and applied to two MDO methods: Analytical Target Cascading (ATC) and Network Target Coordination (NTC). Numerical results on test problems show the proposed parametric ATC and NTC methods effectively solve parametric MDO problems and provide useful insights to designers. In addition, a novel Two-Stage ATC method is proposed to solve nonparametric MDO problems. In this new approach elements of the subproblems are treated as parameters and optimal design functions are constructed for each one. When the ATC loop is engaged, steps involving the lengthy optimization of subproblems are replaced with simple function evaluations.

In this paper, the hierarchical optimization problem of multi-player systems with matched uncertainties is investigated via adaptive dynamic programming. In the hierarchical optimization problem, there exist one leader and multiple followers, the leader chooses a policy in advance based on the actions of the followers, and the followers make optimal responses to the leader's policy. The optimal policies of the leader and the followers form the Stackelberg equilibrium. By designing appropriate value functions for the leader and the followers, the hierarchical optimization problem is formulated as a Stackelberg game and the robust stabilization problem is transformed into an optimal regulation problem. Moreover, the critic-only structure is established to obtain the approximate Stackelberg equilibrium. Theoretical analysis shows that the developed ADP-based robust control guarantees the closed-loop system to be asymptotically stable. Finally, simulation example is adopted to verify the effectiveness of the present scheme.

This paper studies a hierarchical optimization problem of scheduling n jobs on a serial-batching machine, in which two objective functions are maximum costs. By a hierarchical optimization problem, we mean the problem of optimizing the secondary criterion under the constraint that the primary criterion is optimized. A serial-batching machine is a machine that can handle up to b jobs in a batch and jobs in a batch start and complete respectively at the same time and the processing time of a batch is equal to the sum of the processing times of jobs in the batch. When a new batch starts, a constant setup time s occurs. We confine ourselves to the bounded model, where b < n. We present an O(n4)-time algorithm for this hierarchical optimization problem. For the special case where two objective functions are maximum lateness, we give an O(n3 log n)-time algorithm.

ATC provides a systematic approach in solving decomposed large scale systems that has solvable subsystems. However， complex engineering system usually has a high computational cost , which result in limiting real-life applications of ATC based on high-fidelity simulation models. To address these problems, this paper aims to develop an efficient approximation model building techniques under the analytical target cascading (ATC) framework, to reduce computational cost associated with multidisciplinary design optimization problems based on high-fidelity simulations. An approximation model building techniques is proposed: approximations in the subsystem level are based on variable-fidelity modeling (interaction of low- and high-fidelity models). The variable-fidelity modeling consists of computationally efficient simplified models (low-fidelity) and expensive detailed (high-fidelity) models. The effectiveness of the method for modeling under the ATC framework using variable-fidelity models is studied. Overall results show the methods introduced in this paper provide an effective way of improving computational efficiency of the ATC method based on variable-fidelity simulation models.

Comparative Frameworks for Risk Mitigation in Canal Networks

In this paper, an interactive trade-off sequential optimization method for multiobjective decision problem is integrated with the hierarchical optimization technique,and a hierarchical multiobjective optimization problem for large-scale time-delay system is studied. Decomposition of problem in decision and objective spaces is directly realized through incorporating the necessary conditions for the preferred solution with /spl epsiv/-constraint problem, and a hierarchical multiobjective optimization algorithm for large-scale time-delay system is developed.It can be considered as an extension of the time-delay algorithm of Tamura to multiobjective case. Finally,the algorithm proposed is applied to ten years educational planning with three criteria and five -year time-delay. It is shown that the advantages of less computation time and fast convergence can be gained by using the proposed algorithm.

Under dynamic conditions, the dynamic force between the suspended equipment and the car body is substantially increased. This increase not only affects the ride comfort but also substantially raises the likelihood of fatigue damage to the suspended equipment. A multiobjective analytical target cascading (ATC) optimization is proposed to improve the ride comfort of high-speed trains and reduce the vibration of the suspended equipment. A mathematical simulation model of the vehicle equipment system is established, and the acceleration frequency-response function expression of the car body and the suspension system is obtained. The comfort index of the car body and the acceleration root mean square (RMS) of the suspended equipment are calculated by combining the German vertical irregular track spectrum and the comfort filter function. Meanwhile, the ATC method is used to optimize the car body comfort index and the acceleration RMS of the underframe suspended equipment. Then, the availability of the optimization method is evaluated via numerical simulation. Compared with the original suspension scheme of the underframe equipment, when the running speed of the vehicle is 300 km/h, the RMS value of the vibrational acceleration of the underframe equipment after optimization decreases by 19.9 %, and the ride comfort indexes at the car body center and above the front and rear bogies are improved by 6.4 %, 0.1 %, and 1.1 %, respectively. Simulation results demonstrate that ATC optimization can improve the railway vehicle ride comfort and reduce the vibration of the suspended equipment. This paper provides a new approach to the suspension parameter design of equipment.

A novel approach to find optimal real power generation pattern will be introduced in this study. This has been achieved by maximizing the power system voltage stability margin and minimizing generation cost. The loadability limit index is used to assess static voltage stability security margin. Based on proposed algorithm, a Toolbox has been developed to recognize the loadability limit by using Lagrangian method. The mentioned problem has been modeled as a non-linear and multi objective optimization problem along with Hybrid Genetic Algorithm Particle Swarm Optimization (HGAPSO) method to reach the optimal generation pattern. The method has been performed as a hierarchical optimization problem. In the first stage, HGAPSO method has been used to reach the securest pattern of real power. Also, mentioned voltage stability toolbox has been executed to evaluate the voltage stability margin respectively each real power generation pattern during HGAPSO search. These simulations are performed on IEEE 30 bus test system. To validation of proposed method, obtained results have been compared with Particle Swarm Optimization (PSO) results.

In this study, an efficient approach is proposed to find best real power generation participation for improving static voltage stability and reducing total generation cost. Therefore, loadability limit index is used to assess static voltage stability security margin, which is associated to the point of voltage collapse. Based on it, a toolbox is developed to recognize the loadability margin by using Lagrangian method. Finally, the mentioned problem is modeled as a non-linear and multiobjective optimization problem. In this paper, Hybrid Genetic Algorithm Particle Swarm Optimization (HGAPSO) and Particle Swarm Optimization (PSO) algorithms are applied to find out best generation direction. This problem is performed as a hierarchical optimization problem. In first stage, an algorithm is used to search possible solution area and reach tradeoff between secure and economic direction of real power generations. In second stage of proposed approach, respectively to each generation pattern, power system voltage stability margin and generation cost are evaluated during search algorithm. The simulations are performed on IEEE 14 and IEEE 30 bus test systems to find best generation direction.

Decomposition-based strategies, such as analytical target cascading (ATC), are often employed in design optimization of complex systems. Achieving convergence and computational efficiency in the coordination strategy that solves the partitioned problem is a key challenge. A new convergent strategy is proposed for ATC, which coordinates the interactions among subproblems using sequential lineralizations. Linearity of subproblems is maintained using L∞ norms to measure deviations between targets and responses. A subproblem suspension strategy is used to temporarily suspend inclusion of subproblems that do not need significant redesign, based on trust region and target value step size. The proposed strategy is intended for use in optimization problems where sequential linearizations are typically effective, such as problems with extensive monotonicities, large number of constraints relative to variables, and propagation of probabilities with normal distributions. Experiments with test problems show that, relative to standard ATC coordination, the number of subproblem evaluations is reduced considerably while maintaining accuracy.

This article focuses on the efficient multi-objective particle swarm optimization algorithm to solve multidisciplinary design optimization problems. The objective is to extend the formulation of collaborative optimization which has been widely used to solve single-objective optimization problems. To examine the proposed structure, racecar design problem is taken as an example of application for three objective functions. In addition, a fuzzy decision maker is applied to select the best solution along the pareto front based on the defined criteria. The results are compared to the traditional optimization, and collaborative optimization formulations that do not use multi-objective particle swarm optimization. It is shown that the integration of multi-objective particle swarm optimization into collaborative optimization provides an efficient framework for design and analysis of hierarchical multidisciplinary design optimization problems.