Methods for finding feasible points in constrained optimization

We present a stochastic approximation algorithm based on penalty function method and a simultaneous perturbation gradient estimate for solving stochastic optimisation problems with general inequality constraints. We present a general convergence result that applies to a class of penalty functions including the quadratic penalty function, the augmented Lagrangian, and the absolute penalty function. We also establish an asymptotic normality result for the algorithm with smooth penalty functions under minor assumptions. Numerical results are given to compare the performance of the proposed algorithm with different penalty functions.

Suspended sediment dynamics play an important role in controlling nearshore and estuarine geomorphology and the associated ecological environments. Modeling the transport of suspended sediment is a complicated and challenging research topic. The goal of this study is to improve the accuracy of modeling the suspended sediment concentrations (SSCs) with newly developed techniques. Based on a three-dimensional suspended cohesive sediment transport model, the transport of suspended sediment and SSCs are simulated by assimilating SSCs retrieved from the Geostationary Ocean Color Imager (GOCI) with the adjoint data assimilation in the Hangzhou Bay, a typical strong tidal estuary along the coast of the East China Sea. To improve the effect of the data assimilation, the penalty function method, in which the reasonable constraints of the estimated model parameters are added to the cost function as penalty terms, will be introduced for the first time into the adjoint data assimilation in the SSCs modeling. In twin experiments, the prescribed spatially varying settling velocity is estimated by assimilating the synthetic SSC observations, and the results show that the penalty function method can further improve the effect of data assimilation and parameter estimation, regardless of synthetic SSC observations being contaminated by random artificial errors. In practical experiments, the spatially varying settling velocity is firstly estimated by assimilating the actual GOCI-retrieved SSCs. The results demonstrate that the simulated results can be improved by the adjoint data assimilation, and the penalty function method can additionally reduce the mean absolute error (MAE) between the independent check observations and the corresponding simulated SSCs from 1.44 × 10−1 kg/m3 to 1.30 × 10−1 kg/m3. To pursue greater simulation accuracy, the spatially varying settling velocity, resuspension rate, critical shear stress and initial conditions are simultaneously estimated by assimilating the actual GOCI-retrieved SSCs to simulate the SSCs in the Hangzhou Bay. When the adjoint data assimilation and the penalty function method are simultaneously used, the MAE between the independent check observations and the corresponding simulated SSCs is just 9.90 × 10−2 kg/m3, which is substantially less than that when only the settling velocity is estimated. The MAE is also considerably less than that when the four model parameters are estimated to be without using the penalty function method. This study indicates that the adjoint data assimilation can effectively improve the SSC simulation accuracy, and the penalty function method can limit the variation range of the estimated model parameters to further improve the effect of data assimilation and parameter estimation.

The non-monotonic reasoning system based on penalty function was originally introduced by Pinkas in 1995. The main idea of this method is to get unknown knowledge via the voting method according to some known knowledge. However, the penalty function method, when applied to the problem of intelligent route planning, cannot effectively reflect human intelligence. A new voting principle and a new method to construct its penalty function are presented. Experimental results show that, in some real applications, our new penalty function is more reasonable and effective than that of Pinkas.

When we have prior information of approximate and subjective nature other than those contained in the time series, that is knowledge not directly referable to exact linear constraints, we propose to use the penalty function method to tackle the problem of estimation of a linear statistical model alternatively to the mixed estimation proposed by Theil and Goldberger. This method bases itself on the minimization of a mathematical function known as penalty function where constraint expressed in a quadratic form is linearly combined with an objective function by some coefficients known as penalty parameters. The particular functional form utilized for the constraint can be seen as a way to catch all the uncertainty included in this type of prior knowledge. The algorithm that derives from the application of this method depends on the penalty parameters that can be interpreted in two different ways: a) as weights assigned to the constraint in the minimization procedure. In this case the result can be seen as a generalization of the ridge estimator. b) as variances of a stochastic constraint. By this way the algorithm is similar to the mixed estimator but, unlike this one, it has not unknown parameters in its formulation and so it can be seen as a feasible estimator. Whatever the interpretation of this parameters is, the penalty function method produces an algorithm that is a flexible instrument both from a theoretical viewpoint and a practical one.

In this work, the reliability analysis is employed to take into account the uncertainties in a structure. Reliability analysis is a tool to compute the probability of failure corresponding to a given failure mode. In this study, one of the most commonly used reliability analysis method namely first order reliability method is used to calculate the probability of failure. Since finding the most probable point (MPP) or design point is a constrained optimization problem, in contrast to all the previous studies based on the penalty function method or the preference of the feasible solutions technique, in this study one of the latest versions of the differential evolution metaheuristic algorithm named improved (μ+λ)-constrained differential evolution (ICDE) based on the multi-objective constraint-handling technique is utilized. The ICDE is very easy to implement because there is no need to the time-consuming task of fine tuning of the penalty parameters. Several test problems are used to verify the accuracy and efficiency of the ICDE. The statistical comparisons revealed that the performance of ICDE is better than or comparable with the other considered methods. Also, it shows acceptable convergence rate in the process of finding the design point. According to the results and easier implementation of ICDE, it can be expected that the proposed method would become a robust alternative to the penalty function based methods for the reliability assessment problems in the future works.

Artificial Searching Swarm Algorithm (ASSA) is a novel optimization algorithm. This paper presents the comparison results on the performance of the ASSA for solving constrained optimization problems. The penalty function method and non-parameter penalty method are applied to a set of constrained problem. The simulation results show that ASSA is an efficient algorithm for constrained optimization problems.

A two-phase constraint-handling technique is integrated into the evolutionary algorithms to solve constrained optimization problems (called TPDE) in this article. In phase one, denoted as the exploration phase, an exterior penalty function method with the dynamic penalty coefficients is developed to compare any two candidate solutions, which aims to push the population into the feasible region. To reduce the computational burden, in phase two, denoted as the exploitation phase, an interior penalty function method with the dynamic penalty coefficients is developed, which enhances the search ability by using the information of constraints in feasible solutions. During the optimization process, differential evolution is adopted as the search algorithm to produce the offspring population. Experiment results on four benchmark test suites, namely, IEEE CEC 2006, IEEE CEC 2010, IEEE CEC 2017, and IEEE CEC 2020, indicate that TPDE is competitive with other popular algorithms.

Due to significant improvements in the performance of computers, Genetic Algorithms are actively applied to actual engineering problems. Most engineering problems are constrained optimization problems that are used to optimize objective functions under many constraints. The penalty method is well-known solution to such constrained optimization problems. However, the benchmarks of constrained optimization problems have only a small number of constraints. Thus, the effectiveness of the penalty method has not been investigated in numerous constrained problems. In many constrained optimization problems, penalty method has a risk whereby all constraints cannot be satisfied. This paper proposes the stepwise satisfaction method of constraints to satisfy conditions that involve many constraints. In the proposed method, the priority of constraints to be satisfied is defined based on the initial population and the objective functions are optimized after the satisfaction of all constraints. Furthermore, this paper studies the effects of classifying constraints into difficult and easy ones as well as combining the proposed method with the penalty method. In the experiment, the performance of the proposed method and the penalty method was compared in two problems with more than 50 constraints.

The reservoir flood control operation problem has the characteristics of multiconstraint, high-dimension, nonlinearity, and being difficult to solve. In order to better solve this problem, this paper proposes an improved bald eagle search algorithm (CABES) coupled with ε-constraint method (ε-CABES). In order to test the performance of the CABES algorithm, a typical test function is used to simulate and verify CABES. The results are compared with the bald eagle algorithm and particle swarm optimization algorithm to verify its superiority. In order to further test the rationality and effectiveness of the CABES method, two single reservoirs and a multi-reservoir system are selected for flood control operation, and the ε constraint method and the penalty function method (CF-CABES) are compared, respectively. Results show that peak clipping rates of ε-CABES and CF-CABES are both 60.28% for Shafan Reservoir and 52.03% for Dahuofang Reservoir, respectively. When solving the multi-reservoir joint flood control operation system, only ε-CABES flood control operation is successful, and the peak clipping rate is 51.76%. Therefore, in the single-reservoir flood control operation, the penalty function method and the ε constraint method have similar effects. However, in multi-reservoir operation, the ε constraint method is better than the penalty function method. In summary, the ε-CABES algorithm is more reliable and effective, which provides a new method for solving the joint flood control scheduling problem of large reservoirs.

Constrained engineering design optimization problems are usually computationally expensive due to non-linearity and non convexity of the constraint functions. Penalty function methods are found to be quite popular due to their simplicity and ease of implementation, but they require an appropriate value of the penalty parameter. Bi-objective approach is one of the methods to handle constraints, in which the minimization of the constraint violation is included as an additional objective. In this paper, constrained engineering design optimization problems are solved by combining the penalty function approach with a bi-objective evolutionary approach which play complementary roles to help each other. The penalty parameter is approximated using bi-objective approach and a classical method is used for the solution of unconstrained penalized function. In this methodology, we have also eliminated the local search parameter which was needed in our previous study.

This study focuses on developing a multi-event simultaneous departure routing for the Amsterdam Airport Schiphol. The aircraft ground tracks are shaped by rational Bezier curve, and the trajectory optimization problem is reduced to a constrained parametric optimization formulation with few unknown optimizers. In order to handle the constraints, the continuous tracks are discretized into a series of data points and approximated by piecewise linear curves. A data-points selection strategy is introduced to improve the calculation efficiency and guarantee the feasibility of the optimized track. The original constrained parameter optimization problem is transformed to an unconstrained one using a penalty function method and subsequently solved by a genetic algorithm and pattern search method. The details of penalty function formulation are given in this study. The feasibility of the proposed method is demonstrated in numerical examples. The results show that both the number of people highly annoyed and track distance are decreased in simultaneous optimization relative to a scenario where departure routes are optimized in isolation.

The problem of optimizing an absorber system for three-dimensional seismic structures is addressed. The objective is to determine the number and position of absorbers to minimize the coupling effects of translation-torsion of structures at minimum cost. A procedure for a multi-objective optimization problem is developed by integrating a dominance-based selection operator and a dominance-based penalty function method. Based on the two-branch tournament genetic algorithm, the selection operator is constructed by evaluating individuals according to their dominance in one run. The technique guarantees the better performing individual winning its competition, provides a slight selection pressure toward individuals and maintains diversity in the population. Moreover, due to the evaluation for individuals in each generation being finished in one run, less computational effort is taken. Penalty function methods are generally used to transform a constrained optimization problem into an unconstrained one. The dominance-based penalty function contains necessary information on non-dominated character and infeasible position of an individual, essential for success in seeking a Pareto optimal set. The proposed approach is used to obtain a set of non-dominated designs for a six-storey three-dimensional building with shape memory alloy dampers subjected to earthquake.

Two algorithms for the local energy minimization of the structure of macromolecules in the presence of constraints are proposed. They are a combination of the method of steepest descents and the method of conjugate gradients with the procedure SHAKE, by which distance constraints can be satisfied. The two algorithms are tested by applying them to a small protein, the bovine pancreatic trypsin inhibitor (BPTI), and compared with the penalty function method for conserving constraints. The efficiency of the proposed methods depends on the level of interdependence of the constraints. For bond‐length constraints, the use of SHAKE is superior to the penalty function method. However, when bond‐angle constraints are included, SHAKE is more efficient only if the curvature of the penalty function is considerably greater than that of the potential function being minimized. The results indicate that with bond‐length constraints the minimization behavior is similar to that without constraints. However, the simultaneous application of bond‐length and bond‐angle constraints appears to confine the molecule to a very limited part of configuration space, very different from the part covered by an unconstrained minimization. This conclusion calls into question energy minimizations of protein systems in which only the dihedral angles are allowed to vary.

Adaptive beamforming (ABF) methods typically yield beam patterns that contain regions of high sidelobe gain. This can become a major problem when the platform is mobile with respect to noise sources. Real world ABF systems demand some sort of sidelobe control or stabilisation - one such approach is via penalty function methods. Computational speed is an important issue and their exists a requirement for computationally efficient methods of calculating penalty functions. One such method is described in this paper.

Penalty function methods have been widely used for handling constraints, but it’s still a challenge about how to set the penalty parameter effectively though many related methods have been proposed. In this paper, the penalty parameter is firstly analyzed systematically by introducing four rules. Based on this analysis, a new Dynamic Penalty Function (DyPF) is proposed by adjusting penalty parameter in three different situations during the evolution (i.e., the infeasible situation, the semi-feasible situation, and the feasible situation). The experiments are designed to verify the effectiveness of our newly proposed DyPF. The results show that DyPF presents a better overall performance than other five dynamic or adaptive state-of-the-art methods in the community of constrained evolutionary optimization.