- Book Chapter
1
- 10.1016/b978-0-444-81844-7.50021-8
ADAPTIVE MODELING FOR BIT RATE CONTROL
- Jan 01, 1994
- Signal Processing of HDTV, V
- Limin Wang + 2 more +2
ADAPTIVE MODELING FOR BIT RATE CONTROL
In this paper we consider two alternative choices for the factor used to scale the initial Hessian approximation, before updating by a member of the Broyden family of updates for quasi-Newton optimization methods. By extensive computational experiments carried out on a set of standard test problems from the CUTE collection, using efficient implemen-tations of the quasi-Newton method, we show that the proposed new scaling factors are better, in terms of efficiency achieved (number of iterations, number of function and gradient evaluations), than the standard choice proposed in the literature
ADAPTIVE MODELING FOR BIT RATE CONTROL
ADAPTIVE MODELING FOR BIT RATE CONTROL
Investigating the Multi-memetic Mind Evolutionary Computation Algorithm Efficiency
При решении практически значимых задач глобальной оптимизации целевая функция зачастую имеет высокую размерность и вычислительную сложность, а также нетривиальный ландшафт. Исследования показывают, что зачастую одного метода оптимизации недостаточно для эффективного решения такого рода задач – необходима гибридизация нескольких методов оптимизации.
Read more����� ������-�������-����� ��� ���I�I���� ����� ��������� �������I� �� ������������ ���� �I�����
There are a lot of dierent physical processes modeling of which lead us to a nonlinear least squares problems. Usually, they arise in cases of modeling or evaluating physical processes by values obtained as a result of measurements. In this article, we consider problems in which the nonlinear function consists of a dierentiable and nondierentiable part. For numerical solving such nonlinear least-squares problems, we proposed the Gauss-NewtonPotra method. Since the full Jacobian does not exist, this method uses instead of Jacobian the sum of the derivative of the dierentiable part of the operator and the combination of rst-order divided dierences of the nondierentiable part of the operator. The local convergence analysis of the method is conducted under the generalized Lipschitz conditions for rst-order derivatives and rst- and second-order divided dierences. These conditions, instead of Lipschitz constant, contain some positive integrable function. The classical Lipschitz conditions are a partial case of the generalized Lipschitz conditions. An equation for the radius of convergence domain and error estimates were obtained. A convergence order of the method in case of zero residual was established. We carry out numerical experiments on a set of test problems and compare results with other known methods by the number of iterations. The obtained results show that the Gauss-Newton-Potra method is more ecient than iterative-dierence methods.
Read moreAncestral Benders' Cuts and Multi-term Disjunctions for Mixed-Integer Recourse Decisions in Stochastic Programming
This paper focuses on solving two-stage stochastic mixed integer programs (SMIPs) with general mixed integer decision variables in both stages. We develop a decomposition algorithm in which the first stage approximation is solved using a branch-and-bound tree with nodes inheriting Benders' cuts that are valid for their ancestor nodes. In addition, we develop two closely related convexification schemes which use multi-term disjunctive cuts to obtain approximations of the second stage mixed-integer programs. We prove that the proposed methods are finitely convergent. One of the main advantages of our decomposition scheme is that we use a Benders-based branch-and-cut approach in which linear programming approximations are strengthened sequentially. Moreover as in many decomposition schemes, these subproblems can be solved in parallel. We also illustrate these algorithms using several variants of an SMIP example from the literature, and present preliminary evidence of the scalability of these algorithms as the number of scenarios increases.
Read moreA parallel unconstrained quasi-Newton algorithm and its performance on a local memory parallel computer
A parallel unconstrained quasi-Newton algorithm and its performance on a local memory parallel computer
A Newton-type method for solving an inverse Sturm-Liouville problem
A Newton-type method is proposed for the recovery of the unknown coefficient function in the canonical Sturm-Liouville differential equation from two spectral data. Specifically, the two spectral data will be used to produce two Cauchy data, which in turn will serve as the input in a non-linear equation whose unknown is the coefficient function in the canonical Sturm-Liouville differential equation. This non-linear equation is to be solved numerically by the Newton method. Each Newton iterate requires that a Goursat-Cauchy boundary value problem be solved numerically. The Fréchet differentiability of the non-linear map is also discussed in the present paper. The numerical implementation of the Newton method for this inverse Sturm-Liouville problem is illustrated with examples, and it will be compared with a quasi-Newton method, and with a variational method discussed in previous literature. The numerical examples show that the Newton method needs fewer iterates to recover the true coefficient function than the quasi-Newton method. The Newton method is comparable with the variational method in terms of accuracy and number of iterates when the boundary parameters are given, and it requires a much smaller number of iterates than the variational method, when the boundary parameters are reconstructed from the available spectra.
Read moreEquality-Constrained Engineering Design Optimization Using a Novel Inexact Quasi-Newton Method
For engineering design optimization, the full-space formulation offers the potential for greater efficiency than the more commonly used reduced-space formulation. This potential is greater when the numerical model involves discretized partial differential equations or coupled disciplines. However, the full-space formulation results in a larger optimization problem with at least a factor of two increase in the number of optimization variables and equality constraints. Using Newton-type methods to solve such problems involves solving a large-scale and, often, ill-conditioned Karush–Kuhn–Tucker linear system at each optimization iteration. This can be time-consuming to solve even with a Krylov solver. If the number of iterations is reduced, the full-space formulation could be applied to a broader class of problems. This paper presents an inexact quasi-Newton algorithm with an adaptive extension for solving large-scale equality-constrained optimization problems. The new algorithm inexactly solves the Karush–Kuhn–Tucker system using new inexactness criteria that are derived to ensure a descent direction. The adaptive extension chooses the stopping condition of the Krylov solver by also taking its convergence rate into account. The paper presents results of numerical experiments applying this algorithm to three types of problems: six constrained optimization problems from the widely used CUTEst test suite, a bar thickness optimization problem, and a two-dimensional topology optimization problem. For all problems, the new algorithm consistently shows a roughly 50% reduction in the total number of Krylov solver iterations and a minimum of roughly 15% reduction in the optimization time. Moreover, the proposed approach for selecting the Krylov solver tolerance shows an improvement in all cases, whereas the existing forcing-parameter approach shows an increase in the number of Krylov iterations in some cases. These results indicate that this new method for selecting solver tolerances is effective and robust, and a good choice in algorithms that use a Krylov solver for solving the Karush–Kuhn–Tucker linear system.
Read moreStatistical Quasi-Newton: A New Look at Least Change
A new method for quasi-Newton minimization outperforms BFGS by combining least-change updates of the Hessian with step sizes estimated from a Wishart model of uncertainty. The Hessian update is in the Broyden family but uses a negative parameter, outside the convex range, that is usually regarded as the safe zone for Broyden updates. Although full Newton steps based on this update tend to be too long, excellent performance is obtained with shorter steps estimated from the Wishart model. In numerical comparisons to BFGS the new statistical quasi-Newton (SQN) algorithm typically converges with about 25% fewer iterations, functions, and gradient evaluations on the top 1/3 hardest unconstrained problems in the CUTE library. Typical improvement on the 1/3 easiest problems is about 5%. The framework used to derive SQN provides a simple way to understand differences among various Broyden updates such as BFGS and DFP and shows that these methods do not preserve accuracy of the Hessian, in a certain sense, while the new method does. In fact, BFGS, DFP, and all other updates with nonnegative Broyden parameters tend to inflate Hessian estimates, and this accounts for their observed propensity to correct eigenvalues that are too small more readily than eigenvalues that are too large. Numerical results on three new test functions validate these conclusions.
Read moreMulti-directional parallel algorithms for unconstrained optimization
Parallel algorithms for solving unconstrained nonlinear optimization problems are presented. These algorithms are based on the quasi-Newton methods. At each step of the algorithms, several search directions are generated in parallel using various quasi-Newton updates. Our numerical results show significant improvement in the number of iterations and function evaluations required by the parallel algorithms over those required by the serial quasi-Newton updates such as the SR1 method or the BFGS method for many of the test problems.
Read moreDescent Symmetrization of the Dai–Liao Conjugate Gradient Method
Symmetrizing the Dai–Liao (DL) search direction matrix by a rank-one modification, we propose a one-parameter class of nonlinear conjugate gradient (CG) methods which includes the memoryless Broyden–Fletcher–Goldfarb–Shanno (MLBFGS) quasi-Newton updating formula. Then, conducting an eigenvalue analysis, we suggest two choices for the parameter of the proposed class of CG methods which simultaneously guarantee the descent property and well-conditioning of the search direction matrix. A global convergence analysis is made for uniformly convex objective functions. Computational experiments are done on a set of unconstrained optimization test problems of the CUTEr collection. Results of numerical comparisons made by the Dolan–Moré performance profile show that proper choices for the mentioned parameter may lead to promising computational performances.
Read morePreconditioning on subspace quasi-Newton method for large scale unconstrained optimization
Recently, subspace quasi-Newton (SQN) method has been widely used in solving large scale unconstrained optimization. Besides constructing sub-problems in low dimensions so that the storage requirement as well as computational cost can be reduced, it can also be implemented extremely fast when the objective function is a combination of computationally cheap non-linear functions. However, the main deficiency of SQN method is that it can be very slow on certain type of non-linear problem. Hence, a preconditioner which is computationally cheap and is a good approximation to the actual Hessian is constructed to speed up the convergence of the quasi-Newton methods since the evaluation of the actual Hessian is considered as impractical and costly. For this purpose, a diagonal updating matrix has been derived to replace the identity matrix in approximating the initial inverse Hessian. The numerical results show that the preconditioned SQN method performs better than the standard SQN method that without preconditioning.
Read moreFast CORDIC Algorithm Based on a New Recoding Scheme for Rotation Angles and Variable Scale Factors
This work proposes a new rotation mode CORDIC algorithm, which considerably reduces the iteration number. It is achieved by combining several design techniques. Particularly, a new table-lookup recoding scheme for rotation angles and variable scale factors is developed to reduce the iteration numbers for rotation and scale factor compensation. By addressing the MSB parts of the residual rotation angles to a lookup table, two micro rotation angles are retrieved that in combination best matches the MSB parts. We also combine the leading-one bit detection operations for residual rotation angles, to skip unnecessary rotations. The resulting problems of variable scale factors are then solved by our previous fast decomposition and compensation algorithm (C.C. Li and S.G. Chen, in Proceedings of 1996 IEEE International Symposium Circuits and Systems, May 1996, Atlanta, USA, pp. 264–267s C.C. Li and S.G. Chen, in Proceedings of 1997 IEEE International Conference on Acoustic, Speech and Signal Processing, Munich, 1997, Germany, pp. 639–642). To further reduce the iteration number of scale factor compensation, we again apply the mentioned residual recoding technique and the leading-one bit detection scheme to the fast variable scale factor algorithm. Those techniques collectively reduce the iteration number significantly. Simulations show that in average the new design needs only 9.78 iterations to generate results with 22-bit accuracy, including all the iterations for rotations and scale factor compensations. Statistically, the total iteration number is less than n/2 for results with n-bit accuracy. The introduced extra table size is of the same order of magnitude as that for the angle set ltan−1 2−i, i e 0,1,…,nr, required by general CORDIC algorithms. The new recoding scheme can be applied to other elementary function such as division and square-root functions.
Read moreCombining surrogate models and local search for dealing with expensive multi-objective optimization problems
The development of multi-objective evolutionary algorithms (MOEAs) assisted by surrogate models has significantly increased in the last few years. However, in realworld applications, the high modality and dimensionality that functions normally have, often causes problems to such models. Therefore, if the Pareto optimal set of a multi-objective optimization problem is located in a search space in which the surrogate model is not able to shape the corresponding region, the search could be misinformed and thus converge to wrong regions. This has motivated the idea of incorporating refinement mechanisms to such approaches, in order to improve the search. In this paper, we present a local search mechanism which improves the search of a MOEA assisted by surrogate models. Our preliminary results indicate that our proposed approach can produce good quality results when it is restricted to performing only between 1,000 and 5,000 fitness function evaluations. Our proposed approach is validated using a set of standard test problems and an airfoil design problem.
Read moreMONTHLY STREAM FLOWS ESTIMATION IN THE KARASU RIVER OF EUPHRATES BASIN WITH ARTIFICIAL NEURAL NETWORKS APPROACH
This study aims to estimate streamflow values with artificial neural networks (ANN) using various meteorological parameters. In developing the ANN model, various combinations of precipitation, air temperatures, and potential evapotranspiration values were used as inputs, and streamflow values were obtained. Meteorological data is divided into 70% train, 15% test, and 15% validation. In the model's design, various training algorithms, network architecture, input combinations, and the number of iterations were tried, and the most suitable model was tested. Correlation coefficient (R), coefficient of determination (R2), absolute error (AE), and absolute relative error (ARE) coefficients were compared, and the most suitable model was selected. According to the analysis results, the optimal model was obtained using 2000 iterations, the architecture of the 4-4-1 model, and the Quasi-Newton algorithm. It was determined that the ANNs successfully modeled the rainfall-runoff relationship and produced reliable estimates. In addition, it was revealed that the inclusion of potential evapotranspiration values obtained by the Thornthwaite method into the model increases the model's success.
Read moreUpdating of conjugate direction matrices using members of Broyden's family
Many iterative algorithms for optimization calculations use a second derivative approximation,B say, in order to calculate the search directiond = −B −1∇f(x). In order to avoid invertingB we work with matricesZ, whose columns satisfy the conjugacy relationsZ T BZ = I. We present an update ofZ that is compatible with members of the Broyden family that generate positive definite second derivative approximations. The algorithm requires only 3n 2+O(n) flops for the update ofZ and the calculation ofd. The columns of the resultantZ matrices have interesting conjugacy and orthogonality properties with respect to previous second derivative approximations and function gradients, respectively. The update also provides a simple proof of Dixon's theorem. For the BFGS method we adapt the algorithm in order to obtain a null space method for linearly constrained calculations.
Read more