A line-search filter adaptive regularization algorithm using cubics based on affine scaling interior-point methods for constrained optimization

In recent years, cubic regularization algorithms for unconstrained optimization have been defined as alternatives to trust-region and line search schemes. These regularization techniques are based on the strategy of computing an (approximate) global minimizer of a cubic overestimator of the objective function. In this work we focus on the adaptive regularization algorithm using cubics (ARC) proposed in Cartis et al. [Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results, Mathematical Programming A 127 (2011), pp. 245–295]. Our purpose is to design a modified version of ARC in order to improve the computational efficiency preserving global convergence properties. The basic idea is to suitably combine a Goldstein-type line search and a nonmonotone accepting criterion with the aim of advantageously exploiting the possible good descent properties of the trial step computed as (approximate) minimizer of the cubic model. Global convergence properties of the proposed nonmonotone ARC algorithm are proved. Numerical experiments are performed and the obtained results clearly show satisfactory performance of the new algorithm when compared to the basic ARC algorithm.

In radar and communication systems, a broad frequency band can be synthesized of several narrow ones. At millimeter and sub-millimeter waves, the multiplexer for combining narrow-banded channels can be composed of quasi-optical mirror cavities. However, mutual adjustment of the cavities for radar and communication should be quite different. In the radar with synthesized frequency band, an adaptive spectrum regularization algorithm provides automatic shrinkage of the range resolution scale with the target approaching.

Spaceborne scatterometers are active non-imaging radar systems, become one of the most effective sensors in the field of quantitative remote sensing for global observation. However, their nominal resolution of 25 to 50 km limits their applicability in scenarios requiring higher resolution. In this paper, an improved adaptive bilateral-total-variation regularization reconstruction algorithm with Lorentzian norm (LABTV+ RR algorithm) is specifically designed for the world’s first dual-frequency scatterometer Fengyun-3E Wind Radar (FY-3E WindRAD) data. LABTV+ RR algorithm introduces dynamically adaptive regularization parameters based on our previously proposed LABTV RR algorithm, which effectively suppresses the noise while maintaining the image texture details, enabling more flexible adaptation to different image contents and noise levels. In addition, the performance of the LABTV+ RR algorithm is further optimized by employing Barzilai-Borwein (BB) stepsize, which dynamically adjusts the stepsize and accelerates the convergence speed of the solution. In this study, the effectiveness of the LABTV+ RR algorithm is validated by comparing actual data and simulated images. The spatial response function (SRF) derived from the actual antenna patterns is used to validate the algorithm’s performance on FY-3E WindRAD Level 1B (L1B) data. Specifically, the algorithm is tested on C-band with a spatial resolution pixel size of 25 km×0.25 km and Ku-band data with a spatial resolution pixel size of 10 km×0.25 km. The validation includes both horizontally polarized (HH-pol) and vertically polarized (VV-pol) transmitted and received signals. Additionally, two representative regions in China are chosen as the study regions to demonstrate the algorithm’s capability to enhance resolution to 3.125 km in both C-band and Ku-band. Furthermore, a multi-pass reconstruction strategy is proposed to achieve an even higher resolution pixel size of 1.5625 km for FY-3E WindRAD C-band and Ku-band data. The study has demonstrated the effectiveness of the proposed LABTV+ RR algorithm in enhancing the resolution of FY-3E WindRAD data, as evidenced by both qualitative visual assessments and quantitative evaluation metrics.

In this chapter, an implementation of an interior point filter line-search algorithm for large-scale nonlinear programming proposed by Wachter and Biegler (2005a, b) is presented. As we know, to allow convergence from poor starting points and to enforce progress to the solution, interior point methods both in trust region and in line-search frameworks with exact penalty merit function have been developed. For example, KNITRO uses the l 1 exact penalty function (Byrd, Hribar, & Nocedal, 1999; Byrd, Gilbert, & Nocedal, 2000). On the other hand, Fletcher and Leyffer (2002) proposed filter methods as an alternative to merit functions, as a tool for global convergence guarantee in algorithms for nonlinear optimization. The idea of filter is that trial points are accepted if they improve the objective function value or improve the constraint violation instead of a combination of these two measures defined by the merit function. Even if the filter methods include different heuristics, they have been adapted to barrier methods in a number of ways. For example, Ulbrich et al. (2004) considered a trust-region filter method and accept the trial step on the basis of the norm of the optimality conditions. Benson et al. (2002a) proposed several heuristics using the idea of filter methods, for which improved efficiency is reported compared to their previous merit function approach. The global convergence of an interior point algorithm with a filter line search was analyzed by Wachter and Biegler (2001). Here, the assumptions made for the analysis of the global convergence are less restrictive than those made for line-search interior point methods for nonlinear programming developed, for example, by El-Bakry et al. (1996), Yamashita (1998), or Tits et al. (2003).

Because of the traditional regularization parameters in the regularization method is fixed, in reconstruction images are not good to keep details such as image edge and texture information.In view of these shortcomings is proposed in this paper a bilateral total variation based on adaptive regularization image super-resolution algorithm, through changing regularized parameter to control the data fidelity term in the objective function and the proportion of regularization.Experimental results show that compared with the traditional reconstruction method in this paper, the method to the determination of adaptive regularization parameter, find the optimal solution, and in the region of the edge and texture details such as embodies better reconstruction effect.

The structure of block sparsity in multi-band signals is prevalent. Among the block-sparse signal problems for compressive sensing, the most presenting recovery algorithms require block sparsity as prior information, whereas it may not be available in many practical applications. In this paper, a block sparsity adaptive regularized orthogonal matching pursuit algorithm (BAROMP) for compressive sensing is presented. The proposed algorithm could guarantee the accuracy of recovery by both the adaptive process which chooses the candidate set automatically and the regularization process which decides the atoms in the final support set. The simulation results show that the recovery probability of BAROMP which does not require sparsity as prior information is near to BROMP.

In this paper, we present a new filter line search method based on two measures that is integrated into the primal‐dual barrier method developed by Wächter and Biegler [Mathematical Programming 106 (2006), pp. 25–57] for nonlinear programming. One measure arises directly from the first order optimality conditions of the problem and the other is the barrier function. Primary assessment of the method has been done with a well‐known collection of problems and compared with the solver IPOPT.

The propagation of single numerical errors is studied for several different adaptation algorithms for linear regression problems. The case where the regressors are poorly exciting (the information matrix is singular or close to singular) is of particular interest. It is shown how appropriate regularization algorithms may secure exponential stability of the error propagation (and of the propagation of initial conditions in general). Gradient and Gauss-Newton algorithms with forgetting factors as well as directional forgetting are studied.

In this letter, we propose an adaptive step-size algorithm for the adaptive interference canceller (AIC) in the space-time coded DS-CDMA system. To improve the performance of the fixed step-size AIC (FS-AIC), the regular adaptive step-size algorithm is extended in complex domain and applied to the joint AIC and ML decoder scheme. Simulation results show that the joint adaptive step-size AIC (AS-AIC) and ML decoder scheme using the proposed algorithm has better performance than not only the conventional ML decoder but also the joint FS-AIC and ML decoder scheme without much increase of the decoding delay and complexity.

Hanging nodes have some disadvantages in the implementation of adaptive finite element methods on quadrilateral meshes, which usually need further techniques to treat them. In this paper, we present a shape regular local refinement algorithm on quadrilateral meshes without hanging nodes, which can be viewed as an extension of the original red-green refinement proposed by Bank, Sherman, and Weiser [Refinement algorithms and data structures for regular local mesh refinement, in Scientific Computing: Applications of Mathematics and Computing to the Physical Sciences, R. S. Stepleman, ed., North-Holland, Amsterdam, 1983, pp. 3-17]. We also prove this quadrilateral red-green refinement can hold the shape regularity and conformity of quadrilateral meshes. Furthermore, we have successfully accomplished the adaptive finite element computation on quadrilateral meshes without hanging nodes. Numerical results show that the adaptive algorithm with this red-green refinement is quasi-optimal. Some properties of this local refinement on quadrilateral meshes are also covered in this paper.

In this paper, a new nonmonotone adaptive trust region algorithm is proposed for unconstrained optimization by combining a multidimensional filter and the Goldstein-type line search technique. A modified trust region ratio is presented which results in more reasonable consistency between the accurate model and the approximate model. When a trial step is rejected, we use a multidimensional filter to increase the likelihood that the trial step is accepted. If the trial step is still not successful with the filter, a nonmonotone Goldstein-type line search is used in the direction of the rejected trial step. The approximation of the Hessian matrix is updated by the modified Quasi-Newton formula (CBFGS). Under appropriate conditions, the proposed algorithm is globally convergent and superlinearly convergent. The new algorithm shows better performance in terms of the Dolan–Moré performance profile. Numerical results demonstrate the efficiency and robustness of the proposed algorithm for solving unconstrained optimization problems.

It is known that predictor-corrector methods in a large neighborhood of the central path are among the most efficient interior-point methods (IPMs) for linear optimization (LO) problems. The best iteration bound based on the classical logarithmic barrier function is O(nlog (n/∊)). In this paper, we propose a family of self-regular proximity-based predictor-corrector (SRPC) IPMs for LO in a large neighborhood of the central path. In the predictor step, we use either an affine scaling or a self-regular direction; in the corrector step, we use always a self-regular direction. Our new algorithms use a special proximity function with different search directions and thus allows us to improve the so far best theoretical iteration complexity for a family of SRPC IPMs. An O\((\sqrt{n}{\exp} ((1 - q + {\log} n)/2) {\log} n {\log} (n/\epsilon))\) worst-case iteration bound with quadratic convergence is established, where q is the barrier degree of the SR proximity function.

Atomic decomposition is an alternative method for frequency detection in harmonic signals. This type of method produces very concentrated solutions with few nonzero components. It can be used as an alternative to traditional approaches, such as, principal-components frequency estimation methods. In this paper, we consider the basis pursuit principle to find the representation (frequency) coefficients of a harmonic signal by minimizing the l 1 norm. For the l 1 minimization, we compare two interior-point methods. A primal-dual method, which consists of the perturbed optimality conditions of the linear program, results in solutions that are more accurate and sparse than using a primal (affine scaling) method to solve the same linear program. We contrast the solutions obtained by the interior-point methods using the size of the given data and a bound for perfect recovery of the harmonic signals to establish the better performance of the primal-dual method. In addition, experimental results are shown.

The main purpose of this research is to investigate stable synthesis algorithms of a neural network regulator for controlling dynamic systems. In this research, the synthesis of neural network regulator and analysis of the modern condition of intelligent control systems has been conducted. This paper presents stable synthesis algorithms of a multi-mode neural network controller on the basis of methods for solving variational inequalities, which can ensure the consistency of desired estimations and high accuracy of the intelligent control system. Based on estimation theory, the regular algorithms have been proposed in combination with adaptive identification algorithms. These derived algorithms can provide consistency of desired estimations with convergence properties and can be used in solving different kinds of problems related to system synthesis for controlling dynamic objects used for several functional purposes.KeywordsNeural network regulatorTheory of artificial intelligenceNeural networksDynamic systemControl objectAdaptive identificationEstimation theoryIncorrectly posed problem

Simplex algorithms have been proven to be a reliable nonlinear programming pattern search algorithm. The effectiveness of simplex however reduces when the solved problem has large number of variables, several local minima, or when initial guess is not readily available. Recent results obtained by introducing a technique of random selection of the variables in optimization processes, encouraged studying the effect of this idea on the Nelder & Mead version of the simplex algorithm to improve its semi-global behavior. This paper proposes several enhancements to the simplex. The algorithm is run for several attempts. Each attempt uses the final answer of the previous attempt as an initial guess. At each attempt, the search starts by generating a simplex with n+1 vertices. A random subset of the variables is involved in the movement (reflection, expansion, contraction, shrinking) of the simplex. This process is called twinkling. The paper presents several variations of twinkling the simplex. The random nature of the algorithm lends itself to problems with large dimensionality or complex topography. The algorithm is applied successfully to several engineering design problems. The results compare favorably with both regular simplex and genetic algorithms.