A new super-memory gradient method with curve search rule

In this article, we propose a new line search method for solving unconstrained optimization problems in that we combine a nonmonotone strategy into a modified Armijo rule and design a new algorithm that possibly chooses a larger steplength. This can decrease the number of iterations and function evaluations and can improve the efficiency of the algorithm. The global convergence and convergence rate are analysed under some suitable conditions. Preliminary numerical experiments establish that the new approach is robust and efficient for unconstrained optimization problems.

In this paper, we propose a new class of adaptive trust region methods for unconstrained optimization problems and develop some convergence properties. In the new algorithms, we use the current iterative information to define a suitable initial trust region radius at each iteration. The initial trust region radius is more reasonable in the sense that the trust region model and the objective function are more consistent at the current iterate. The global convergence, super-linear and quadratic convergence rate are analyzed under some mild conditions. Numerical results show that some special adaptive trust region methods are available and efficient in practical computation.

We present a new solution framework to solve the generalized trust region subproblem (GTRS) of minimizing a quadratic objective over a quadratic constraint. More specifically, we derive a convex quadratic reformulation (CQR) via minimizing a linear objective over two convex quadratic constraints for the GTRS. We show that an optimal solution of the GTRS can be recovered from an optimal solution of the CQR. We further prove that this CQR is equivalent to minimizing the maximum of the two convex quadratic functions derived from the CQR for the case under investigation. Although the latter minimax problem is nonsmooth, it is well structured and convex. We thus develop two steepest descent algorithms corresponding to two different line search rules. We prove global sublinear convergence rates for both algorithms. We also obtain a local linear convergence rate of the first algorithm by estimating the Kurdyka-Łojasiewicz exponent at any optimal solution under mild conditions. We finally demonstrate the efficiency of our algorithms with numerical experiments.

<p style='text-indent:20px;'>Motivated by recent derivative-free projection methods proposed in the literature for solving nonlinear constrained equations, in this paper we propose a unified derivative-free projection method model for large-scale nonlinear equations with convex constraints. Under mild conditions, the global convergence and convergence rate of the proposed method are established. In order to verify the feasibility and effectiveness of the model, a practical algorithm is devised and the corresponding numerical experiments are reported, which show that the proposed practical method is efficient and can be applied to solve large-scale nonsmooth equations. Moreover, the proposed practical algorithm is also extended to solve the obstacle problem.</p>

The purpose of this research is to develop an advanced reconstruction method for low-count, hence high-noise, single-photon emission computed tomography (SPECT) image reconstruction. It consists of a novel reconstruction model to suppress noise while conducting reconstruction and an efficient algorithm to solve the model. A novel regularizer is introduced as the nonconvex denoising term based on the approximate sparsity of the image under a geometric tight frame transform domain. The deblurring term is based on the negative log-likelihood of the SPECT data model. To solve the resulting nonconvex optimization problem a preconditioned fixed-point proximity algorithm (PFPA) is introduced. We prove that under appropriate assumptions, PFPA converges to a local solution of the optimization problem at a global convergence rate. Substantial numerical results for simulation data are presented to demonstrate the superiority of the proposed method in denoising, suppressing artifacts and reconstruction accuracy. We simulate noisy 2D SPECT data from two phantoms: hot Gaussian spheres on random lumpy warm background, and the anthropomorphic brain phantom, at high- and low-noise levels (64k and 90k counts, respectively), and reconstruct them using PFPA. We also perform limited comparative studies with selected competing state-of-the-art total variation (TV) and higher-order TV (HOTV) transform-based methods, and widely used post-filtered maximum-likelihood expectation-maximization. We investigate imaging performance of these methods using: contrast-to-noise ratio (CNR), ensemble variance images (EVI), background ensemble noise (BEN), normalized mean-square error (NMSE), and channelized hotelling observer (CHO) detectability. Each of the competing methods is independently optimized for each metric. We establish that the proposed method outperforms the other approaches in all image quality metrics except NMSE where it is matched by HOTV. The superiority of the proposed method is especially evident in the CHO detectability tests results. We also perform qualitative image evaluation for presence and severity of image artifacts where it also performs better in terms of suppressing ‘staircase’ artifacts, as compared to TV methods. However, edge artifacts on high-contrast regions persist. We conclude that the proposed method may offer a powerful tool for detection tasks in high-noise SPECT imaging.

The Sixth International Symposium on Neural Networks (ISNN 2009)

In wireless sensor networks, clustering is a very crucial problem. Basically clustering means grouping some specific objects based on their behavior and functionality. Clustering can be formulated for different optimization problems, such as nonsmooth, nonconvex problems. This paper is based on the review of the optimization algorithm that was proposed in the paper A Convergent Incremental Gradient Method With Constant Step Size by Blatt et al called Incremental Aggregate Gradient method. A novel algorithm called Incremental Clustered Aggregate Gradient Method was proposed in this paper to counter the shortcomings of the previous one. It has many similarities with the earlier method but it is more efficient for wireless sensor networks. The main aim of Incremental Gradient Method was to minimize the sum of continuously differentiable functions and also it required a single gradient evaluation per iteration and used a constant step size. For quadratic functions, a global linear rate of convergence was proved. It was claimed that it is more suitable for sensor networks. Although the experiments performed in this work confirm the convergence properties of it, it was found that it is not suitable for sensor networks. The proposed method addresses the flaws of the previous method as regards to sensor networks. When both algorithms operate with their respective optimal step sizes, they require approximately the same number of gradient evaluations for convergence.

A superlinearly convergent SQP algorithm for mathematical programs with linear complementarity constraints

In this paper we propose a distributed dual gradient algorithm for minimizing linearly constrained separable convex problems and analyze its rate of convergence. In particular, we show that under the assumption that the Hessian of the primal objective function is bounded we have a global error bound type property for the dual problem. Using this error bound property we devise a fully distributed dual gradient scheme for which we derive global linear rate of convergence. The proposed dual gradient method is fully distributed, requiring only local information, since is based on a weighted stepsize. Our method can be applied in many applications, e.g. distributed model predictive control, network utility maximization or optimal power flow.

In this paper, we present a new nonmonotone memory gradient algorithm for unconstrained optimization problems. An attractive property of the proposed method is that the search direction always provides sufficient descent step at each iteration. This property is independent of the line search used. Under mild assumptions, the global and local convergence results of the proposed algorithm are established respectively. Numerical results are also reported to show that the proposed method is suitable to solve large-scale optimization problems and is more stable than other similar methods in practical computation.

A dwindling filter inexact projected Hessian algorithm for large scale nonlinear constrained optimization

In this paper, we propose new linearly convergent second-order methods for minimizing convex quartic polynomials. This framework is applied for designing optimization schemes, which can solve general convex problems satisfying a new condition of quartic regularity. It assumes positive definiteness and boundedness of the fourth derivative of the objective function. For such problems, an appropriate quartic regularization of Damped Newton Method has global linear rate of convergence. We discuss several important consequences of this result. In particular, it can be used for constructing new second-order methods in the framework of high-order proximal-point schemes (Nesterov, Math. Program. 197, 1–26, 2023 and Nesterov, SIAM J. Optim. 31, 2807–2828, 2021). These methods have convergence rate O~(k-p), where k is the iteration counter, p is equal to 3, 4, or 5, and tilde indicates the presence of logarithmic factors in the complexity bounds for the auxiliary problems, which are solved at each iteration of the schemes.

This paper proposes a new proximal iteratively reweighted nuclear norm method for a class of nonconvex and nonsmooth optimization problems. The primary contribution of this work is the incorporation of line search technique based on dimensionality reduction and extrapolation. This strategy overcomes parameter constraints by enabling adaptive dynamic adjustment of the extrapolation/proximal parameters (αk, βk, μk). Under the Kurdyka–Łojasiewicz framework for nonconvex and nonsmooth optimization, we prove the global convergence and linear convergence rate of the proposed algorithm. Additionally, through numerical experiments using synthetic and real data in matrix completion problems, we validate the superior performance of the proposed method over well-known methods.

Recently, several convergence rate results for Douglas-Rachford splitting and the alternating direction method of multipliers (ADMM) have been presented in the literature. In this paper, we show global linear convergence rate bounds for Douglas-Rachford splitting and ADMM under strong convexity and smoothness assumptions. We further show that the rate bounds are tight for the class of problems under consideration for all feasible algorithm parameters. For problems that satisfy the assumptions, we show how to select step-size and metric for the algorithm that optimize the derived convergence rate bounds. For problems with a similar structure that do not satisfy the assumptions, we present heuristic step-size and metric selection methods.

It is well-known that the biogeography-based optimization (BBO) algorithm lacks searching power in some circumstances. In order to address this issue, an adaptive opposition-based biogeography-based optimization algorithm (AO-BBO) is proposed. Based on the BBO algorithm and opposite learning strategy, this algorithm chooses different opposite learning probabilities for each individual according to the habitat suitability index (HSI), so as to avoid elite individuals from returning to local optimal solution. Meanwhile, the proposed method is tested in 9 benchmark functions respectively. The results show that the improved AO-BBO algorithm can improve the population diversity better and enhance the search ability of the global optimal solution. The global exploration capability, convergence rate and convergence accuracy have been significantly improved. Eventually, the algorithm is applied to the parameter optimization of soft-sensing model in plant medicine extraction rate. The simulation results show that the model obtained by this method has higher prediction accuracy and generalization ability.