Novel Reformulations and Efficient Algorithms for the Generalized Trust Region Subproblem

We investigate in this paper the generalized trust region subproblem (GTRS) of minimizing a general quadratic objective function subject to a general quadratic inequality constraint. By applying a simultaneous block diagonalization approach, we obtain a congruent canonical form for the symmetric matrices in both the objective and constraint functions. By exploiting the block separability of the canonical form, we show that all GTRSs with an optimal value bounded from below are second order cone programming (SOCP) representable. Our result generalizes the recent work of Ben-Tal and Hertog (Math. Program. 143(1-2):1-29, 2014), which establishes the SOCP representability of the GTRS under the assumption of the simultaneous diagonalizability of the two matrices in the objective and constraint functions. Compared with the state-of-the-art approach to reformulate the GTRS as a semi-definite programming problem, our SOCP reformulation delivers a much faster solution algorithm. We further extend our method to two variants of the GTRS in which the inequality constraint is replaced by either an equality constraint or an interval constraint. Our methods also enable us to obtain simplified versions of the classical S-lemma, the S-lemma with equality, and the S-lemma with interval bounds.

An efficient inertial subspace minimization CG algorithm with convergence rate analysis for constrained nonlinear monotone equations

Though quasi-Newton methods have been extensively studied in the literature, they either suffer from local convergence or use a series of line searches for global convergence. In this work, we propose a line search free greedy quasi-Newton method with adaptive steps and establish explicit nonasymptotic bounds for both the global convergence rate and local superlinear rate. Our novel idea lies in the design of multiple greedy quasi-Newton updates to control the Hessian approximation error and a simple mechanism to adjust stepsizes to ensure function improvement per iteration. The global superlinear convergence <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> of our method is validated via numerical experiments.

&lt;p style='text-indent:20px;'&gt;Motivated by recent derivative-free projection methods proposed in the literature for solving nonlinear constrained equations, in this paper we propose a unified derivative-free projection method model for large-scale nonlinear equations with convex constraints. Under mild conditions, the global convergence and convergence rate of the proposed method are established. In order to verify the feasibility and effectiveness of the model, a practical algorithm is devised and the corresponding numerical experiments are reported, which show that the proposed practical method is efficient and can be applied to solve large-scale nonsmooth equations. Moreover, the proposed practical algorithm is also extended to solve the obstacle problem.&lt;/p&gt;

Feature selection (FS) constitutes a critical stage within the realms of machine learning and data mining, with the objective of eliminating irrelevant features while guaranteeing model accuracy. Nevertheless, in datasets featuring a multitude of features, choosing the optimal feature poses a significant challenge. This study presents an enhanced Sand Cat Swarm Optimization algorithm (MSCSO) to improve the feature selection process, augmenting the algorithm's global search capacity and convergence rate via multiple innovative strategies. Specifically, this study devised logistic chaotic mapping and lens imaging reverse learning approaches for population initialization to enhance population diversity; balanced global exploration and local development capabilities through nonlinear parameter processing; and introduced a Weibull flight strategy and triangular parade strategy to optimize individual position updates. Additionally, the Gaussian-Cauchy mutation strategy was employed to improve the algorithm's ability to overcome local optima. The experimental results demonstrate that MSCSO performs well on 65.2% of the test functions in the CEC2005 benchmark test; on the 15 datasets of UCI, MSCSO achieved the best average fitness in 93.3% of the datasets and achieved the fewest feature selections in 86.7% of the datasets while attaining the best average accuracy across 100% of the datasets, significantly outperforming other comparative algorithms.

A new super-memory gradient method with curve search rule

In this article, we propose a new line search method for solving unconstrained optimization problems in that we combine a nonmonotone strategy into a modified Armijo rule and design a new algorithm that possibly chooses a larger steplength. This can decrease the number of iterations and function evaluations and can improve the efficiency of the algorithm. The global convergence and convergence rate are analysed under some suitable conditions. Preliminary numerical experiments establish that the new approach is robust and efficient for unconstrained optimization problems.

In this paper, we introduce a geometric model for linear symmetric eigenvalue problem, which is motivated by the fact that any eigenvalue of a symmetric positive definite matrix A is the reciprocal of the square length of an axis of the ellipsoid xTAx = 1. Hence, to find the largest eigenvalue is equivalent to calculate the shortest axis of the corresponding ellipsoid. Two sequential subspace projection algorithms based on this idea are proposed, and we establish the global convergence and local linear convergence rate of our proposed algorithms. Numerical experiments demonstrate that our algorithm outperforms the MATLAB built-in solver "EIGS" which calls the famous package "ARPACK".

This paper proposes a new proximal iteratively reweighted nuclear norm method for a class of nonconvex and nonsmooth optimization problems. The primary contribution of this work is the incorporation of line search technique based on dimensionality reduction and extrapolation. This strategy overcomes parameter constraints by enabling adaptive dynamic adjustment of the extrapolation/proximal parameters (αk, βk, μk). Under the Kurdyka–Łojasiewicz framework for nonconvex and nonsmooth optimization, we prove the global convergence and linear convergence rate of the proposed algorithm. Additionally, through numerical experiments using synthetic and real data in matrix completion problems, we validate the superior performance of the proposed method over well-known methods.

A dwindling filter inexact projected Hessian algorithm for large scale nonlinear constrained optimization

Recently, several convergence rate results for Douglas-Rachford splitting and the alternating direction method of multipliers (ADMM) have been presented in the literature. In this paper, we show global linear convergence rate bounds for Douglas-Rachford splitting and ADMM under strong convexity and smoothness assumptions. We further show that the rate bounds are tight for the class of problems under consideration for all feasible algorithm parameters. For problems that satisfy the assumptions, we show how to select step-size and metric for the algorithm that optimize the derived convergence rate bounds. For problems with a similar structure that do not satisfy the assumptions, we present heuristic step-size and metric selection methods.

In this article, non-linear minimax problems with general constraints are discussed. By means of solving one quadratic programming an improved direction is yielded and a second-order correction direction can also be at hand via one system of linear equations. So a new algorithm for solving the discussed problems is presented. In connection with a special merit function, the generalized monotone line search is used to yield the step size at each iteration. Under mild conditions, we can ensure global and superlinear convergence. Finally, some numerical experiments are operated to test our algorithm, and the results demonstrate that it is promising.

Recently, with the growth of cyber physical systems (CPS), several applications have begun to deploy in the CPS for connecting the cyber space with the physical scale effectively. Besides, the cloud computing (CC) enabled CPS offers huge processing and storage resources for CPS that finds helpful for a range of application areas. At the same time, with the massive development of applications that exist in the CPS environment, the energy utilization of the cloud enabled CPS has gained significant interest. For improving the energy effectiveness of the CC platform, virtualization technologies have been employed for resource management and the applications are executed via virtual machines (VMs). Since effective scheduling of resources acts as an important role in the design of cloud enabled CPS, this paper focuses on the design of chaotic sandpiper optimization based VM scheduling (CSPO-VMS) technique for energy efficient CPS. The CSPO-VMS technique is utilized for searching for the optimum VM migration solution and it helps to choose an effective scheduling strategy. The CSPO algorithm integrates the concepts of traditional SPO algorithm with the chaos theory, which substitutes the main parameter and combines it with the chaos. In order to improve the process of determining the global optimum solutions and convergence rate of the SPO algorithm, the chaotic concept is included in the SPO algorithm. The CSPO-VMS technique also derives a fitness function to choose optimal scheduling strategy in the CPS environment. In order to demonstrate the enhanced performance of the CSPO-VMS technique, a wide range of simulations were carried out and the results are examined under varying aspects. The simulation results ensured the improved performance of the CSPO-VMS technique over the recent methods interms of different measures.

BEES algorithm, inspired by the natural foraging behaviour of honey bees, is one among new meta-heuristic optimisation techniques. However, the performance of the algorithm depends on its parameters levels and need to be determined and analysed before its implementation. This paper proposes the integrated algorithm based on steepest descent algorithm (SDA) and the Nelder–Mead modified simplex method (MSM) for investigating the proper BEES parameter levels for truckload trucking management. The dynamic multi-zone dispatching problem concentrates on the quantities of inbound and outbound in each area and it is modified from the multi-zone dispatching. The factors of the rearrangement penalty of the area, in each zone, including time periods are also included. In this research there are various levels of areas, zones, inbound, outbound and time periods. In practice both elements from this integrated algorithm are very easy to implement. An aim to integrate SDA and MSM is to produce faster and more accurate convergence when managing zones with minimal imbalance scenario via an application of the BEES algorithm. The main purpose of the paper is to demonstrate how the BEES can be improved by incorporating a proper parameter level selection strategy. Experimental results were analysed in terms of best solutions found so far, mean and standard deviation on both the imbalance and execution time to converge to the optimum. In a suite of tested problems taken from Thai leading transportation company, computational results of the investigation of parameter selection show that the integrated approach outperforms other two original techniques of the SDA and MSM in terms of solution quality and convergence rate. The comparison shows that the integrated algorithm provides largely favourable performance measures of accuracy, robustness and problem evaluation. Finally a recommendation of proper level settings of BEES parameters for some selected problem sizes that can be used as a guideline for future applications of the BEES. This is to promote ease of use of the BEES in real life problems. This study also found that number of zones affect iterations toward the optimum. Number of areas affects the imbalance. The parameters of zone and area are then the important variables for these multi-zone dispatching systems.

Bat algorithm (BA) is a new stochastic optimization technique for global optimization. In the paper, we introduce both simulated annealing and Gaussian perturbations into the standard bat algorithm so as to enhance its search performance. As a result, we propose a simulated annealing Gaussian bat algorithm (SAGBA) for global optimization. Our proposed algorithm not only inherits the simplicity and efficiency of the standard BA with a capability of searching for global optimality, but also speeds up the global convergence rate. We have used BA, simulated annealing particle swarm optimization and SAGBA to carry out numerical experiments for 20 test benchmarks. Our simulation results show that the proposed SAGBA can indeed improve the global convergence. In addition, SAGBA is superior to the other two algorithms in terms of convergence and accuracy.