A variant of trust-region methods for unconstrained optimization

It is well known that among the current methods for unconstrained optimization problems the quasi-Newton methods with global strategy may be the most efficient methods, which have local superlinear convergence. However, when the iterative point is far away from the solution of the problem, quasi-Newton method may proceed slowly for the general unconstrained optimization problems. In this article an adaptive conic trust-region method for unconstrained optimization is presented. Not only the gradient information but also the values of the objective function are used to construct the local model at the current iterative point. Moreover, we define a concept of super steepest descent direction and embed its information into the local model. The amount of computation in each iteration of this adaptive algorithm is the same as that of the standard quasi-Newton method with trust region. Some numerical results show that the modified method requires fewer iterations than the standard methods to reach the solution of the optimization problem. Global and local convergence of the method is also analyzed.

In this paper, we present a new line search and trust region algorithm for unconstrained optimization problems. The trust region center locates at somewhere in the negative gradient direction with the current best iterative point being on the boundary. By doing these, the trust region subproblems are constructed at a new way different with the traditional ones. Then, we test the efficiency of the new line search and trust region algorithm on some standard benchmarking. The computational results reveal that, for most test problems, the number of function and gradient calculations are reduced significantly.

In classical trust-region optimization algorithms, the radius of the trust region is reduced, kept constant, or enlarged after, respectively, unsuccessful, successful, and very successful iterations. We propose here to refine the empirical rules used for this update by the definition of a new set of iterations that we call "too successful iterations." At such iterations, a large reduction of the objective function is obtained despite a crude local approximation of the objective function; the trust region is thus kept nearly constant instead of being enlarged. The new update rules preserve the strong convergence property of traditional trust-region methods. They can also be generalized to define a self-adaptive trust-region algorithm along the lines introduced by Hei [J. Comput. Math., 21 (2003), pp. 229-236]. Numerical experiments carried out on 70 unconstrained problems from the CUTEr collection demonstrate the positive impact of the modified update strategy on the efficiency and robustness of quasi-Newton variants of a trust-region solver, when BFGS or SR1 updates of the approximation of the Hessian matrix are carried at all iterations.

The main goal of this research article is to discuss the mathematical and statistical aspects of linear models and their characteristic properties. Linear model is the most common modeling used in science. Actually linear models have many different meanings depend on the context. Linear model is often preferred than other model such as quadratic model because of its ability to interpret easily. In the other hand most of the real life cases have linear relationship .Modeling the cases using linear model will able us to computethe relative influence of one or more independent variables to the dependent variable. In the present talk an attempt has been made to propose the specific forms of simple and multiple linear regression models. In this conversation mathematical aspects of linear statistical models have been extensively depicted. Different types of mathematical models are depictedhere and the methods of fitting transformed models are proposed.Furthermore specific form of linear statistical model is presented and the crucial assumptions of general linear model are extensively discussed.At the last stage of this article the procedure of OLS estimation of parameters of a linear model has been proposed

In this paper, we have proposed a new method to construct an auxiliary function for the discriminative training of HMMs in speech recognition. The new auxiliary function serves as a first-order approximation of the original objective function but more importantly it remains as a lower bound of the original objective function as well. Furthermore, the trust region (TR) method in [1] is applied to find the globally optimal point of the new auxiliary function. Due to its lower-bound property, the found optimal point is theoretically guaranteed to increase the original discriminative objective function. The proposed bounded trust region method has been investigated on two LVCSR tasks, namely WSJ-5k and Switchboard 60-hour subset tasks. Experimental results show that the bounded TR method yields much better convergence behavior than both the conventional EBW method and the original TR method.

Aims: This study considered the optimal maintenance cost of water boreholes for a multiple pumping scenario in South-East Nigeria. The objective of the study was to determine the optimal maintenance cost of water boreholes for the Southeastern region of Nigeria.
 Place and Duration of Study: The data used in this study is secondary data obtained from selected community and commercial boreholes in the Abia State, Anambra State, Ebonyi State, Enugu State and Imo State.
 Methodology: The linear model and the quadratic model were used to determine the cost parameters for the optimization of the maintenance cost of the water boreholes.
 Results: The result of the study showed that the maintenance cost obtained for Anambra State is more adequate than the other states since it recorded a higher coefficient of determination value using the linear and quadratic model. Also, it was found the quadratic model was better than the linear model for the estimation of cost parameters used for determining the optimum maintenance cost of water boreholes in South-East Nigeria. The findings showed that the quadratic model recorded the best net savings for Abia, Anambra, and Enugu with the corresponding cost of ₦4,079,246.817, ₦5,198,716.141, and ₦4,767,680.392 respectively. It was found that the linear model recorded the best net savings for Ebonyi and Imo with the corresponding cost of ₦1,593,013.72 and ₦8,692,452.91 respectively. Hence, the quadratic models was identified as the best model for the estimation of cost parameters and enhance the performance of the optimal total operating cost of the water borehole model in South-East Nigeria.
 Conclusion: The study concludes by recommending the quadratic model for the estimation of cost parameters for the optimization of maintenance cost of water boreholes until further studies prove otherwise.

Trust-region based solver for nonlinear transport in heterogeneous porous media

Based on simple quadratic models of the trust region subproblem, we combine the trust region method with the nonmonotone and adaptive techniques to propose a new nonmonotone adaptive trust region algorithm for unconstrained optimization. Unlike traditional trust region method, our trust region subproblem is very simple by using a new scale approximation of the minimizing function’s Hessian. The new method needs less memory capacitance and computational complexity. The convergence results of the method are proved under certain conditions. Numerical results show that the new method is effective and attractive for large scale unconstrained problems.

In this article, we propose an algorithm which solves unconstrained optimization problems by combining the trust region method with the quasi-Newton line search method. During consecutive trust region steps, when the quasi-Newton matrix B k approximates the Hessian matrix of the objective function at x k well, the algorithm tries a full quasi-Newton step and, if this step is successful, continues trying quasi-Newton steps. On the other hand, when employing the line search method, if the quasi-Newton direction is nearly orthogonal to the gradient of the objective function or the step size becomes too small, the algorithm switches back to trust region steps. Convergence properties of the algorithm are proved, and numerical results are presented.

Advances in surface electromyographic (EMG) signal decomposition now allow investigators to analyze firing rate data for 20-50 motor units per contraction. To simplify data interpretation, some investigators have relied on group mean analysis of the mean firing rate versus recruitment threshold relationship. It is unclear, however, whether this association is consistently linear. PURPOSE: To determine whether the motor unit mean firing rate versus recruitment threshold relationship is strongest when analyzed via linear, quadratic, or cubic regression. METHODS: Twenty-one men (mean ± SD age = 24 ± 4 years) participated in this study. After determining maximal voluntary contraction (MVC) force of the dominant knee extensors, participants performed trapezoidal isometric contractions at 50% MVC by tracing a visual template displayed on a monitor directly in front of them. Participants were instructed to increase force from 0 to 50% in five seconds, maintain 50% MVC for 15 seconds, and decrease force from 50% to 0 in five seconds. Bipolar surface EMG signals were recorded from the vastus lateralis during each contraction. A surface EMG signal decomposition algorithm was used to calculate mean firing rate and recruitment threshold of each detected motor unit. Motor units with decomposition accuracy levels <90% were discarded. Polynomial regression was used to determine if each mean firing rate versus recruitment threshold relationship was best fit with a linear, quadratic, or cubic model. Data were interpreted on an individual participant basis. RESULTS: Statistically significant (p <.05) moderate to strong (r2=.599-.964) linear relationships existed for all 21 participants. Fourteen of the 21 participants demonstrated relationships that were best fit with a linear model (r2=.599-.964). Of the remaining seven participants, five were best fit with a quadratic model (r2=.864-.953) and two were best fit with a cubic model (r2=.977-.989). CONCLUSION: While moderate to strong linear relationships were found between the firing rate of motor units and their recruitment thresholds for all participants, in certain cases the strength of the association was enhanced when analyzed via a non-linear model. Our findings provide further support for the need to examine motor unit data on a participant-by-participant basis.

A trust-region algorithm combining line search filter method with Lagrange merit function for nonlinear constrained optimization

Background: India is one of the leading global producers of groundnut, with the states of Gujarat, Madhya Pradesh, and Tamil Nadu at the forefront. However, the area dedicated to groundnut cultivation and its production in India has experienced significant fluctuations over the years, leading to challenges in agricultural planning.Methods: This research seeks to examine the trends in both area and production of groundnut in selected districts of Karnataka, specifically Tumkur and Chitradurga was major purposively selected for the study , because due to production and productivity is more for last five years. The study employs various time series models, including linear, quadratic, cubic, exponential, and logistic regression models were formulated, utilizing a dataset spanning twenty-five years (1997- 2022).Results: The results indicate that, the cubic model is the most appropriate for assessing the area under cultivation, while the quadratic model is deemed to be most robust for analyzing production trends of groundnut in different agro climatic zones. Additionally, the study addressed various challenges and risks associated with groundnut production in Karnataka. Linear, quadratic and exponential models were fitted .Asper the results , the cubic and logistic models were found to be insignificant. Additionally, the Run’s test and Shapiro-Wilk test statistics were found to be non-significant for quadratic model. Therefore, the data on groundnut production in Tumkur district during the study period was well-fitted by the quadratic model as compared with rest of the study sites.Conclusion: The findings aim to enhance the understanding of different oilseed production trends, thereby aiding farmers and government stakeholders in making informed decisions to ensure stable production across larger areas.

In this work, we consider methods for large-scale and nonconvex unconstrained optimization. We propose a new trust-region method whose subproblem is defined using a so-called ‘shape-changing’ norm together with densely-initialized multipoint symmetric secant (MSS) matrices to approximate the Hessian. Shape-changing norms and dense initializations have been successfully used in the context of traditional quasi-Newton methods, but have yet to be explored in the case of MSS methods. Numerical results suggest that trust-region methods that use densely-initialized MSS matrices together with shape-changing norms outperform MSS with other trust-region methods.

In this paper, a new trust region subproblem is proposed. The trust radius in the new subproblem adjusts itself adaptively. As a result, an adaptive trust region method is constructed based on the new trust region subproblem. The local and global convergence results of the adaptive trust region method are proved. Numerical results indicate that the new method is very efficient.

In this paper, we introduce a new concept of approximate optimal stepsize for gradient method, use it to interpret the Barzilai-Borwein (BB) method, and present an efficient gradient method with approximate optimal stepsize for large unconstrained optimization. If the objective function f is not close to a quadratic on a line segment between the current iterate x k and the latest iterate x k−1, we construct a conic model to generate the approximate optimal stepsize for gradient method if the conic model is suitable to be used. Otherwise, we construct a new quadratic model or two other new approximation models to generate the approximate optimal stepsize for gradient method. We analyze the convergence of the proposed method under some suitable conditions. Numerical results show the proposed method is very promising.