A subgradient projection method for quasiconvex minimization

It is significant to solve \(l_1\) minimization problems efficiently and reliably in compressed sensing (CS) since the \(l_1\) minimization is essential for the recovery of sparse signals. In view of this, a neurodynamic optimization approach is proposed for solving the \(l_1\)-minimization problems for reconstruction of sparse signals based on a projection neural network (PNN). The proposed neurodynamic optimization approach differs from most \(l_1\)-solvers in that it operates in continuous time rather than being specified by discrete iterations; i.e., it evolves according to deterministic neurodynamics. The proposed PNN is designed based on subgradient projection methods. The neural network has a simple structure, giving it a potential to be implemented as a large-scale analog circuit. It is proved that under appropriate conditions on the measurement matrix, every neuronal state of the proposed neural network is convergent to the optimal solution of the \(l_1\)-minimization problem under study. Simulation results are provided to substantiate the effectiveness of the proposed approach.

Introduction. Minimization of ravine convex functions, both smooth and non-smooth, arises in many problems of planning, control, stability analysis of dynamic systems, artificial intelligence, and machine learning. Therefore, the development of new and improvement of existing methods is an important task, taking into account the fact that more and more frequently the functions to be minimized depend on a large number of variables. Special attention should be paid to the methods using the operation of linear transformation of space, which allow to improve the properties of the objective function and significantly accelerate its minimization. Known methods of this type, in particular, ellipsoid methods and r-algorithms, require recalculation of the transformation matrix at each iteration. Therefore, the development of methods with a one-time transformation of space, which have a lower cost of iteration, is definitely an urgent task. The purpose of the article is to review modifications of subgradient method with Polyak’s step that use a scalar parameter m(1 and one-time space transformation operation. Supplement the justification of the convergence and convergence rate of the modifications described. Give recommendations regarding determination of parameter m and space transformation matrix B. Results. Subgradient method with Polyak’s step in transformed space and parameter m>1 is an effective method for minimizing smooth and non-smooth convex functions that have a ravine structure. Determination of an appropriate value of the parameter m(1 and space transformation matrix B allows to significantly accelerate this method and use it for minimization functions that depend on a large number of variables. Conclusions. The development of fast methods for minimization of non-smooth convex functions of many variables with a ravine structure makes it possible to effectively solve modern problems of artificial intelligence, in particular, the problems of machine learning, image recognition, big data analysis, etc. Keywords: subgradient method, space transformation, ravine functions.

This paper presents a neurodynamic model for solving ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> minimization problems for sparse signal reconstruction. The essence of the proposed approach lies in its capability to operate in continuous time, which enables it to outperform most existing iterative ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -solvers in dynamic environments. The model is described by a goal-seeking recurrent neural network and it evolves according to its deterministic neurodynamics. It is proved that the model globally converges to the optimal solution to the ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -minimization problem under study. The connection weights of the neural network model are determined by using subgradient projection methods and the activation function is designed based on subdifferential. Due to its simple structure, the hardware implementation of this neurodynamic model is viable and cost-effective, which sheds light on real-time sparse signal recovery via large scale ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> minimization formulations.

Subgradient methods (method A and method B) are considered for finding the minimum point of a convex function for the known optimal value of the function. Both methods use known Polyak’s step, which depends on scalar parameter m ≥ 1 and guarantees monotonous decrease of the distance to the minimum point. If m = 1 the methods A and B are applicable to arbitrary convex function. Parameter m > 1 permits to take into account special classes of convex functions – convex quadratic functions (m = 2), differentiable homogeneous of degree σ (m = σ), etc. For both methods theorems are proven about the convergence rate O 1/ √ k for arbitrary convex function and convergence with geometric progression rate for a convex function with acute minimum. Method A is a subgradient method with Polyak’s step in original space of variables. The parameter m = 1 corresponds to classical Polyak’s step for arbitrary convex function. Parameter m = 2 can be used for quadratic functions minimization, for which the value of Polyak’s step is doubled comparing to the classical Polyak’s step with m = 1. For onedimentional functions method A finds the minimum point in one step from the arbitrary start point, which corresponds to one iteration of the Newton’s method. Proof of theorems about convergence rate of the method A is based on convex function characteristics in original space of variables, among which a constant c1 plays a crucial role – it bounds a subgradient norm. Method B is a subgradient method with space transformation, where the Polyak’s step is calculated in space transformed by linear operator. The method is defined by nonsingular matrix B. The parameter m = 1 corresponds to classical Polyak’s step for arbitrary convex function in transformed space of variables. If m = 2, for one-dimentional quadratic function, the method B finds the minimum point in one step with both arbitrary matrix B and start point. Proof of theorems about convergence rate of the method B is based on function characteristics in transformed space of variables, which are analogous to the method A characteristics in original space.

In this research, for a given time interval, which is the general period of melanoma treatment, a bilinear control model is considered, given by a system of differential equations, which describes the interaction between drug-sensitive and drug-resistant cancer cells both during drug therapy and in the absence of it. This model also contains a control function responsible for the transition from the stage of such therapy to the stage of its absence and vice versa. To find the optimal moments of switching between these stages, the problem of minimizing the cancer cells load both during the entire period of melanoma treatment and at its final moment is stated. Such a minimization problem has a nonconvex control set, which can lead to the absence of an optimal solution to the stated minimization problem in the classes of admissible modes traditional for applications. To avoid this problem, the control set is imposed to be convex. As a result, a relaxed minimization problem arises, in which the optimal solution exists. An analytical study of this minimization problem is carried out using the Pontryagin maximum principle. The corresponding optimal solution is found in the form of synthesis and may contain a singular arc. It shows that there are values of the parameters of the bilinear control model, its initial conditions, and the time interval for which the original minimization problem does not have an optimal solution, because it has a sliding mode. Then for such values it is possible to find an approximate optimal solution to the original minimization problem in the class of piecewise constant controls with a predetermined number of switchings. This research presents the results of the analysis of the connection between such an approximate solution of the original minimization problem and the optimal solution of the relaxed minimization problem based on numerical calculations performed in the Maple environment for the specific values of the parameters of the bilinear control model, its initial conditions, and the time interval.

The work proves that under conditions of instability of the second derivatives of the function in the minimization region, the estimate of the convergence rate of Newton’s method is determined by the parameters of the irreducible part of the conditionality degree of the problem. These parameters represent the degree of difference between eigenvalues of the matrices of the second derivatives in the coordinate system, where this difference is minimal, and the resulting estimate of the convergence rate subsequently acts as a standard. The paper studies the convergence rate of the relaxation subgradient method (RSM) with optimization of the parameters of two-rank correction of metric matrices on smooth strongly convex functions with a Lipschitz gradient without assumptions about the existence of second derivatives of the function. The considered RSM is similar in structure to quasi-Newton minimization methods. Unlike the latter, its metric matrix is not an approximation of the inverse matrix of second derivatives but is adjusted in such a way that it enables one to find the descent direction that takes the method beyond a certain neighborhood of the current minimum as a result of one-dimensional minimization along it. This means that the metric matrix enables one to turn the current gradient into a direction that is gradient-consistent with the set of gradients of some neighborhood of the current minimum. Under broad assumptions on the parameters of transformations of metric matrices, an estimate of the convergence rate of the studied RSM and an estimate of its ability to exclude removable linear background are obtained. The obtained estimates turn out to be qualitatively similar to estimates for Newton’s method. In this case, the assumption of the existence of second derivatives of the function is not required. A computational experiment was carried out in which the quasi-Newton BFGS method and the subgradient method under study were compared on various types of smooth functions. The testing results indicate the effectiveness of the subgradient method in minimizing smooth functions with a high degree of conditionality of the problem and its ability to eliminate the linear background that worsens the convergence.

In this paper we study general $$l_p$$ l p regularized unconstrained minimization problems. In particular, we derive lower bounds for nonzero entries of the first- and second-order stationary points and hence also of local minimizers of the $$l_p$$ l p minimization problems. We extend some existing iterative reweighted $$l_1$$ l 1 ( $$\mathrm{IRL}_1$$ IRL 1 ) and $$l_2$$ l 2 ( $$\mathrm{IRL}_2$$ IRL 2 ) minimization methods to solve these problems and propose new variants for them in which each subproblem has a closed-form solution. Also, we provide a unified convergence analysis for these methods. In addition, we propose a novel Lipschitz continuous $${\epsilon }$$ ∈ -approximation to $$\Vert x\Vert ^p_p$$ ? x ? p p . Using this result, we develop new $$\mathrm{IRL}_1$$ IRL 1 methods for the $$l_p$$ l p minimization problems and show that any accumulation point of the sequence generated by these methods is a first-order stationary point, provided that the approximation parameter $${\epsilon }$$ ∈ is below a computable threshold value. This is a remarkable result since all existing iterative reweighted minimization methods require that $${\epsilon }$$ ∈ be dynamically updated and approach zero. Our computational results demonstrate that the new $$\mathrm{IRL}_1$$ IRL 1 method and the new variants generally outperform the existing $$\mathrm{IRL}_1$$ IRL 1 methods (Chen and Zhou in 2012; Foucart and Lai in Appl Comput Harmon Anal 26:395---407, 2009).

The study of the stability of m-dimensional systems is a difficult one especially when m /spl ges/ 3. There exist only a few results in the literature and unfortunately, there does not exist any practical criterion. In this work, the stability of an m-dimensional system is dealt as a minimization problem of the absolute value of its characteristic polynomial over the boundaries of its variables (i.e., on the m unit circles). This minimization problem is solved by using genetic algorithms (GAs). Using GAs we obtain, in general, better results than other methods of minimization (numerical techniques, neural networks, etc.). Numerical examples are presented.

In this paper, we introduce an inertial parallel CQ subgradient extragradient method for finding a common solutions of variational inequality problems. The novelty of this paper is using linesearch methods to find unknown L constant of L-Lipschitz continuous mappings. Strong convergence theorem has been proved under some suitable conditions in Hilbert spaces. Finally, we show applications to signal and image recovery, and show the good efficiency of our proposed algorithm when the number of subproblems is increasing.

A new inertial condition on the subgradient extragradient method for solving pseudomonotone equilibrium problem

We consider the resource allocation problem and its numerical solution. The following constructions are demonstrated: 1) Walrasian price-adjustment mechanism for determining the equilibrium; 2) Decentralized role of the prices; 3) Slater's method for price restrictions (dual Lagrange multipliers); 4) A new mechanism for determining equilibrium prices, in which prices are fully controlled not by Center (Government), but by economic agents -- nodes (factories). In economic literature the convergence of the considered methods is only proved. In contrast, this paper provides an accurate analysis of the convergence rate of the described procedures for determining the equilibrium. The analysis is based on the primal-dual nature of the suggested algorithms. More precisely, in this article we propose the economic interpretation of the following numerical primal-dual methods of convex optimization: dichotomy and subgradient projection method. Numerical experiments conclude the paper.

Global Error Minimization method for solving strongly nonlinear oscillator differential equations

Most of the time, theoreticians first design a new minimization method and, then, seek for (real-world) applications. Our situation is different: we first formulate an astronomical problem as a minimization problem and, then, we seek for the most suitable algorithm (among the existing ones) to get the minimum.KeywordsLocal SearchSimulated AnnealingRadial VelocityOrbit DeterminationOrbital ParameterThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Multiplicative noise and blur removal problems have attracted much attention in recent years. In this paper, we propose an efficient minimization method to recover images from input blurred and multiplicative noisy images. In the proposed algorithm, we make use of the logarithm to transform blurring and multiplicative noise problems into additive image degradation problems, and then employ l 1-norm to measure in the data-fitting term and the total variation to measure the regularization term. The alternating direction method of multipliers (ADMM) is used to solve the corresponding minimization problem. In order to guarantee the convergence of the ADMM algorithm, we approximate the associated nonconvex domain of the minimization problem by a convex domain. Experimental results are given to demonstrate that the proposed algorithm performs better than the other existing methods in terms of speed and peak signal noise ratio.

This paper deals with the implementation of stochastic process coupling with classical minimization methods in order to solve an identification problem. A model for an industrial process (elastomer vulcanization) is proposed and then the minimization problems and methods are presented. The effect of random perturbations on these methods is studied. Numerical results are given. The interest of the coupling of stochastic and deterministic methods is shown. INTRODUCTION We are interested in a situation issued from thermal industry: the vulcanization. Gamier [1] has shown that, while the curing of some elastomers, an exothermical reaction can occur and damage the material. In order to avoid this destruction, a sharp control is essential and a fine modelling of the physical phenomena taking place during the industrial process is important. The identification of some of the elastomer's parameters must be performed by means of physical experiments (differential scanning calorimetry,...) or numerical methods using temperature measurements can be carried out. 1 A MODEL FOR THE INDUSTRIAL PROCESS In this section, we present a model for the curing of an elastomer. In the industrial process, the natural rubber is injected under pressure in a parallelepipedic cell heated by the top and the bottom (see figure la). Heat flux is supposed equal to 0 on the other faces. This 3-D geometry involved is simplified to 1-D (see figure Ib). Transactions on Engineering Sciences vol 5, © 1994 WIT Press, www.witpress.com, ISSN 1743-3533