Projection-based optimal mode scheduling

Within the time-dependent formulation of the projection-operator theory of resonant electron-molecule scattering, systematic corrections to the local complex potential approximation can be derived. In this work the performance of the first-order corrected local approximation is assessed in comparison with the standard local approximation and exact results for two representative models of electron-molecule shape resonances. The results shed light on the importance of short-time nonexponential decay effects in resonant electron-molecule scattering. The first-order corrected local theory is shown to yield more accurate results than the conventional local approximation, without requiring additional computational effort.

Many signal and image estimation problems such as maximum entropy reconstruction and positron emission tomography, require the minimization of a criterion containing a barrier function i.e., an unbounded function at the boundary of the feasible solution domain. This function has to be carefully handled in the optimization algorithm. When an iterative descent method is used for the minimization, a search along the line supported by the descent direction is usually performed at each iteration. However, standard line search strategies tend to be inefficient in this context. In this paper, we propose an original line search algorithm based on the majorize-minimize principle. A tangent majorant function is built to approximate a scalar criterion containing a barrier function. This leads to a simple line search ensuring the convergence of several classical descent optimization strategies, including the most classical variants of nonlinear conjugate gradient. The practical efficiency of the proposal scheme is illustrated by means of two examples of signal and image reconstruction.

For engineering design optimization, the full-space formulation offers the potential for greater efficiency than the more commonly used reduced-space formulation. This potential is greater when the numerical model involves discretized partial differential equations or coupled disciplines. However, the full-space formulation results in a larger optimization problem with at least a factor of two increase in the number of optimization variables and equality constraints. Using Newton-type methods to solve such problems involves solving a large-scale and, often, ill-conditioned Karush–Kuhn–Tucker linear system at each optimization iteration. This can be time-consuming to solve even with a Krylov solver. If the number of iterations is reduced, the full-space formulation could be applied to a broader class of problems. This paper presents an inexact quasi-Newton algorithm with an adaptive extension for solving large-scale equality-constrained optimization problems. The new algorithm inexactly solves the Karush–Kuhn–Tucker system using new inexactness criteria that are derived to ensure a descent direction. The adaptive extension chooses the stopping condition of the Krylov solver by also taking its convergence rate into account. The paper presents results of numerical experiments applying this algorithm to three types of problems: six constrained optimization problems from the widely used CUTEst test suite, a bar thickness optimization problem, and a two-dimensional topology optimization problem. For all problems, the new algorithm consistently shows a roughly 50% reduction in the total number of Krylov solver iterations and a minimum of roughly 15% reduction in the optimization time. Moreover, the proposed approach for selecting the Krylov solver tolerance shows an improvement in all cases, whereas the existing forcing-parameter approach shows an increase in the number of Krylov iterations in some cases. These results indicate that this new method for selecting solver tolerances is effective and robust, and a good choice in algorithms that use a Krylov solver for solving the Karush–Kuhn–Tucker linear system.

In this work a new technique of line search is proposed to solve the nonlinear monotone equations. For this purpose we combine a modified line search rule with monotone technique. The new proposed algorithm can decrease the CPU time, the number of iterations and the functions evaluations and can increase the efficiency of the approach. The global convergence result of the recommended procedures is established under some standard conditions. Preliminary numerical results indicate that the proposed algorithm are interesting and remarkably promising.

To minimize the continuous and differentiable objective function f, the gradient descent, a first-order iterative method, can provide a sequence number {x_i} that converges to the minimum. The iteration can be written as x_(i+1) = x_i - t ∇f_i where t is a learning rate and i is an iterative number. However, the value of the learning rate affects the whole algorithm. If the value is too small, the sequence {x_i} will slowly converge, and if the value is too large, the sequence {x_i} will oscillate or diverge. To prevent this problem, the learning rate t is adjusted for each iteration i. Therefore, the problem is reduced to minimize f ̃(t_i ) = f(x_i - t_i ∇f_i). The line search method will choose the optimal t_i be the learning rate in iteration i. In this project, we improve the backtracking line search by choosing a smaller learning rate for gradient descent based on the previous iteration. After the experiment, the modified method performs as well as the standard method for simple functions and better in more complex functions in terms of running time and number of iterations.

This paper presents a logarithmic barrier method without line search for solving linear programming problem. The descent direction is the classical Newton's one. However, the displacement step is determined by a simple and efficient technique based on the notion of the minorant function approximating the barrier function. Numerical tests show the efficiency of this approach versus classical line search methods.

Quantum optimal control is an important technology that enables fast state preparation and gate design. In the absence of an analytic solution, most quantum optimal control methods rely on an iterative scheme to update the solution estimate. At present, the convergence rate of existing solvers is at most superlinear. This paper develops a new general purpose solver for quantum optimal control based on the PRojection Operator Newton method for Trajectory Optimization, or PRONTO. Specifically, the proposed approach uses a projection operator to incorporate the Schr\"odinger equation directly into the cost function, which is then minimized using a quasi-Newton method. At each iteration, the descent direction is obtained by computing the analytic solution to a Linear-Quadratic trajectory optimization problem. The resulting method guarantees monotonic convergence at every iteration and quadratic convergence in proximity of the solution. To highlight the potential of PRONTO, we present an numerical example that employs it to solve the optimal state-to-state mapping problem for a qubit and compares its performance to a state-of-the-art quadratic optimal control method.

The Barzilai-Borwein (BB) method is a popular and efficient tool for solving large-scale unconstrained optimization problems.Its search direction is the same as for the steepest descent (Cauchy) method, but its stepsize rule is different.Owing to this, it converges much faster than the Cauchy method.A feature of the BB method is that it may generate too long steps, which throw the iterates too far away from the solution.Moreover, it may not converge, even when the objective function is strongly convex.In this paper, a stabilization technique is introduced.It consists in bounding the distance between each pair of successive iterates, which often allows for decreasing the number of BB iterations.When the BB method does not converge, our simple modification of this method makes it convergent.For strongly convex functions with Lipschits gradients, we prove its global convergence, despite the fact that no line search is involved, and only gradient values are used.Since the number of stabilization steps is proved to be finite, the stabilized version inherits the fast local convergence of the BB method.The presented results of extensive numerical experiments show that our stabilization technique often allows the BB method to solve problems in a fewer iterations, or even to solve problems where the latter fails.

The neural networks offer iteration capability for low-density parity-check (LDPC) decoding with superior performance at transmission. However, to cope with increasing code length and rate, the complexity of the neural network increases significantly. This is due to the large amount of feature extraction required to maintain the error correction capability. Based on this gap, we design a new iterative LDPC decoding technique named graph model neural network-belief propagation (GMNN-BP). GMNN-BP uses graph models as a link between deep learning and belief propagation (BP) algorithms, combining the advantages of both. Compared to traditional fully connected neural network decoders, the GMNN-BP decoding has the substantial benefit of avoiding learning and judging codeword categories directly from a large amount of data and requiring less training data as well. The proposed algorithm is verified by simulation and experiment and is tested by using IEEE 802.3ca standard LDPC code word. The results show that the GMNN-BP decoding algorithm is superior to the BP-based iterative decoding method under the same number of iterations, and the maximum gain can reach 1.9dB. When achieving the same performance, the GMNN-BP decoding algorithm only requires half the number of iterations of other algorithms.

In this chapter, an implementation of an interior point filter line-search algorithm for large-scale nonlinear programming proposed by Wachter and Biegler (2005a, b) is presented. As we know, to allow convergence from poor starting points and to enforce progress to the solution, interior point methods both in trust region and in line-search frameworks with exact penalty merit function have been developed. For example, KNITRO uses the l 1 exact penalty function (Byrd, Hribar, & Nocedal, 1999; Byrd, Gilbert, & Nocedal, 2000). On the other hand, Fletcher and Leyffer (2002) proposed filter methods as an alternative to merit functions, as a tool for global convergence guarantee in algorithms for nonlinear optimization. The idea of filter is that trial points are accepted if they improve the objective function value or improve the constraint violation instead of a combination of these two measures defined by the merit function. Even if the filter methods include different heuristics, they have been adapted to barrier methods in a number of ways. For example, Ulbrich et al. (2004) considered a trust-region filter method and accept the trial step on the basis of the norm of the optimality conditions. Benson et al. (2002a) proposed several heuristics using the idea of filter methods, for which improved efficiency is reported compared to their previous merit function approach. The global convergence of an interior point algorithm with a filter line search was analyzed by Wachter and Biegler (2001). Here, the assumptions made for the analysis of the global convergence are less restrictive than those made for line-search interior point methods for nonlinear programming developed, for example, by El-Bakry et al. (1996), Yamashita (1998), or Tits et al. (2003).

In numerical analysis, the successive over-relaxation method, an iterative technique that is a variation of the Gauss-Seidel technique, is applied to determine the solutions of systems of linear equations Ax=b. The main purpose of this study is to statistically analyze this method by filling the coefficient matrix A and the vector b with random numbers from different probability distributions- uniform inputs - both discrete uniform U[1, 2, 3,…,k] and continuous uniform U[0, θ] inputs are considered and non-uniform inputs, random numbers simulated from the binomial B(n, p) distribution. The values of the parameters are gradually varied during the course of the study. We also changed the size of the matrix as well to see if that has any effect on the overall outcome. There is nothing in the data to suggest that the number of iterations depends upon the parameter in case of uniform inputs, whether discrete or continuous. However, in the case of binomial distribution, a definite trend is observed when the value of parameter p is fixed and n is varied, and also when n is fixed and p is varied. Regression analysis further shows that the number of iterations can be estimated by a third degree polynomial in p for fixed n and another third degree polynomial in n when p is fixed.

We present a method to transmit digital information through a highly scattering medium in a MIMO-MU (multiple input multiple output multiple users) context. It is based on iterations of a time-reversal process, and permits us to focus short pulses, both spatially and temporally, from a base antenna to different users. This iterative technique is shown to be more efficient (lower inter-symbol interference and lower error rate) than classical time-reversal communication, while being computationally light and stable. Experiments are presented: digital information is conveyed from 15 transmitters to 15 receivers by ultrasonic waves propagating through a highly scattering slab. From a theoretical point of view, the iterative technique achieves the inverse filter of propagation in the subspace of non-null singular values of the time-reversal operator. We also investigate the influence of external additive noise, and show that the number of iterations can be optimized to give the lowest error rate. (Some figures in this article are in colour only in the electronic version)

In complex real-world environments, traditional (tabular) techniques for solving Reinforcement Learning (RL) do not scale. Function approximation is needed, but unfortunately, existing approaches generally have poor convergence and optimality guarantees. Additionally, for the case of human environments, it is valuable to be able to leverage human input. In this paper we introduce Expanding Value Function Approximation (EVFA), a function approximation algorithm that returns the optimal value function given sufficient rounds. To leverage human input, we introduce a new human-agent interaction scheme, training regimens, which allow humans to interact with and improve agent learning in the setting of a machine learning game. In experiments, we show EVFA compares favorably to standard value approximation approaches. We also show that training regimens enable humans to further improve EVFA performance. In our user study, we find that non-experts are able to provide effective regimens and that they found the game fun.

Recent works in the boundary element method (BEM) community have been devoted to the derivation of fast techniques to perform the matrix-vector product needed in the iterative solver. Fast BEMs are now very mature. However, it has been shown that the number of iterations can significantly hinder the overall efficiency of fast BEMs. The derivation of robust preconditioners is now inevitable to increase the size of the problems that can be considered. Analytical preconditioners offer a very interesting strategy by improving the spectral properties of the boundary integral equations ahead of discretization. The main contribution of this paper is to propose new analytical preconditioners to treat Neumann exterior scattering problems in 2D and 3D elasticity. These preconditioners are local approximations of the adjoint Neumann-to-Dirichlet map. We propose three approximations with different orders. The resulting boundary integral equations are preconditioned Combined Field Integral Equations (CFIEs). An analytical spectral study confirms the expected behavior of the preconditioners, i.e., a better eigenvalue clustering especially in the elliptic part contrary to the standard CFIE of the first-kind. We provide various 2D numerical illustrations of the efficiency of the method for smooth and non smooth geometries. In particular, the number of iterations is shown to be independent of the density of discretization points per wavelength which is not the case for the standard CFIE. In addition, it is less sensitive to the frequency.

In this paper, a technique is introduced by which to extend the applicability of the existing analytic solutions of electromagnetic field problems to cases where random-noisy-sampled data (such as measurement outputs) are available, rather than analytic input functions. We address those problems for which a theoretical solution exists in the form of a superposition of some basis functions. The algorithm introduced employs this same set of basis functions, and finds the expansion coefficients by the use of an iterative error-minimization technique, which resembles those found in the process of training of artificial neural networks. In cases where the expansion functions are orthonormal, guaranteed fast convergence is proved. As well, we show how neuro-computation techniques can be employed to circumvent the effects of various types of measurement errors and noise. Satisfactory performance of the algorithm is shown for a test problem driven by random inputs corrupted with various levels of Gaussian noise. >