A Recursive Descent Algorithm for Finding the Optimal Minimax Piecewise Linear Approximation of Convex Functions

Residual gradient (RG) was proposed as an alternative to TD(0) for policy evaluation when function approximation is used, but there exists little formal analysis comparing them except in very limited cases. This paper employs techniques from online learning of linear functions and provides a worst-case (non-probabilistic) analysis to compare these two types of algorithms when linear function approximation is used. No statistical assumptions are made on the sequence of observations, so the analysis applies to non-Markovian and even adversarial domains as well. In particular, our results suggest that RG may result in smaller temporal differences, while TD(0) is more likely to yield smaller prediction errors. These phenomena can be observed even in two simple Markov chain examples that are non-adversarial.

The approximation of functions and linear functionals: best vs. good approximation

We present an extended mixed-integer programming formulation of the stochastic lot-sizing problem for the static-dynamic uncertainty strategy. The proposed formulation is significantly more time efficient as compared to existing formulations in the literature and it can handle variants of the stochastic lot-sizing problem characterized by penalty costs and service level constraints, as well as backorders and lost sales. Also, besides being capable of working with a predefined piecewise linear approximation of the cost function—as is the case in earlier formulations—it has the functionality of finding an optimal cost solution with an arbitrary level of precision by means of a novel dynamic cut generation approach. The online appendix is available at https://doi.org/10.1287/ijoc.2017.0792 .

The task of simultaneous localization and mapping (SLAM) is to build environmental map and locate the position of mobile robot at the same time. FastSLAM 2.0 is one of powerful techniques to solve the SLAM problem. However, there are two obvious limitations in FastSLAM 2.0, one is the linear approximations of nonlinear functions which would cause the filter inconsistent and the other is the “particle depletion” phenomenon. A kind of PSO & H∞-based FastSLAM 2.0 algorithm is proposed. For maintaining the estimation accuracy, H∞ filter is used instead of EKF for overcoming the inaccuracy caused by the linear approximations of nonlinear functions. The unreasonable proposal distribution of particle greatly influences the pose state estimation of robot. A new sampling strategy based on PSO (particle swarm optimization) is presented to solve the “particle depletion” phenomenon and improve the accuracy of pose state estimation. The proposed approach overcomes the obvious drawbacks of standard FastSLAM 2.0 algorithm and enhances the robustness and efficiency in the parts of consistency of filter and accuracy of state estimation in SLAM. Simulation results demonstrate the superiority of the proposed approach.

The goal of this paper is to investigate linear approximations of random functions and permutations. Our motivation is twofold. First, before the distinguishability of a practical cipher from an ideal one can be analysed, the cryptanalyst must have an accurate understanding of the statistical behaviour of the ideal cipher. Secondly, this issue has been neglected both in old and in more recent studies, particularly when multiple linear approximations are being used simultaneously. Traditional models have been based on the average behaviour and simplified using other assumptions such as independence of the linear approximations. Multidimensional cryptanalysis was introduced to avoid making artificial assumptions about statistical independence of linear approximations. On the other hand, it has the drawback of including many trivial approximations that do not contribute to the attack but just cause a waste of time and memory. We show for the first time in this paper that the trivial approximations reduce the degree of freedom of the related χ2 distribution. Previously, the affine linear cryptanalysis was proposed to allow removing trivial approximations and, at the same time, admitting a solid statistical model. In this paper, we identify another type of multidimensional linear approximation, called Davies-Meyer approximation, which has similar advantages, and present full statistical models for both the affine and the Davies-Meyer type of multidimensional linear approximations. The new models given in this paper are realistic, accurate and easy to use. They are backed up by standard statistical tools such as Pearson’s χ2 test and finite population correction and demonstrated to work accurately using practical examples.

This paper presents an approach to combinatorial optimization problems using linear function approximation (LFA) to solve the Travelling Salesman Problem (TSP). We create a reinforcement learning model in which we parameterize our policy using linear function approximation instead of the more commonly used neural networks. We then evaluated our models based on two factors: training time and optimality. When we compared our results with a state-of-the-art neural network solver, we found that our model was able to solve the TSP accurately while using drastically less computational resources and time to train than the neural network algorithm (Kool et al., 2019).

The term “nexting” has been used by psychologists to refer to the propensity of people and many other animals to continually predict what will happen next in an immediate, local, and personal sense. The ability to “next” constitutes a basic kind of awareness and knowledge of one’s environment. In this paper we present results with a robot that learns to next in real time, predicting thousands of features of the world’s state, including all sensory inputs, at timescales from 0.1 to 8 seconds. This was achieved by treating each state feature as a reward-like target and applying temporal-difference methods to learn a corresponding value function with a discount rate corresponding to the timescale. We show that two thousand predictions, each dependent on six thousand state features, can be learned and updated online at better than 10Hz on a laptop computer, using the standard TD(λ) algorithm with linear function approximation. We show that this approach is efficient enough to be practical, with most of the learning complete within 30 minutes. We also show that a single tile-coded feature representation suffices to accurately predict many different signals at a significant range of timescales. Finally, we show that the accuracy of our learned predictions compares favorably with the optimal off-line solution.

In the paper the differential and the linear approximations of two classes of S-box functions are considered. The classes are the permutations and arbitrary functions with n binary inputs and m binary outputs, where 1 ≤ n=m ≤10. For randomly chosen functions from each of the classes, the distribution of the best nonzero approximations is investigated. The based on the definitions of differential and linear approximation algorithms to compute a single element of the approximation tables, are of exponential complexity. The presented in the paper fast algorithms compute the best nonzero approximations in at worst linear time for a single element, without memory needed for storage of the whole table.

Converting general nonlinear programming problems into separable programming problems with feedforward neural networks

The competitive associative nets called CAN2 involve competitive and associative schemes for learning to achieve piecewise linear approximation of nonlinear functions. Although the learning schemes of the nets have been shown effective in many applications such as function approximation, control, and rainfall estimation, the competitive learning basically has local minima problems. To overcome the problems, we here introduce asymptotic situation, where the number of units are very large, and then derive a condition of asymptotic optimality for minimizing the mean square error (MSE) of approximation. We next embed the condition into the incremental learning algorithm, where the condition is used for deciding whether the learning process is stuck at a local minimum or not, and reinitializing a unit for global optimum. By means of numerical experiments with a number of benchmark functions, we have verified the CAN2 with the present algorithm achieves smaller MSE than the widely used schemes using learning or optimization techniques as follows: the BPN (backpropagation net), the RBFN (radial basis function net) and the SVR (support vector regression).

Energy harvesting (EH) two-hop communications are considered. The transmitter and the relay harvest energy from the environment and use it exclusively for transmitting data. A data arrival process is assumed at the transmitter. At the relay, a finite data buffer is used to store the received data. We consider a realistic scenario in which the EH nodes have only local causal information, i.e., at any time instant, each EH node only knows the current value of its EH process, channel state and data arrival process. Our goal is to find a power allocation policy to maximize the throughput at the receiver. We show that because the EH nodes have local causal information, the two-hop communication problem can be separated into two point-to-point problems. Consequently, independent power allocation problems are solved at each EH node. To find the power allocation policy, reinforcement learning with linear function approximation is applied. Moreover, to perform function approximation two feature functions which consider the data arrival process are introduced. Numerical results show that the proposed approach has only a small degradation as compared to the offline optimum case. Furthermore, we show that with the use of the proposed feature functions a better performance is achieved compared to standard approximation techniques.

In reinforcement learning, when the state space is enormous or infinite, it is not feasible to find the exact value for each state in the memory. A common way to tackle this problem is to adopt linear value function approximation technique. In this paper, we review some commonly used linear value function approximation algorithms, which divided into two classes. The first class is the so-called on-policy algorithms that mainly include temporal difference and least-squares-based methods. The second is the off-policy algorithms that use two strategies to solve the exploration-exploitation problem. From the perspective of algorithm design, on-policy is simple, but it does not take both exploration and exploitation into consideration when collecting data, while the off-policy can well balance the relation between exploration and exploitation. In the end, we first present some experiments to demonstrate the usefulness of linear value function approximation technique in the continuous and high dimensional scenes, and then in-class and out-class experiments are respectively conducted to further illustrate the performance of different approximation algorithms.KeywordsReinforcement learningLinear value function approximationLeast-squares temporal difference

In this study, we investigate the problem of min-max continuous optimization in a black-box setting $\min_{x} \max_{y}f(x,y)$. A popular approach updates $x$ and $y$ simultaneously or alternatingly. However, two major limitations have been reported in existing approaches. (I) As the influence of the interaction term between $x$ and $y$ (e.g., $x^\mathrm{T} B y$) on the Lipschitz smooth and strongly convex-concave function $f$ increases, the approaches converge to an optimal solution at a slower rate. (II) The approaches fail to converge if $f$ is not Lipschitz smooth and strongly convex-concave around the optimal solution. To address these difficulties, we propose minimizing the worst-case objective function $F(x)=\max_{y}f(x,y)$ directly using the covariance matrix adaptation evolution strategy, in which the rankings of solution candidates are approximated by our proposed worst-case ranking approximation (WRA) mechanism. Compared with existing approaches, numerical experiments show two important findings regarding our proposed method. (1) The proposed approach is efficient in terms of $f$-calls on a Lipschitz smooth and strongly convex-concave function with a large interaction term. (2) The proposed approach can converge on functions that are not Lipschitz smooth and strongly convex-concave around the optimal solution, whereas existing approaches fail.

Approximation and regularization of arbitrary functions in Hilbert spaces by the Lasry-Lions method

More on piecewise linear approximation