Second-Order Consensus Filtering Algorithm

Communication-efficient distributed cubic Newton with compressed lazy Hessian

In this paper we study lthorder (l >= 3) consensus algorithms, which generalize the existing first-order and second-order consensus algorithms in the literature. We will show sufficient conditions under which each information variable and their higher-order derivatives converge to common values. We will present the idea of higher-order consensus with a leader and introduce the concept of an lthorder model-reference consensus problem, where each information variable and their high-order derivatives not only reach consensus but also converge to the solution of a prescribed dynamic model. The effectiveness of these algorithms are demonstrated through simulations and a multi-vehicle cooperative control application which mimics flocking behavior in birds.

In order to improve the accuracy and the convergence rate of distributed consensus under quantized communication, in the paper, based on adaptive quantization scheme, we propose the fast second-order distributed consensus algorithm. We analyze the convergence performance. The fast second-order distributed consensus with adaptive quantization achieves a consensus in a mean square sense, and the consensus is equal to the average of the initial states. Simultaneously, Simulations are done about the fast second-order distributed consensus based on adaptive quantization. Results show that the fast second-order distributed consensus algorithm based on adaptive quantization can reach an average consensus, and its convergence rate is higher than that of the first-order adaptive quantized distributed consensus algorithm, moreover, the mean square errors are smaller within the finite steps.

The dynamic programming formulation of the forward principle of optimality in the solution of optimal control problems results in a partial differential equation with initial boundary condition whose solution is independent of terminal cost and terminal constraints. Based on this property, two computational algorithms are described. The first-order algorithm with minimum computer storage requirements uses only integration of a system of differential equations with specified initial conditions and numerical minimization in finite-dimensional space. The second-order algorithm is based on the differential dynamic programming approach. Either of the two algorithms may be used for problems with nondifferentiable terminal cost or terminal constraints, and the solution of problems with complicated terminal conditions (e.g., with free terminal time) is greatly simplified.

In this paper, we extend the consensus algorithm for double integrator dynamics to the case that the information exchange topologies switch randomly with time and to the case that the final consensus value evolves according to a given nonlinear reference model. We show sufficient conditions under which consensus is reached under switching directed information exchange topologies. Unlike the consensus algorithm for single integrator dynamics, more stringent conditions are required to guarantee consensus under switching directed topologies in the case of the consensus algorithm for double integrator dynamics. In addition, we propose consensus algorithms so that the information variables of each vehicle approach the solution of a nonlinear reference model when only a portion of the vehicles in the team have access to the model.

In this paper, we propose a simple consensus algorithm for systems modeled by second-order dynamics, and derive necessary and sufficient conditions under which consensus can be reached based on algebraic graph theory and matrix analysis. We apply the simple second-order consensus algorithm to tackle formation control problems. The second-order consensus protocol and a multi-vehicle system formation control example are shown in simulation to illustrate our strategies.

This article investigates a second-order multi-agent distributed optimisation algorithm on matrix-weighted networks. To address the limitations of traditional methods on matrix-weighted networks, a novel second-order distributed optimisation algorithm is proposed. By constructing auxiliary variables, the algorithm eliminates the requirement for velocity information updates. Under the proposed control protocol, algebraic and graph-theoretic conditions for achieving optimisation consensus are established based on matrix theory and Barbalat's lemma. Specifically, the given parameter conditions and local cost functions are convex; the second-order consensus at the global optimal solution is achieved when the Laplacian matrix's null space spans the consensus subspace or the matrix-weighted graph contains a positive spanning tree. Finally, a numerical experiment is conducted to validate the effectiveness of proposed algorithms.

In a letter to Robert Hooke, Isaac Newton drew the orbit of a mass moving under a constant attracting central force. The drawing of the orbit may indicate how and when Newton developed dynamic categories. Some historians claim that Newton used a method contrived by Hooke; others that he used some method of curvature. We prove that Hooke’s method is a second-order symplectic area-preserving algorithm, and the method of curvature is a first-order algorithm without special features; then we integrate the Hamiltonian equations. Integration by the method of curvature can also be done, exploring the geometric properties of curves. We compare three methods: Hooke’s method, the method of curvature and a first-order method. A fourth-order algorithm sets a standard of comparison. We analyze which of these methods best explains Newton’s drawing.

The iterative extended Kalman filter (IEKF) algorithm is dependent on the previous data and estimation accuracy of the noise. The noise will result in filtering precision decrease and divergence. This paper puts forward an adaptive algorithm based on Levenberg-Marquardt method and two-factor for IEKF, which optimizes the iterative process by using Levenberg-Marquardt method, adjusts the prediction covariance by suboptimal fading factor. A simplified calculation method of suboptimal fading factor is also presented. The previous process noise is suppressed by exponential fading factor, and the observation noise is corrected by the residual and innovation, so that filter gain becomes larger, which improves the accuracy and robustness of the filter algorithm. The experiment results indicate the improved algorithm can effectively suppress the filter divergence. Compared with the classical extended Kalman filter (EKF), IEKF, and the modified IEKF, the filter accuracy and robustness of the improved algorithm are sufficient, and the algorithm can better meet the requirements of engineering applications.

Functional to anatomical brain image registration is needed for accurate localization of brain activation maps. Due to the presence of nonlinear distortions, it is more effective to consider non-rigid transformations to achieve such a registration. In this paper, a non-rigid registration technique based on the B-spline free-form deformation model and mutual information similarity measure is introduced. An optimization formulation is devised to achieve a fast and robust registration. This formulation differs from the previous formulations by utilizing a limited-memory, second-order optimization algorithm instead of the usual first-order gradient-based algorithms. It also enforces hard parameter constraints instead of constraints based upon physics or Jacobian smoothness. The results obtained indicate that this registration technique provides improvements over rigid and affine techniques when registering functional to anatomical magnetic resonance brain images.

This paper mainly introduces several kinds of global conformal parameterization theories and algorithms, which can deal with complex surfaces under different genus. Pure first-order differential algorithm and pure second-order differential algorithm are based on the contents of global differential geometry, whose core ideas come from Riemann's single-valued theorem and Teichmuller theory. Ricci flow algorithm and quasi-conformal mapping algorithm are based on the contents of the field of geometric partial differential equations, which can be optimized by minimizing energy to get results. The optimal transport algorithm is based on convex differential geometry and geometric variational principle, which can be transformed into computational geometry algorithm to find the solution.

The paper proposes and develops new globally convergent algorithms of the generalized damped Newton type for solving important classes of nonsmooth optimization problems. These algorithms are based on the theory and calculations of second-order subdifferentials of nonsmooth functions with employing the machinery of second-order variational analysis and generalized differentiation. First we develop a globally superlinearly convergent damped Newton-type algorithm for the class of continuously differentiable functions with Lipschitzian gradients, which are nonsmooth of second order. Then we design such a globally convergent algorithm to solve a structured class of nonsmooth quadratic composite problems with extended-real-valued cost functions, which typically arise in machine learning and statistics. Finally, we present the results of numerical experiments and compare the performance of our main algorithm applied to an important class of Lasso problems with those achieved by other first-order and second-order optimization algorithms.

To address the convergence issues in the natural occupation optimization of reduced density matrix functional theory (RDMFT), we recently proposed the explicit-by-implicit (EBI) idea to handle the ensemble N-representability constraint (Yao et al. J. Phys. Chem. Lett. 2021, 12, 6788). This work continues to focus on these issues that can affect the reliability of the electronic structure description in RDMFT; further explores the combination of EBI, as well as the (augmented) Lagrangian methods (both LM and ALM), with both first- and second-order numerical optimization algorithms; and carefully evaluates their performances in natural occupation optimizations of various systems, including strongly correlated systems and large molecules. By comparing both converged energies and elapsed times, it can be seen that the LM and ALM have serious convergence issues for systems of different sizes. In contrast, the optimizations of EBI can converge to better energies with fewer iterations. However, due to the local convergence nature of the Newton's Method (NM) algorithm, EBI@NM still suffers from the local minimum issue for both strongly correlated systems and large molecules. Overall, the combination of EBI with the simple first-order algorithm of gradient descent (GD), namely EBI@GD, consistently provides the lowest converged energies for different types of systems, with the lowest computational scaling. These tests demonstrate the advantages of EBI in the calculations of transition states, strongly correlated systems, and large molecules. Meanwhile, the insights gained from this work are helpful to further develop more efficient algorithms for RDMFT.

A key challenge of the collaborative filtering (CF) information filtering is how to obtain the reliable and accurate results with the help of peers' recommendation. Since the similarities from small-degree users to large-degree users would be larger than the ones in opposite direction, the large-degree users' selections are recommended extensively by the traditional second-order CF algorithms. By considering the users' similarity direction and the second-order correlations to depress the influence of mainstream preferences, we present the directed second-order CF (HDCF) algorithm specifically to address the challenge of accuracy and diversity of the CF algorithm. The numerical results for two benchmark data sets, MovieLens and Netflix, show that the accuracy of the new algorithm outperforms the state-of-the-art CF algorithms. Comparing with the CF algorithm based on random walks proposed by Liu et al. (Int. J. Mod. Phys. C, 20 (2009) 285) the average ranking score could reach 0.0767 and 0.0402, which is enhanced by 27.3% and 19.1% for MovieLens and Netflix, respectively. In addition, the diversity, precision and recall are also enhanced greatly. Without relying on any context-specific information, tuning the similarity direction of CF algorithms could obtain accurate and diverse recommendations. This work suggests that the user similarity direction is an important factor to improve the personalized recommendation performance.

This paper presents a comparison of four different three-dimensional (3D) magnetotelluric (MT) modelling algorithms in terms of accuracy and computation time. Three of them use the finite difference method while the last one uses the edge finite-element method. The modelling algorithms are analysed with respect to governing equations, boundary conditions, solvers, preconditioners and static divergence correction. The accuracy of the algorithms is examined by simulating MT responses for two different models, both of which have anomalous bodies in simple background media. Although the simulated responses generally show good agreement, small discrepancies are observed over the anomalous bodies. These discrepancies are caused by the difference between the electric- and magnetic-field formulations used by the algorithms. The three algorithms using the finite difference method are adequate for 3D inversion of MT data since they produce solutions in a short computation time. Although the algorithm using the edge finite-element method requires longer computation time than the others, it can provide more accurate solutions and model surface topography.