Accelerating the Construction of Neural Network Potential Energy Surfaces: A Fast Hybrid Training Algorithm

We construct a full-dimensional ab initio neural network potential energy surface (PES) for the isomerization system of the formic acid dimer (FAD). This is based upon ab initio calculations using the DLPNO-CCSD(T) approach with the aug-cc-pVTZ basis set, performed at over 14000 symmetry-unique geometries. An accurate fit to the obtained energies is generated using a general neural network fitting procedure combined with the fundamental invariant method, and the overall energy-weighted root-mean-square fitting error is about 6.4 cm-1. Using this PES, we present a multidimensional quantum dynamics study on tunneling splittings with an efficient theoretical scheme developed by our group. The ground-state tunneling splitting of FAD calculated with a four-mode coupled method is in good agreement with the most recent experimental measurements. The PES can be applied for further dynamics studies. The effectiveness of the present scheme for constructing a high-dimensional PES is demonstrated, and this scheme is expected to be feasible for larger molecular systems.

Representing high-dimensional potential-energy surfaces for reactions at surfaces by neural networks

A new potential energy surface (PES) and dynamical study of the reactive process of H2CO + OH toward the formation of HCO + H2O and HCOOH + H are presented. In this work, a source of spurious long range interactions in symmetry adapted neural network (NN) schemes is identified, which prevents their direct application for low temperature dynamical studies. For this reason, a partition of the PES into a diabatic matrix plus a NN many-body term has been used, fitted with a novel artificial neural network scheme that prevents spurious asymptotic interactions. Quasi-classical trajectory (QCT) and ring polymer molecular dynamics (RPMD) studies have been carried on this PES to evaluate the rate constant temperature dependence for the different reactive processes, showing good agreement with the available experimental data. Of special interest is the analysis of the previously identified trapping mechanism in the RPMD study, which can be attributed to spurious resonances associated with excitations of the normal modes of the ring polymer.

On extreme learning machines in sequential and time series prediction: A non-iterative and approximate training algorithm for recurrent neural networks

Artificial neural networks represent an accurate and efficient tool to construct high-dimensional potential-energy surfaces based on first-principles data. However, so far the main drawback of this method has been the limitation to a single atomic species. We present a generalization to compounds of arbitrary chemical composition, which now enables simulations of a wide range of systems containing large numbers of atoms. The required incorporation of long-range interactions is achieved by combining the numerical accuracy of neural networks with an electrostatic term based on environment-dependent charges. Using zinc oxide as a benchmark system we show that the neural network potential-energy surface is in excellent agreement with density-functional theory reference calculations, while the evaluation is many orders of magnitude faster.

To investigate the state-to-state dynamics of the P(D2) + H2(XΣg+1)(v = 0, j = 0) reaction, we reconstructed the PH2(X2B1) potential energy surface (PES) using the permutation invariant polynomial neural network (PIP-NN) method based on 40 595 abinitio points. The aug-cc-pVQZ basis sets with Davidson correction were employed throughout the calculations. The reference wave function for the multi-reference configuration interaction calculations was constructed from a full valence complete-active-space self-consistent field wave function. To achieve higher PES accuracy, the double many-body expansion-scaled external correlation (DMBE-SEC) method was applied to extrapolate to the one-electron complete basis set limit, yielding a total root-mean square deviation of 2.4meV for the final NN-PES. Based on the refined PH2(X2B1) NN-PES, geometries, energies, and harmonic frequencies of stationary points were obtained and analyzed in detail, showing excellent agreement with other theoretical results. Subsequently, quantum time-dependent wave packet(TDWP) and quasi-classical trajectory (QCT) methods were utilized to compute reaction probability, integral cross section (ICS), differential cross section (DCS), product rovibrational distribution, and rate constant on the developed NN-PES. The TDWP results reveal rich resonant structures, while the QCT calculations provide a qualitatively correct description of the coarse-grained reaction cross sections. These results also demonstrate a threshold-type microscopic reaction mechanism characterized by dominant forward-backward scattering, attributable to long-lived collision complex formation. Collectively, these findings provide fundamental mechanistic insights into the microscopic reaction mechanism and dynamics of phosphorus chemistry in interstellar environments.

Wind speed forecasting is critical for a variety of engineering tasks, such as wind energy harvesting, scheduling of a wind power system, and dynamic control of structures (e.g., wind turbine, bridge, and building). Wind speed, which has characteristics of random, nonlinear and uncertainty, is difficult to forecast. Nowadays, machine learning approaches (generalized regression neural network (GRNN), back propagation neural network (BPNN), and extreme learning machine (ELM)) are widely used for wind speed forecasting. In this study, two schemes are proposed to improve the forecasting performance of machine learning approaches. One is that optimization algorithms, i.e., cross validation (CV), genetic algorithm (GA), and particle swarm optimization (PSO), are used to automatically find the optimal model parameters. The other is that the combination of different machine learning methods is proposed by finite mixture (FM) method. Specifically, CV-GRNN, GA-BPNN, PSO-ELM belong to optimization algorithm-assisted machine learning approaches, and FM is a hybrid machine learning approach consisting of GRNN, BPNN, and ELM. The effectiveness of these machine learning methods in wind speed forecasting are fully investigated by one-year field monitoring data, and their performance is comprehensively compared.

Speed and accuracy investigations of neural network algorithms for ionospheric modelling at an equatorial region

The reproducing kernel Hilbert space (RKHS) method has been previously shown to be accurate and efficient in the construction of potential energy surfaces (PES) by interpolating fully gridded high level ab initio data. This paper extends the RKHS method to handle partially filled data calculated over arbitrarily shaped regions, while keeping nearly intact its accuracy and efficiency. The extension permits points or regions to be added to or removed from the grid as needed before doing expensive ab initio calculations, thus enabling the construction of RKHS PESs from the data distributions that are most likely to occur in practice. The utility of the new technique is demonstrated using data from the lowest global RKHS PES for the reaction O(1D)+H2, showing that ignoring the irrelevant regions of the PES does not adversely impact the accuracy of the surfaces if the relevant region is adequately sampled.

An implementation of the Levenberg–Marquardt algorithm for simultaneous-energy-gradient fitting using two-layer feed-forward neural networks

Due to the complex nature of training neural network (NN), this problem has gained popularity in the nonlinear optimization field. In order to avoid falling into local minimum because of inappropriate initial weights, a number of global search techniques are developed. This paper applies a novel global algorithm, which is electromagnetism-like mechanism (EM) algorithm, to train NN and the EM based algorithm for neural network training is presented. The performance of the proposed algorithm is evaluated in classification problems and the comparison with BP and GA algorithms shows its effectiveness.

Although the Levenberg-Marquardt (LM) algorithm has been extensively applied as a neural-network training method, it suffers from being very expensive, both in memory and number of operations required, when the network to be trained has a significant number of adaptive weights. In this paper, the behavior of a recently proposed variation of this algorithm is studied. This new method is based on the application of the concept of neural neighborhoods to the LM algorithm. It is shown that, by performing an LM step on a single neighborhood at each training iteration, not only significant savings in memory occupation and computing effort are obtained, but also, the overall performance of the LM method can be increased.

The Levenberg-Marquardt and Newton are two algorithms that use the Hessian for the artificial neural network learning. In this article, we propose a modified Levenberg-Marquardt algorithm for the artificial neural network learning containing the training and testing stages. The modified Levenberg-Marquardt algorithm is based on the Levenberg-Marquardt and Newton algorithms but with the following two differences to assure the error stability and weights boundedness: 1) there is a singularity point in the learning rates of the Levenberg-Marquardt and Newton algorithms, while there is not a singularity point in the learning rate of the modified Levenberg-Marquardt algorithm and 2) the Levenberg-Marquardt and Newton algorithms have three different learning rates, while the modified Levenberg-Marquardt algorithm only has one learning rate. The error stability and weights boundedness of the modified Levenberg-Marquardt algorithm are assured based on the Lyapunov technique. We compare the artificial neural network learning with the modified Levenberg-Marquardt, Levenberg-Marquardt, Newton, and stable gradient algorithms for the learning of the electric and brain signals data set.

With the establishment of machine learning (ML) techniques in the scientific community, the construction of ML potential energy surfaces (ML-PES) has become a standard process in physics and chemistry. So far, improvements in the construction of ML-PES models have been conducted independently, creating an initial hurdle for new users to overcome and complicating the reproducibility of results. Aiming to reduce the bar for the extensive use of ML-PES, we introduce Asparagus, a software package encompassing the different parts into one coherent implementation that allows an autonomous, user-guided construction of ML-PES models. Asparagus combines capabilities of initial data sampling with interfaces to ab initio calculation programs, ML model training, as well as model evaluation and its application within other codes such as ASE or CHARMM. The functionalities of the code are illustrated in different examples, including the dynamics of small molecules, the representation of reactive potentials in organometallic compounds, and atom diffusion on periodic surface structures. The modular framework of Asparagus is designed to allow simple implementations of further ML-related methods and models to provide constant user-friendly access to state-of-the-art ML techniques. Program summaryProgram Title:AsparagusCPC Library link to program files:https://doi.org/10.17632/9w9xw7mp2h.1Developer's repository link:https://github.com/MMunibas/AsparagusLicensing provisions: MITProgramming language: PythonSupplementary material: Access to Documentation at https://asparagus-bundle.readthedocs.ioNature of problem: Constructing machine-learning (ML) based potential energy surfaces (PESs) for atomistic simulations is a multi-step process that requires a broad knowledge in quantum chemistry, nuclear dynamics and programming. So far, efforts mainly focused on developing and improving ML model architectures. However, there was less effort spent on providing tools for consistent and reproducible workflows that support the construction of ML-PES for a variety of chemical systems for the broader science community.Solution method:Asparagus is a program package written in Python that provides a streamlined and extensible workflow with a user-friendly command structure to support the construction of ML-PESs. This is achieved by bundling and linking data generation and sampling techniques, data management, model training, testing and evaluation tools into one modular, comprehensive workflow including interfaces to other simulation packages for the application of the ML-PESs. By lowering the entrance barriers especially for new users, Asparagus supports the generation and adjustment of ML-PESs that allow an increased focus on the physico-chemical evaluation of the chemical system or application in molecular dynamics simulations.Additional comments including restrictions and unusual features:Asparagus is a modular package written in Python providing an underlying structure for further extensions and maintenance. Currently, methods based on message-passing neural network (NN) models using the PyTorch Python package are available. Additions of new models and interfaces to already implemented modules are possible. The NN architecture and hyperparameters are stored in a global configuration module and as a json file for documentation. Except for essential input information, default input parameters are used if not specifically defined otherwise, which allows a quick setup for the construction of a ML-PES but also the fine-tuning for specific needs.

The sensitivity of the outputs of a neural network to perturbations in its weights is an important consideration in both the design of hardware realizations and in the development of training algorithms for neural networks. In designing dense, high-speed realizations of neural networks, understanding the consequences of using simple neurons with significant weight errors is important. Similarly, in developing training algorithms, it is important to understand the effects of small weight changes to determine the required precision of the weight updates at each iteration. In this paper, an analysis of the sensitivity of feedforward neural networks (Madalines) to weight errors is considered. We focus our attention on Madalines composed of sigmoidal, threshold, and linear units. Using a stochastic model for weight errors, we derive simple analytical expressions for the variance of the output error of a Madaline. These analytical expressions agree closely with simulation results. In addition, we develop a technique for selecting the appropriate accuracy of the weights in a neural network realization. Using this technique, we compare the required weight precision for threshold versus sigmoidal Madalines. We show that for a given desired variance of the output error, the weights of a threshold Madaline must be more accurate.