Sampling Augmented Bilevel Trajectory Optimization for Constrained Quadrotor Flights

We consider the task of low-multilinear-rank functional regression, i.e., learning a low-rank parametric representation of functions from scattered real-valued data. Our first contribution is the development and analysis of gradient-based optimization procedures, including stochastic gradient descent and quasi-Newton methods, for learning the parameters of a functional tensor-train (FT) that yields improved accuracy over standard alternating least squares methods. The functional tensor-train uses the tensor-train (TT) representation of low-rank arrays as an ansatz for a class of low-multilinear-rank functions. The FT is represented by a set of matrix-valued functions that contain a set of univariate functions, and the regression task is to learn the parameters of these univariate functions. Our second contribution demonstrates that using nonlinearly parameterized univariate functions, e.g., symmetric kernels with moving centers, within each core can outperform the standard approach of using a linear expansion of basis functions. Our final contributions are new rank adaptation and group-sparsity regularization procedures to minimize overfitting. We use several benchmark problems to demonstrate at least an order of magnitude lower accuracy with gradient-based optimization methods than ALS in the low-sample number regime. We also demonstrate an order of magnitude reduction in accuracy on a test problem resulting from nonlinear parameterizations over linear parameterizations. Finally we compare regression performance with 22 other nonparametric and parametric regression methods on 10 real-world data sets. We achieve top-five accuracy for seven of the data sets and best accuracy for two of the data sets. These rankings are the best amongst parametric models and competetive with the best non-parametric methods.

Improved Multidisciplinary Multiobjective Collaborative Optimization Technique Based on NSGA-II Algorithm1

Surface wave inversion is a crucial technique in geophysics for subsurface imaging. However, traditional methods can be computationally intensive, especially for complex models. This study introduces automatic differentiation (AD) as an efficient alternative to finite difference (FD) methods for gradient calculation in surface wave inversion. We compare AD and FD methods using three synthetic examples of varying complexity. Our results demonstrate that AD is significantly faster, with speed improvements of 3 to 12 times over FD, depending on model complexity. Moreover, AD requires up to 3 times less memory than FD. In terms of accuracy, AD provides gradient calculations that are exact up to machine precision, while FD is subject to truncation errors. This improved accuracy translates to more reliable inversion results, particularly for complex models. The efficiency and accuracy gains of AD are especially beneficial for gradient-based inversion methods in geophysics, where computational resources often limit the scale and complexity of problems that can be addressed. Our findings suggest that integrating AD into gradient-based inversion methods could significantly enhance subsurface imaging techniques across various geophysical applications.

A variety of substructure concepts for floating offshore wind have been developed. Design optimization can be used to efficiently explore this design space and guide further work. Previous design optimization studies have been limited by model simplifications, including the assumption of rigid body motions, lack of substructure flexibility, and a focus on a single floating foundation concept. High computational costs of gradient-free optimization methods have limited the number of design variables considered. In this work, gradient-based optimization methods and a frequency-domain modal analysis model based on three-dimensional modeshapes make it possible to consider a more detailed structural model with reasonable computational cost. This work implements a linearized aero-hydro-servo-elastic model of a tension-leg platform wind turbine. The optimization varies sizing parameters and the model computes responses to wind and wave forcing in multiple environmental conditions. The model computes forcing and response in generic modes rather than assuming rigid body motions. Implementation of the model with analytical gradients in OpenMDAO allows for efficient optimization with dozens of design variables. The optimization results show reduction of the objective while satisfying constraints. Furthermore, there is potential for this approach to be adapted to other floating wind turbine substructure designs.

Neural network model has undergone many modifications from the original model through the development of network architecture and optimization methods. By default, the gradient-based optimization method is a method used to find network weights. But some of the weaknesses and limitations of these methods inspired many researchers to try to use other methods. Non-gradient based heuristic method is a reasonable choice considering the learning algorithm in artificial neural networks is inspired by the characteristics of creatures so that optimization methods that also mimic life patterns in nature will be appropriate. One disadvantage of the heurisic method is the length of the iteration process. In this paper, a method which combines heuristic optimization methods and gradient-based methods is applied, namely particle swarm optimization (PSO) and Levenberg Marquardt. The weight obtained from the PSO method becomes the initial weight for the Levenberg Marquardt method. The proposed procedure is applied to the rainfall data in Cokrotulung Klaten. The results showed that this procedure succeeded in providing better predictions than the Levenberg Marquardt method.

Modern flexible production systems benefit from collaborative robot systems that support to teach robot configurations by hand to quickly implement collision-free motions. However, appropriate interpolation schemes that allow a fast and smooth motion through this sequence of waypoints is often not part of robot control systems. Instead, only point-to-point motions or a set of predefined curves like circular motions are supported. We propose an approach to improve the efficiency of finding a time parameterization with cubic splines that computes minimum trajectory durations while respecting actuator limits and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$C^2$</tex-math></inline-formula> -continuity. Compared to the existing method, our contribution consists in the analytical gradients of the underlying nonlinear optimization problem and thus we are able to propose an efficient solution approach using state-of-the-art optimization solvers. We validate our implementation in simulation and in experiments, and benchmark it with the three trajectory generation methods implemented in <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MoveIt!</i> .

Machine learning interatomic potentials have become an indispensable tool for materials science, enabling the study of larger systems and longer timescales. State-of-the-art models are generally graph neural networks that employ message passing to iteratively update atomic embeddings that are ultimately used for predicting properties. In this work we extend the message passing formalism with the inclusion of a continuous variable that accounts for fractional atomic existence. This allows us to calculate the gradient of the Gibbs free energy with respect to both the Cartesian coordinates of atoms and their existence. Using this we propose a gradient-based grand canonical optimization method and document its capabilities for a Cu(110) surface oxide.

A wide class of problems involves the minimization of a coercive and differentiable function F on $${\mathbb {R}}^N$$ whose gradient cannot be evaluated in an exact manner. In such context, many existing convergence results from standard gradient-based optimization literature cannot be directly applied and robustness to errors in the gradient is not necessarily guaranteed. This work is dedicated to investigating the convergence of Majorization-Minimization (MM) schemes when stochastic errors affect the gradient terms. We introduce a general stochastic optimization framework, called StochAstic suBspace majoRIzation-miNimization Algorithm SABRINA that encompasses MM quadratic schemes possibly enhanced with a subspace acceleration strategy. New asymptotical results are built for the stochastic process generated by SABRINA. Two sets of numerical experiments in the field of machine learning and image processing are presented to support our theoretical results and illustrate the good performance of SABRINA with respect to state-of-the-art gradient-based stochastic optimization methods.

Significant computational resources are required to obtain an optimised wing jig shape by solving a high-fidelity large-scale aero-structural design optimisation problem. Gradient-based methods are efficient; however, some of the features of real-life engineering problems including numerical noise that pollutes the function values and occurrences of failed evaluations in the optimisation may limit their performance. To address these issues, this paper presents the latest developments in the multipoint approximation method (MAM) based on a gradient-assisted metamodel assembly technique within a trust-region optimisation framework. The proposed method is tested by a benchmark case first, and then, an aircraft wing jig shape optimisation problem is offered to demonstrate its performance. The gradient-based optimisation is used as a benchmark case, and the metamodel-based optimisation utilises the latest developments in MAM to solve the same problem. The results show that the proposed method can achieve the same design goal as the gradient-based method but with enhanced robustness and efficient performance. In the wing jig shape optimisation, the difference in the design objective, the global equivalent drag coefficient, between the two aforementioned optimisation approaches is 0.20 counts, whose relative difference is approximately 0.10%. Three approximate sub-optimisations have been conducted in every iteration of the metamodel-based optimisation to reduce the possibility of local optimality, while the overall elapsed time of the metamodel-based optimisation is approximately 1.98 times that of one gradient-based optimisation, which confirms the competitiveness of the proposed method bearing in mind the added safeguards for numerical noise, failed evaluations and possible local optimality.

Optimization-based model calibration (OBMC) determines statistical parameters of unknown input variables by optimizing statistical similarity between simulations and observations. Gradient-based optimization algorithms have been utilized for OBMC since they require less computational cost compared with non-gradient optimization algorithms. However, OBMC using numerical gradients such as finite differences has disadvantages in the aspect of computational cost and numerical instability. For these reasons, a previous study derived analytical gradients under the assumption that performance functions are a linear single response and unknown inputs follow a normal distribution. However, to apply the model calibration to general engineering problems, analytical gradients need to be derived without any assumption. Therefore, it is proposed in this study to derive analytical gradients of marginal and joint likelihood functions to calibrate both linear and nonlinear responses. In addition, the analytical gradients are derived for five distribution types of unknown input random variables to be estimated: normal, lognormal, Gumbel, extreme, and uniform distributions. For derivation of the analytical gradients, the chain rule is utilized to combine the derivatives of the inverse function of cumulative distribution function (CDF), performance function, and joint likelihood function. Numerical examples and one engineering example using finite element analysis (FEA) are employed to verify the accuracy and efficiency of the proposed analytical gradient, and it is demonstrated that accurately calibrated output responses can be obtained by the proposed method by finding a better optimum in OBMC.

A full implementation of analytical energy gradients for molecular and periodic systems is reported in the TURBOMOLE program package within the framework of Kohn-Sham density functional theory using Gaussian-type orbitals as basis functions. Its key component is a combination of density fitting (DF) approximation and continuous fast multipole method (CFMM) that allows for an efficient calculation of the Coulomb energy gradient. For exchange-correlation part the hierarchical numerical integration scheme (Burow and Sierka, Journal of Chemical Theory and Computation 2011, 7, 3097) is extended to energy gradients. Computational efficiency and asymptotic O(N) scaling behavior of the implementation is demonstrated for various molecular and periodic model systems, with the largest unit cell of hematite containing 640 atoms and 19,072 basis functions. The overall computational effort of energy gradient is comparable to that of the Kohn-Sham matrix formation. © 2016 Wiley Periodicals, Inc.

General analytic gradient expressions (with the frozen-core approximation) are presented for density-fitted post-HF methods. An efficient implementation of frozen-core analytic gradients for the second-order Møller-Plesset perturbation theory (MP2) with the density-fitting (DF) approximation (applying to both reference and correlation energies), which is denoted as DF-MP2, is reported. The DF-MP2 method is applied to a set of alkanes, conjugated dienes, and noncovalent interaction complexes to compare the computational cost of single point analytic gradients with MP2 with the resolution of the identity approach (RI-MP2) [F. Weigend and M. Häser, Theor. Chem. Acc. 97, 331 (1997); R. A. Distasio, R. P. Steele, Y. M. Rhee, Y. Shao, and M. Head-Gordon, J. Comput. Chem. 28, 839 (2007)]. In the RI-MP2 method, the DF approach is used only for the correlation energy. Our results demonstrate that the DF-MP2 method substantially accelerate the RI-MP2 method for analytic gradient computations due to the reduced input/output (I/O) time. Because in the DF-MP2 method the DF approach is used for both reference and correlation energies, the storage of 4-index electron repulsion integrals (ERIs) are avoided, 3-index ERI tensors are employed instead. Further, as in case of integrals, our gradient equation is completely avoid construction or storage of the 4-index two-particle density matrix (TPDM), instead we use 2- and 3-index TPDMs. Hence, the I/O bottleneck of a gradient computation is significantly overcome. Therefore, the cost of the generalized-Fock matrix (GFM), TPDM, solution of Z-vector equations, the back transformation of TPDM, and integral derivatives are substantially reduced when the DF approach is used for the entire energy expression. Further application results show that the DF approach introduce negligible errors for closed-shell reaction energies and equilibrium bond lengths.

In this paper, a new shape identification method in the inverse heat conduction problem (IHCP) is applied to detect the location and size of defects in a solid body. Different defects are modeled in a solid body as an elliptical geometry whose parameters are estimated with a proposed inverse algorithm. The inverse algorithm consists of direct, inverse analysis, and gradient-based optimization method. The direct analysis is used a finite-element method in an unstructured grid system to solve the direct heat conduction problem. The inverse analysis is based on recording temperatures data on surface of solid body that calculates the objective function. The employed gradient-based optimization method is constructed using the adjoint, sensitivity, and conjugate gradient method (Powell-Beal’s version) that are used to calculate the gradient of objective function, step size, and minimizing the objective function, respectively. The effects of different noisy temperature data, different cavities on some domains, and different type of defects such as poor cure, porosity, and crack are investigated in this work. The results show that this proposed inverse algorithm is more efficient in detection of defects.

Structural topology optimization (STO) is usually treated as a constrained minimization problem, which is iteratively addressed by solving the equilibrium equations for the problem under consideration. To reduce the computational effort, several reduced basis approaches that solve the equilibrium equations in a reduced space have been proposed. In this work, we apply functional principal component analysis (FPCA) to generate the reduced basis, and we couple FPCA with a gradient-based optimization method for the first time in the literature. The proposed algorithm has been tested on a large STO problem with 4.8 million degrees of freedom. Results show that the proposed algorithm achieves significant computational time savings with negligible loss of accuracy. Indeed, the density maps obtained with the proposed algorithm capture the larger features of maps obtained without reduced basis, but in significantly lower computational times, and are associated with similar values of the minimized compliance.

In this article, shape identification in inverse heat conduction problems is applied to estimate the solidliquid interface in a cold storage system. The inverse algorithm consists of direct, inverse analysis, and gradient-based optimization method that iteratively estimates the unknown shape. The direct analysis used the finite-element method in the unstructured grid system to solve the direct heat conduction problem. The inverse analysis is based on recording temperature data on the outer surface of the storage capsule that calculates the objective function with calculated and measurement temperatures. The employed gradient-based optimization method is constructed using the adjoint and sensitivity equations that are used to calculate the gradient of the objective function and the optimal step size, respectively. Both the control point method (CPM) and proposed method (shape variables method) are used to estimate the unknown shape and to compare themselves. The effects of the number of sensors, grid sizes, shape scales, initial shapes, and noisy temperature data on the solution are investigated. The results show that the proposed method is more efficient in the prediction of the solidliquid interface in comparison with CPM.