Neural Network and Regression Approximations in High-Speed Civil Aircraft Design Optimization

SummaryThis paper studies the problem of stabilizing reference trajectories (also called as the trajectory tracking problem) for underactuated marine vehicles under predefined tracking error constraints. The boundary functions of the predefined constraints are asymmetric and time‐varying. The time‐varying boundary functions allow us to quantify prescribed performance of tracking errors on both transient and steady‐state stages. To overcome difficulties raised by underactuation and nonzero off‐diagonal terms in the system matrices, we develop a novel transverse function control approach to introduce an additional control input in backstepping procedure. This approach provides practical stabilization of any smooth reference trajectory, whether this trajectory is feasible or not. By practical stabilization, we mean that the tracking errors of vehicle position and orientation converge to a small neighborhood of zero. With the introduction of an error transformation function, we construct an inverse‐hyperbolic‐tangent‐like barrier Lyapunov function to show practical stability of the closed‐loop systems with prescribed transient and steady‐state performances. To deal with unmodeled dynamic uncertainties and external disturbances, we employ neural network (NN) approximators to estimate uncertain dynamics and present disturbance observers to estimate unknown disturbances. Subsequently, we develop adaptive control, based on NN approximators and disturbance estimates, that guarantees the prescribed performance of tracking errors during the transient stage of on‐line NN weight adaptations and disturbance estimates. Simulation results show the performance of the proposed tracking control.

PurposeBuilding cost is an important part of construction projects, and its correct estimation has important guiding significance for the follow-up decision-making of construction units.Design/methodology/approachThis study focused on the application of back-propagation (BP) neural network in the estimation of building cost. First, the influencing factors of building cost were analyzed. Six factors were selected as input of the estimation model. Then, a BP neural network estimation model was established and trained by ten samples.FindingsAccording to the experimental results, it was found that the estimation model converged at about 85 times; compared with radial basis function (RBF), the estimation accuracy of the model was higher, and the average error was 5.54 per cent, showing a good reliability in cost estimation.Originality/valueThe results of this study provide a reliable basis for investment decision-making in the construction industry and also contribute to the further application of BP neural network in cost estimation.

A reactor integrated with separation process has been widely proposed and studied to enhance the production of high value added chemicals. An example of such a process is a pervaporative membrane reactor which is the combination of a reactor and a pervaporation part. The inclusive of a pervaporation can shift the chemical equilibrium of esterification by passing water out of reaction mixture and therefore increase the yield of a desired product. In addition, the temperature of the reactor is one of key factors to define the rate of reactions involved. In this work, a neural network inverse models based control (NIMC) has been designed to control an optimal temperature of a pervaporation membrane reactor. Furthermore, a neural network estimator (NNE) has been incorporated to the strategy as an estimator to estimate the amount of heat released of an esterification reaction. The performance of the designed controller has been evaluated in both nominal and plant/model mismatch cases, and compared with GMC coupled with the Kalman Filter. Simulation study has shown that the NIMC with the NNE can successfully control the reactor temperature at a desired set point in nominal cases and plant/model mismatches. The performance of the NIMC with the NNE is equivalent to that of the GMC coupled with the Kalman Filter. As a result, the NIMC with the NNE is applicable to control the pervaporative membrane reactor without first principle models of the reactor is available. This shows the advantage of the NIMC with NNE over the GMC with the Kalman filter.

In theory, recent results in nonparametric regression show that neural network estimates are able to achieve good rates of convergence provided suitable assumptions on the structure of the regression function are imposed. However, these theoretical analyses cannot explain the practical success of neural networks since the theoretically studied estimates are defined by minimizing the empirical L_2 risk over a class of neural networks and in practice, solving this kind of minimization problem is not feasible. Consequently, the neural networks examined in theory cannot be implemented as they are defined. This means that neural network in applications differ from the ones that are analyzed theoretically. In this thesis we narrow the gap between theory and practice. We deal with neural network regression estimates for (p,C)-smooth regression functions m that satisfy a projection pursuit model. We construct three implementable neural network estimates and show that each of them achieve up to a logarithmic factor the optimal univariate rate of convergence. Firstly, for univariate regression functions with p contained in [-1/2,1] we construct a neural network estimate with one hidden layer where the weights are learned via gradient descent. The starting weights are randomly chosen from an interval independently of the data. The interval is large enough to guarantee that the estimate is close to a piecewise constant approximation. Secondly, for multivariate regression functions with p contained in (0,1] we construct a neural network estimate with one hidden layer where the weights are learned via gradient descent. The initial weights are chosen from specific intervals dependently on the data and the projection directions. This choice guarantees that the estimate is close to a piecewise constant approximation. The projection directions are repeatedly chosen randomly. Lastly, for multivariate regression functions with p>0 we construct a multilayer neural network estimate. The value of the inner weights are prescribed dependently on the projection directions by a new approximation result for a projection pursuit model by piecewise polynomials. The outer weights are chosen by solving a linear equation system. The projection directions are repeatedly chosen randomly. Since we are able to show a rate of convergence that is independent of the dimension of the data our second and third estimates are able to circumvent the curse of dimensionality.

The aspects of building a transformation function for a multicomponent sensor based on the methods of finite element approximation of functions of several variables, as well as approximation using neural networks, are considered in this paper. Testing of mathematical models, computational methods and developed data processing algorithms was carried out on the experimental data of the calibration of a six-component “6ТВ203” sensor. The analysis of the obtained experimental data is carried out and the metrological characteristics of the sensor are determined. The computational algorithms for the use of the finite element and neural network approximations in modern measuring and computing systems have been developed and tested. It is shown that the use of finite element and neural network approximation allows one to improve the measurement results.

Determining the neural network (NN) approximation sets for adaptive control of strict-feedback uncertain systems has posed a persistent challenge. This article proposes a novel and constructive solution that incorporates signal substitution technique, barrier functions (BFs), and backstepping approach. By applying the signal substitution technique, all system states are transformed into state error variables, facilitating the approximation of unknown system functions through NNs. The use of BFs subsequently allows for the restriction of state errors, enabling the calculation of exact bounds for the NN weight estimators. This process reveals the determination of the approximation sets of NN in advance. Illustrative examples are conducted to validate the effectiveness of the proposed approach.

Modelling of a bioprocess was developed by combining three neural network estimators (NNEs). The bioprocess was for fed-batch culture of L-lysine production. The NNEs were NNE1, NNE2, and NNE3. NNE1 was applied to estimate the consumed sugar concentration, L-lysine concentration, and cell optical density. NNE2 was utilized to determine the off-gas concentration of O2 and CO2 while NNE3 was employed to predict the agitation speed. These NNEs possessed their own distinct functions and could be used independently. Combination of the NNEs based on neural network gave an easy and significant way for modelling a bioprocess.

One of the most common problems in science is to investigate a function describing a system. When the estimate is made based on a classical mathematical model (white-box), the function is obtained throughout solving a differential equation. Alternatively, the prediction can be made by an artificial neural network (black-box) based on trends found in past data. Both approaches have their advantages and disadvantages. Mathematical models were seen as more trustworthy as their prediction is based on the laws of physics expressed in the form of mathematical equations. However, the majority of existing mathematical models include different empirical parameters, and both approaches inherit inevitable experimental errors. Simultaneously, the approximation of neural networks can reproduce the solution exceptionally well if fed sufficient data. The difference is that an artificial neural network requires big data to build its accurate approximation, whereas a typical mathematical model needs several data points to estimate an empirical constant. Therefore, the common problem that developers meet is the inaccuracy of mathematical models and artificial neural networks. Another common challenge is the mathematical models’ computational complexity or lack of data for a sufficient precision of the artificial neural networks. Here we analyze a grey-box solution in which an artificial neural network predicts just a part of the mathematical model, and its weights are adjusted based on the mathematical model’s output using the evolutionary approach to avoid overfitting. The performance of the grey-box model is statistically compared to a Dense Neural Network on benchmarking functions. With the use of Shaffer procedure, it was shown that the grey-box approach performs exceptionally well when the overall complexity of a problem is properly distributed with the mathematical model and the Artificial Neural Network. The obtained calculation results indicate that such an approach could increase precision and limit the dataset required for learning. To show the applicability of the presented approach, it was employed in modeling of the electrochemical reaction in the Solid Oxide Fuel Cell’s anode. Implementation of a grey-box model improved the prediction in comparison to the typically used methodology.

This paper addresses the learning control problem for a group of robot manipulator systems in the presence of homogeneous nonlinear uncertain dynamics, where all the robots have an identical system structure but the reference signals to be tracked are different. Our control law has two objectives: one is tracking control, and the other is learning/identification. To this end, deterministic learning theory is incorporated with consensus theory to find a common neural network (NN) approximation of the nonlinear uncertain dynamics for the multi-robot system. Specifically, we first present a control scheme called cooperative deterministic learning using adaptive NNs to enable the robotic agents to track their respective reference trajectories on one hand, and to exchange their estimated NN weights online through networked communication on the other. As a result, a consensus about one common NN approximation for the nonlinear uncertain dynamics is achieved for all the agents. Thus, the trained distributed NNs have better generalization capability than those obtained by existing techniques. Numerical simulations on a team of two-degrees-of-freedom robot manipulators have been conducted to demonstrate the effectiveness of the proposed results.

Deep neural networks have proven to be particularly effective in visual and audio recognition tasks. Existing models tend to be computationally expensive and memory intensive, however, and so methods for hardware-oriented approximation have become a hot topic. Research has shown that custom hardware-based neural network accelerators can surpass their general-purpose processor equivalents in terms of both throughput and energy efficiency. Application-tailored accelerators, when co-designed with approximation-based network training methods, transform large, dense, and computationally expensive networks into small, sparse, and hardware-efficient alternatives, increasing the feasibility of network deployment. In this article, we provide a comprehensive evaluation of approximation methods for high-performance network inference along with in-depth discussion of their effectiveness for custom hardware implementation. We also include proposals for future research based on a thorough analysis of current trends. This article represents the first survey providing detailed comparisons of custom hardware accelerators featuring approximation for both convolutional and recurrent neural networks, through which we hope to inspire exciting new developments in the field.

A design optimization of a staggered pin fin heat sink made of a thermally conductive polymer is presented. The influence of several design parameters like the pin fin height, the diameter, or the number of pins on thermal efficiency of the natural convection heat sink is studied. A limited number of representative heat sink designs were selected by application of the design of experiments (DOE) methodology and their thermal efficiency was evaluated by application of the antecedently validated and verified numerical model. The obtained results were utilized for the development of a response surface and a typical polynomial model was replaced with a neural network approximation. The particle swarm optimization (PSO) algorithm was applied for the neural network training providing very accurate characterization of the heat sink type under consideration. The quasi-complete search of defined solution domain was then performed and the different heat sink designs were compared by means of thermal performance metrics, i.e., array, space claim and mass based heat transfer coefficients. The computational fluid dynamics (CFD) calculations were repeated for the most effective heat sink designs.

This paper investigates the adaptive neural network prescribed performance control problem for a class of dual switching nonlinear systems with time-delay. By using the approximation of neural networks (NNs), an adaptive controller is designed to achieve tracking performance. Another research point of this paper is tracking performance constraints which can solve the performance degradation in practical systems. Therefore, an adaptive NNs output feedback tracking scheme is studied by combining the prescribed performance control (PPC) and backstepping method. With the designed controller and the switching rule, all signals of the closed-loop system are bounded, and the tracking performance satisfies the prescribed performance.

In recent years, neural networks have achieved remarkable progress in various fields and have also drawn much attention in applying them on scientific problems. A line of methods involving neural networks for solving partial differential equations (PDEs), such as Physics-Informed Neural Networks (PINNs) and the Deep Ritz Method (DRM), has emerged. Although these methods outperform classical numerical methods in certain cases, the optimization problems involving neural networks are typically non-convex and non-smooth, which can result in unsatisfactory solutions for PDEs. In contrast to deterministic neural networks, the hidden weights of random neural networks are sampled from some prior distribution and only the output weights participate in training. This makes training much simpler, but it remains unclear how to select the prior distribution. In this paper, we focus on Barron type functions and approximate them under Sobolev norms by random neural networks with clear prior distribution. In addition to the approximation error, we also derive bounds for the optimization and generalization errors of random neural networks for solving PDEs when the solutions are Barron type functions.

To obtain the optimum spread of radial basis functions (Opt_SPRD) without using test samples for constructing Radial Basis Neural Network (RBNN) approximation models with higher accuracy, a new method of choosing spread based on cross validation was proposed. This method took the function between spread and cross validation error as its basis, and took the spread corresponding with the minimum cross validation error as the approximation of Opt_SPRD. The results of numerical experiments indicate: the proposed method is superior to the current default method; compared with the feedforward neural network approximation models based on L-M backpropagation, the RBNN approximation models based on the proposed method produce smaller errors and have more steady performance.

In this article, a globally stable adaptive control strategy for uncertain strict-feedback systems is proposed within predefined neural network (NN) approximation sets, despite the presence of unknown system nonlinearities. In contrast to the conventional adaptive NN control results in the literature, a primary benefit of the developed approach is that the barrier Lyapunov function is employed to predefine the compact set for maintaining the validity of NN approximation at each step, thus accomplishing the global boundedness of all the closed-loop signals. Simulation results are performed to clarify the effectiveness of the proposed methodology.