Adam Algorithm with Step Adaptation

Hydrate plugging and pipeline leak can impair the normal operation of natural gas pipeline and may lead to serious accidents. Since natural gas pipeline safety monitoring based on active acoustic excitation can detect and locate not only the two abnormal events but also normal components such as valves and pipeline elbows, recognition and classification of these events are of great importance to provide maintenance guidance for the pipeline operators and avoid false alarm. In this article, long short-term memory (LSTM) network is introduced and applied to classify detection signals of hydrate plugging, pipeline leak, and elbow. Adaptive moment estimation (Adam) algorithm is introduced and utilized to accelerate the long short-term memory network convergence in training. Experimental results demonstrate that the network with three layers and 64 units per cell performs the best. The cross-entropy loss in training is 0.0005, and classification accuracies are all 100% in training, validation, and testing which verify the validity of the long short-term memory network. Therefore, the method based on the long short-term memory network and adaptive moment estimation algorithm can work efficiently on pipeline events classification and has great guiding significance for safety assurance of natural gas transmission.

According to the accuracy of BP (Back Propagation) neural network in leveling fitting, an improved BP neural network model is proposed. In order to overcome the defect of the low convergence rate and local minimal in leveling fitting. The BP neural network is improved by using Relu (Rectified Linear Units) function as network activation function and Adam (Adaptive Moment Estimation) algorithm as network optimization function. The experimental results show that the improved BP neural network can effectively improve the accuracy.

Corrosion can affect the reliability of materials, which has attracted the attention of the industry. Corrosion detection and quantitative analysis are particularly important for scientific management and decision-making. In this paper, the imaging method based ultrasonic guided wave (UGW) detection technology and fully connected neural network (FCNN) is proposed to realize real-time imaging of corrosion damages. The imaging method contains offline training and online testing. Offline training aims to establish the relationship between detection signals and velocity maps and it is accelerated by adaptive moment estimation (Adam) algorithm. In the process of online testing, the trained model can be called directly to realize real-time imaging, that is, the detection signals are fed into the model and the network will predict the velocity maps. Finally, the velocity maps are converted to thickness maps according to the dispersion curves. Numerical experimental results show that the mean square errors (mses) are respectively 9.08 × 10−4, 2.47 × 10−3 and 2.59 × 10−3 in training, validation and testing. Compared with irregular corrosion damages, the imaging method has better imaging quality for circular corrosion damages.

Adaptive moment estimation (Adam), as a Stochastic Gradient Descent (SGD) variant, has gained widespread popularity in federated learning (FL) due to its fast convergence. However, federated Adam (FedAdam) algorithms suffer from a threefold increase in uplink communication overhead compared to federated SGD (FedSGD) algorithms, which arises from the necessity to transmit both local model updates and first and second moment estimates from distributed devices to the centralized server for aggregation. Driven by this issue, we propose a novel sparse FedAdam algorithm called FedAdam-SSM, wherein distributed devices sparsify the updates of local model parameters and moment estimates and subsequently upload the sparse representations to the centralized server. To further reduce the communication overhead, the updates of local model parameters and moment estimates incorporate a shared sparse mask (SSM) into the sparsification process, eliminating the need for three separate sparse masks. Theoretically, we develop an upper bound on the deviation between the local model trained by FedAdam-SSM and the target model trained by centralized Adam, which is related to sparsification error and imbalanced data distribution. By minimizing the deviation bound between the model trained by FedAdam-SSM and centralized Adam, we optimize the SSM to mitigate the learning performance degradation caused by sparsification error. Additionally, we provide convergence bounds for FedAdam-SSM in both convex and non-convex objective function settings, and investigate the impact of local epoch, learning rate and sparsification ratio on the convergence rate of FedAdam-SSM. Experimental results show that FedAdam-SSM outperforms baselines in terms of convergence rate (over 1.1× faster than the sparse FedAdam baselines) and test accuracy (over 14.5% ahead of the quantized FedAdam baselines).

The selection of the optimizer is critical for convergence in the field of on-chip training. As one second moment optimizer, adaptive moment estimation (ADAM) shows a significant advantage compared with non-moment optimizers such as stochastic gradient descent (SGD) and first-moment optimizers such as Momentum. However, ADAM is hard to implement on hardware due to the computationally intensive operations, including square, root extraction, and division. This work proposed Hardware-ADAM (HW-ADAM), an efficient fixed-point accelerator for ADAM highlighting hardware-oriented mathematical optimizations. HW-ADAM has two designs: Efficient-ADAM (E-ADAM) unit reduced the hardware resource consumption by around 90% compared with the related work. E-ADAM achieved a throughput of 2.89 MUOP/s (Million Updating Operation per Second), which is 2.8× of the original ADAM. Fast-ADAM (F-ADAM) unit reduced 91.5% flip-flops, 65.7% look-up tables, and 50% DSPs compared with the related work. The F-ADAM unit achieved a throughput of 16.7 MUOP/s, which is 16.4× of the original ADAM.

Optimization algorithms constitute essential components within artificial intelligence systems designed to enhance the diagnosis of cardiovascular diseases. In their absence, even structurally sound neural networks may fail to learn effectively from noisy or heterogeneous clinical datasets. This review focuses on two main categories of non-linear stochastic optimization methods: gradient-based algorithms—such as Stochastic Gradient Descent, Adaptive Gradient Algorithm, Root Mean Square Propagation, and Adaptive Moment Estimation—and gradient-free approaches, including Simulated Annealing, Metropolis-Hastings, and Swarm Intelligence algorithms. Among these, Adaptive Moment Estimation demonstrates robust and efficient convergence during neural network training, while Simulated Annealing proves effective in exploring a broad range of hyperparameter configurations, especially in scenarios involving uncertainty and physiological signal processing. Based on these complementary strengths, a hybrid optimization strategy is proposed, integrating both methods to address local refinement and global exploration in model development. Although no experimental results are included, the proposed framework is expected to outperform conventional techniques—such as logistic regression and standard gradient descent—by improving sensitivity, specificity, and model calibration, while also reducing overfitting and enhancing generalization. The resulting system is envisioned as a secure, web-based application incorporating interpretability and auditability features to support clinical decision-making.

Neural network-based active disturbance rejection control and its application in quadrotor unmanned aerial vehicle trajectory tracking.

Hepatocellular carcinoma (HCC), the primary cancer of the liver, is life-threatening, with few or no symptoms, and detection in the early stage will help for successful treatment with surgery, and transplant for a better life quality. Here, we proposed two stacking classification models based on deep learning with differential hybrid feature selection for the early detection of HCC using novel non-invasive biomarker PIVKA-II. We showed how the variations in hybrid feature selection affect the performance of stacking classification and different supervised machine-learning algorithms on a metaclassifier. The base layers were support vector machine (SVM), gradient boosting (GB), and linear discriminant analysis (LDA). The meta classifier was a multilayer perceptron (MLP) with three different optimizers, stochastic gradient descent (SGD), adaptive moment estimation (ADAM), and Limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS). Our first model outperformed the second with their hybrid features by improving accuracy by 1.5% and F1_score by 1% in both SGD and ADAM optimization, while MLP-LBFGS had a 1.4% increase in accuracy. The precision had hiked by 1.9%, 3.5%, and 1.7% in SGD, ADAM, and LBFGS, respectively, in Model-1. Matthew’s correlation coefficient (MCC) for MLP-SGD increased from 0.82 to 0.85, MLP-ADAM from 0.81 to 0.85, and MLP-LBFGS from 0.75 to 78 for the first model.

Classifying or predicting occupational incidents using both structured and unstructured (text) data are an unexplored area of research. Unstructured texts, i.e., incident narratives are often unutilized or underutilized. Besides the explicit information, there exist a large amount of hidden information present in a dataset, which cannot be explored by the traditional machine learning (ML) algorithms. There is a scarcity of studies that reveal the use of deep neural networks (DNNs) in the domain of incident prediction, and its parameter optimization for achieving better prediction power. To address these issues, initially, key terms are extracted from the unstructured texts using LDA-based topic modeling. Then, these key terms are added with the predictor categories to form the feature vector, which is further processed for noise reduction and fed to the adaptive moment estimation (ADAM)-based DNN (i.e., ADNN) for classification, as ADAM is superior to GD, SGD, and RMSProp. To evaluate the effectiveness of our proposed method, a comparative study has been conducted using some state-of-the-arts on five benchmark datasets. Moreover, a case study of an integrated steel plant in India has been demonstrated for the validation of the proposed model. Experimental results reveal that ADNN produces superior performance than others in terms of accuracy. Therefore, the present study offers a robust methodological guide that enables us to handle the issues of unstructured data and hidden information for developing a predictive model.

Optimizing Time-Series forecasting using stacked deep learning framework with enhanced adaptive moment estimation and error correction

A conventional approach to solving stochastic optimal control problems with time-dependent uncertainties involves the use of the stochastic maximum principle (SMP) technique. For large-scale problems, however, such an algorithm frequently leads to convergence complexities when solving the two-point boundary value problem resulting from the optimality conditions. An alternative approach consists of using continuous random variables to capture uncertainty through sampling-based methods embedded within an optimization strategy for the decision variables; such a technique may also fail due to the computational intensity involved in excessive model calculations for evaluating the objective function and its derivatives for each sample. This paper presents a new approach to solving stochastic optimal control problems with time-dependent uncertainties based on BONUS (Better Optimization algorithm for Nonlinear Uncertain Systems). The BONUS has been used successfully for non-linear programming problems with static uncertainties, but we show here that its scope can be extended to the case of optimal control problems with time-dependent uncertainties. A batch reactor for biodiesel production was used as a case study to illustrate the proposed approach. Results for a maximum profit problem indicate that the optimal objective function and the optimal profiles were better than those obtained by the maximum principle.

This paper concentrates on investigating an enhanced independent component analysis (ICA) method for blind separation of signals corrupted by noise in wireless communications. Because of the traditional classical ICA methods that always have an inadequate capacity of anti-noise or insufficient separable ability in noise circumstance without satisfying practical application requirements. For this reason, two mechanisms are conducted to establish the modified cost function and fulfill the optimization assignment in the process of constituting an enhanced ICA algorithm. This proposed algorithm can benefit tremendously from the derived minimum bit error rate (BER) criterion and the novel adaptive moment estimation (Adam) optimization approach. In this work, firstly, the novel cost function is obtained according to minimum BER fused into maximum likelihood (ML) principle-based ICA cost function. Furthermore, by utilizing Adam processing, the task of blind separation of mixed signals is implemented via optimizing this modified cost function. Lastly, theoretical analysis and experiment results corroborate the better effectiveness and robustness of the proposed enhanced ICA algorithm compared with a series of popular representative ICA methods.

One of the main problems in stochastic optimization is the minimization of mean value functions F(x) = Eu(A(ω) x ‐ b(ω)) s.t. x ∈ D, where (A(ω), b(ω)) is a random matrix, u is a loss function on Rm and D is a convex subset of Rn. If the distribution μ of (A(ω), b(ω)) is invariant with respect to an affine transformation Λ of Rn(m+1), i.e. if Λ(μ) = μ, and if for a given x ∈ Rn there exists a vector y = y(x) such that \documentclass{article}\pagestyle{empty}\begin{document}$ \Lambda \left({A,b} \right)\left({\begin{array}{*{20}c} x \\ {- 1} \\ \end{array}} \right) = Ay - b $\end{document} a.s., then F(y) = F(x), and, consequently, d = y ‐ x is a descent direction of F at x, provided that F is not constant on the line segment joining x and y. For many practically important distributions μ the affine transformation Λ, a vector y(x) and therefore also a direction of decrease d = y(x) ‐ x is constructed without using any derivatives of F.

A guidance approach with obstacle avoidance is proposed for maneuvering target interception. Firstly, by decomposing the constrained optimal guidance problem into two associated subproblems, the overall scheme of guidance is presented. Secondly, to improve the guidance accuracy against the maneuvering target, an MPC controller with disturbance estimation is designed, which transforms the global optimization problem into a finite-horizon optimal control problem. An auxiliary controller is also developed to ensure the system’s stability. Then, to achieve obstacle avoidance without affecting the guidance accuracy, a unified function is built to assess the threats from obstacles of different shapes, based on which the optimal guidance-obstacle avoidance model is established. Finally, to solve the above optimization problems, a hybrid solver is developed based on the adaptive moment estimation (Adam) algorithm, whose output will serve as the real-time guidance command. The numerical simulation results show that the guidance approach presented in this paper can achieve precise guidance and obstacle avoidance in various scenarios, and the Monte Carlo experiment further demonstrates the robustness and computation efficiency of the approach.

Particle swarm optimization (PSO) algorithm is a promising swarm intelligence optimization technology. It has been applied to a variety of complex optimization problems due to its outstanding global search ability. However, it suffers from premature convergence and slow convergence rate. Motivated by adaptive moment estimation (Adam) method, which is computationally efficient, little memory-required and also appropriate for non-stationary objectives, a hybrid algorithm combining adaptive PSO with a modified Adam method (AdamPSO) is proposed in this paper. Adaptive particle swarm optimization (APSO) is first used to perform stochastic and rough search. In the solution space obtained by APSO, Adam method is then used to perform further search, which may establish a new solution space. Depending on the fitness value of particles, the position of each particle switches alternately between APSO and Adam. The experimental results on six well-known benchmark functions show that our proposed algorithm gets better convergence performance compared to other five classical PSOs.