Beyond Backpropagation: Smarter Neural Networks for Smart Manufacturing

The Sixth International Symposium on Neural Networks (ISNN 2009)

Advanced analytics applications in smart manufacturing provide beneficiary effects such as predictive maintenance, energy optimized factories, and automized quality management of products. While the benefits and thus the sustainability effects are well investigated, the research field is characterized by many single applications with little conceptual synthesis. Therefore, this research aims to systematize the field and to identify specific sustainability themes in context of advance analytics applications. This chapter provides an overview of existing advanced analytics applications in smart manufacturing and clusters the addressed sustainability themes. Applying a systematic literature review, following the PRISMA statement, 65 different applications within four research perspective could be identified. Furthermore, ten different sustainability-related effects of the applications in smart manufacturing unfold. The content analysis resulted in 27 sustainability themes, which holistically cover all spheres of sustainability, with a focus on the social and economic spheres.

NOVEL STAGING TOOL FOR LOCALIZED PROSTATE CANCER: A PILOT STUDY USING GENETIC ADAPTIVE NEURAL NETWORKS

We propose a neural network framework to preclude the need to define or observe incompletely or inaccurately defined states of a material in order to describe its response. The neural network design is based on the classical Coleman-Gurtin internal state variable theory. In the proposed framework the states of the material are inferred from observable deformation and stress. A neural network describes the flow of internal states and another represents the map from internal state and strain to stress. We investigate tensor basis, component, and potential-based formulations of the stress model. Violations of the second law of thermodynamics are prevented by a constraint on the weights of the neural network. We extend this framework to homogenization of materials with microstructure with a graph-based convolutional neural network that preprocesses the initial microstructure into salient features. The modeling framework is tested on large datasets spanning inelastic material classes to demonstrate its general applicability.

Automatic data labeling by neural networks for the counting of objects in videos

As Li-ion batteries become more essential in today's economy, tools need to be developed to accurately and rapidly diagnose a battery's internal state-of-health. Using a Li-ion battery's (high-rate) voltage response, it is proposed to determine a battery's internal state through Bayesian calibration.However, Bayesian calibration is notoriously slow and requires thousands of model runs. To accelerate parameter inference using Bayesian calibration, a surrogate model is developed to replace the underlying physics-based Li-ion model. Developing a surrogate model for rapid Bayesian calibration analysis is discussed for both the single particle model (SPM) and the pseudo two-dimensional (P2D) model.Surrogate models are constructed using physics-informed neural networks (PINNs) that encode the influence of internal properties on observed voltage responses. In practice, a neural network can be trained by: 1) using simulation results of the physics-based model (i.e., a data-loss approach); 2) using the residuals of the governing equations themselves (i.e., a physics-loss approach); or 3) using a combination of simulation results and governing equation residuals. In the present work, PINNs are developed using a variety of training losses and neural network architectures. In this analysis, it is shown that a PINN surrogate model can be reliably trained with only physics-informed loss. However, using a coupled data-informed and physics-loss approach produced the most accurate PINNs. Figure~\\ref{fig:spm_2d} illustrates the absolute relative errors of trained PINN networks using several different training losses and neural network architectures.After determining a consistent training strategy for both the SPM and P2D PINN surrogate models, the PINNs are extended to determine additional internal state-of-health parameters. As more and more parameters were introduced, the PINN training suffered from ``the curse of dimensionality", which was mitigated by using a hierarchical training approach (where a PINN trained with fewer variable model parameters was used to train a PINN with more variable model parameters).Next, the high-dimensionality PINN surrogates are then integrated into Bayesian calibration schemes to identify internal Li-ion battery properties from experimentally measured voltages. Interpreting the high-dimensional parameter posteriors is discussed with respect to model error, parameter prior choices, and experimental errors. Figure 1

The aim of a personalized heating system is to provide a desirable microclimate for each individual when heating is needed. In this paper, we present a method based on machine learning algorithms for generation of predictive models for use in control of personalized heating systems. Data was collected from two individual test subjects in an experiment that consisted of 14 sessions per test subject with each session lasting 4 h. A dynamic recurrent nonlinear autoregressive neural network with exogenous inputs (NARX) was used for developing the models for the prediction of personalized heating settings. The models for subjects A and B were tested with the data that was not used in creating the neural network (unseen data) to evaluate the accuracy of the prediction. Trained NARX showed good performance when tested with the unseen data, with no sign of overfitting. For model A, the optimal network was with 12 hidden neurons with root mean square error equal to 0.043 and Pearson correlation coefficient equal to 0.994. The best result for model B was obtained with a neural network with 16 hidden neurons with root mean square error equal to 0.049 and Pearson correlation coefficient equal to 0.966. In addition to the neural network models, several other machine learning algorithms were tested. Furthermore, the models were on-line tested and the results showed that the test subjects were satisfied with the heating settings that were automatically controlled using the models. Tests with automatic control showed that both test subjects felt comfortable throughout the tests and test subjects expressed their satisfaction with the automatic control.

A Physics-Informed Neural Network (PINN) was developed to solve the complex Navier-Stokes (N-S) equations for multiphase flow measurement including flowrates of individual phases in the multiphase mixture, and phase ratios. The results were compared against a mathematical based multiphase flow measurement approach and validated through a controlled loop test using oil, water, and gas. The PINN embeds fundamental fluid dynamics principles into its learning process, ensuring that the neural network adheres to the governing physical laws of motion, continuity, and conservation. Unlike traditional neural networks that rely solely on data-driven learning, PINNs incorporate fluid dynamics driven differential equations, to enhance model accuracy and reliability. This study evaluates the accuracy, efficiency, and repeatability of PINNs by comparing their results against traditional mathematical-based multiphase flow measurement (MPFM) technology and controlled test cases. The developed PINN successfully solved the Navier- Stokes (N-S) equations in their closest approximation to the original form, with minimal assumptions and simplifications, to estimate multiphase flow rates, integrating fundamental fluid dynamics principles into its learning process. The results demonstrated that the PINN-based approach achieved accuracy comparable to traditional mathematical multiphase flow measurement (MPFM) technologies, while significantly improving computational efficiency. During controlled loop tests with oil, water, and gas, the PINN model consistently captured flow dynamics with high precision, validating its effectiveness under real-world conditions. Moreover, the PINN method exhibited strong generalization capabilities, adapting to various flow regimes without the need for extensive empirical calibration. Compared to conventional MPFM technologies, PINN reduced reliance on predefined correlations and offered greater adaptability to changing fluid properties. The repeatability of results confirmed the model's reliability, positioning PINNs as a promising alternative for complex multiphase flow measurement applications. These findings highlight the potential of PINNs as an innovative, physics-guided, and data-efficient solution, effectively bridging the gap between traditional physics-based methods and modern machine learning techniques. This paper presents an innovative application of Physics-Informed Neural Networks (PINNs) for multiphase flow measurement, offering a data-driven yet physics-guided alternative to traditional mathematical MPFM technologies. By embedding fundamental physical laws into the neural network training process, the proposed approach enhances accuracy, computational efficiency, and adaptability, while minimizing reliance on extensive empirical calibration. This advancement significantly improves real-time flow monitoring, equipping practicing engineers with a more reliable, data-efficient, and scalable solution for complex multiphase flow measurement in industrial applications.

This research article proposes a deep learning framework that addresses two major hurdles in adopting deep learning techniques for solving physics-based problems. One is the requirement of a large data set for training the deep learning (DL) model and another is the consistency of a DL model with the physics of a phenomenon. The framework is generic that can be applied to model a phenomenon in physics if its behavior is known. A semi-supervised physics guided neural network (SPGNN) has been developed based on our framework to demonstrate the concept. SPGNN models the I–V characteristics of gallium nitride based high electron mobility transistors (GaN HEMTs). A two-stage method has been proposed to train a DL model. In the first stage, the DL model is trained via an unsupervised learning method using the analytical physics-based model of a field-effect transistor (FET) as a loss function of the DL model that incorporates the physics of the FET in the DL model. Later, the DL model is fine-tuned with a small set of experimental data in the second stage. Performance of SPGNN has been assessed on various sizes of the data set with 100, 500, 1000, 1500, and 2000 samples. SPGNN significantly reduces the training data requirement by more than 80% and provides better performance than a traditionally trained neural network (TTNN), even for the unseen test data set. SPGNN predicts 32.4% of the unseen test data with less than 1% of error and only 0.4% of the unseen test data with more than 10% of error.

Aiming to boost production efficiency and reduce human workload, human-centricity has emerged as the core concept of Industry 5.0 (I5.0). However, current works have not established a unified automation and autonomous framework for human-centric smart manufacturing across various real world applications. Addressing this gap, this research introduces an innovative automated framework, ParallelWorkforce, which integrates blockchain intelligence and decentralized autonomous organizations and operations (DAOs) to drive the evolution from digital twins to parallel intelligence. First, this research conducts a comprehensive investigation into smart manufacturing in I5.0, summarizing the ongoing evolution. Next, a detailed exploration of ParallelWorkforce is provided to offer customized strategies for managing different levels of out-of-distribution events, significantly alleviating the workload on biological workers and maximizing the potential of both digital and robotic workers. Finally, the development of ParallelWorkforce across various key applications of smart manufacturing is demonstrated, including autonomous transportation, task assignment, and worker management. This research provides a viable solution for the further development of human-centered smart manufacturing and paves the way for the realization of “6S” goals in I5.0.

Digital Twin Technology — A bibliometric study of top research articles based on Local Citation Score

Systematic literature review on augmented reality in smart manufacturing: Collaboration between human and computational intelligence

Evaluation criteria of automated transmission control algorithms are an essential research object under increasingly stringent emission requirements and demand for riding comfort. Model-based development and calibration can be employed to optimize the corresponding actuator control algorithm in the transmission control unit (TCU). In this paper, the challenge is to realize on-line adaptively optimization of the controller based on position trajectory. The clutch in an automated manual transmission system is taken as a case study and is considered as a nonlinear dynamic system with uncertainties and unknown external disturbances. A controller with accurate and rapid position trajectory tracking performance is crucial to finding out the correlation between the clutch position changes and the shift quality. Traditionally, the control parameters are manually tuned off-line via trial-and-error, which is time-consuming and often yield poor results. Besides, any controller with constant control parameters has limited ability to adapt to dynamic, real-world conditions. This study proposes a second-order sliding-mode position controller. The parameters are improved based on neural network, to control the clutch system to get better tracking performance on-line adaptive during the vehicle starting. Finally, the clutch controller is embedded into a Modelica® based vehicle model and verified through the Model-in-the-Loop(MiL) simulation. The simulation results demonstrate that the controller has better tracking performance and stronger robustness compared to the conventional controller and its parameters can be efficiently tuned online via neural networking as opposed to trial-and-error.

Alterations in thermal hydraulic parameters directly affect the safety of reactors. Accurately predicting the trends of key thermal hydraulic parameters under various working conditions can greatly improve reactor safety, thereby effectively preventing the occurrence of nuclear power plant accidents. The thermal hydraulic characteristic parameters in the reactor are affected by many factors, in order to preliminarily study whose forecasting methods and determine the feasibility of neural network forecasting, the China Experimental Fast Reactor (CEFR) is selected as the research target in this study, and the maximum surface temperature of fuel rod sheath and mass flow rate are used as predictive variables. After data samples are generated through the reactor sub channel analysis code (named SUBCHANFLOW), two widely used adaptive neural networks are used to perform the thermal hydraulic parameter forecast analysis of CEFR fuel assembly under steady-state conditions. The 1/2 core model of CEFR is used to perform a single-step predictive analysis of thermal hydraulic parameters under transient conditions. The results show that the adaptive radial basis function (RBF) neural network exhibits a better fitting ability and higher forecasting accuracy than that of the adaptive back propagation neural network, and the maximum error under steady-state conditions is 0.5%. Under transient conditions, poor forecasting accuracy is observed for some local points; however, the adaptive RBF neural network is generally excellent at predicting temperature and mass flow. The mean relative error of temperature does not exceed 1%, and the mean relative error of flow does not exceed 6.5%. The proposed RBF neural network model can provide real-time forecasting in a short time under unstable flow conditions, and its forecasting results have a certain reference value.

Continual development in construction techniques results in emergence of specialized formwork systems. A new system will have to compete with in-use systems for adoption in a target operation. Thus, it is essential that decision makers anticipate the acceptability of new systems before making decisions to acquire them. Estimating acceptability basically assesses how features of a new system are comparable to that of in-use systems. Therefore, analogy is a focal factor for the acceptability estimating process. Neural networks (NNs) are more suitable to model construction problems requiring analogy-based solutions. A NN-based approach was employed to anticipate the acceptability of new formwork systems. The study collected data from a group of 40 users in Egypt. A set of six performance characteristics that mostly pertain to acceptability estimating were identified. The study used the analytical hierarchy process to produce pairs of a performance characteristics' vector and the corresponding acceptability value, and utilized the developed pairs to train NNs. Finally, tests on trained NNs using unseen data indicated satisfactory performance.