Research on Structural Mechanics Stress and Strain Prediction Models Combining Multi-Sensor Image Fusion and Deep Learning

Multi-sensor data fusion model aims to improve the model performance by fusing multiple types of sensor data. Although multi-sensor data fusion models have been developed for remarkable performance, there is a lack of studies on the adversarial vulnerability of the multi-sensor data fusion models. In this paper, we propose a robust multi-sensor data fusion method that is not vulnerable to adversarial attacks. To this end, we devise a random feature fusion method to preserve multi-sensor fusion features. Through the random feature fusion, we could explicitly hide the information about which features are being used for the fusion. In experiments, we verify that our proposed random feature fusion method shows the adversarial robustness considerably under diverse adversarial settings.

This paper presents a novel multisensor fusion method using the fuzzy ARTMAP neural network and Dempster-Shafer theory of evidence in INS/CCD/SAR (inertial navigation system/optical charged device/synthetic aperture radar) integrated systems. In this method, the CCD and SAR systems work independently. Fusion work takes effect in the decision-making level, not in the pixel level. First, this method use fuzzy ARTMAP NN to classify both the CCD and SAR real-time images according to the image quality, then those 'good' real-time images are matched with CCD and SAR reference images respectively, and matching positions obtained; and then, the two matching positions are fused by the D-S theory of evidence to judge which position is more accurate; finally, the fusion result is integrated with INS to attain the optimal state estimation of navigation/guidance parameters. Simulation results show that this fusion method can improve the precision of the end guidance system.

In the frame of humanitarian antipersonnel mines detection, a multisensor fusion method using the Dempster-Shafer evidence theory is presented. The multisensor system consists of two sensors-a ground penetrating radar (GPR) and a metal detector (MD). For each sensor, a new features extraction method is presented. The method for the GPR is mainly based on wavelets and contours extraction. First simulations on a limited set of data show that an improvement in detection and false alarms rejection, for the GPR as a standalone sensor, could be obtained. The MD features extraction method is mainly based on contours extraction. All of these features are then fused with the GPR ones in some specific cases in order to determine a new feature. From these results, belief functions, as defined in the evidence theory, are then determined and combined thanks to the orthogonal sum. First results in terms of detection and false alarm rates are presented for a limited set of real data and a comparison is made between the two cases: with or without multisensor fusion.

Research on mobile robots began in the late 1960s. Mobile robots are a typical autonomous intelligent system and a hot spot in the high-tech field. They are the intersection of multiple technical disciplines such as computer artificial intelligence, robotics, control theory and electronic technology. The product not only has potentially very attractive application value and commercial value, but the research on it is also a challenge to intelligent technology. The development of mobile robots provides excellent research for various intelligent technologies and solutions. This dissertation aims to study the research of multi-sensor information fusion and intelligent optimization methods and the methods of applying them to mobile robot related technologies, and in-depth study of the construction of mobile robot maps from the perspective of multi-sensor information fusion. And, in order to achieve this function, combined with autonomous exploration and other related theories and algorithms, combined with the Robot Operating System (ROS). This paper proposes the area equalization method, equalization method, fuzzy neural network and other methods to promote the realization of related technologies. At the same time, this paper conducts simulation research based on the SLAM comprehensive experiment of the JNPF-4WD square mobile robot. On this basis, the high precision and high reliability of robot positioning are further realized. The experimental results in this paper show that the maximum error of the X-axis and Y-axis, FastSLAM algorithm is smaller than EKF algorithm, and the improved FASTSALM algorithm error is further reduced compared with the original FastSLAM algorithm, the value is less than 0.1.

Data fusion has been a trend in the field of imaging and signal/image analysis. Although multisensor data fusion is still not regarded as a formal professional discipline, tremendous progress has been made since the publication of the first edition of this book in 1992. With this second edition, the authors have been successful in updating us with state-of-the-art methods and techniques in multisensor data fusion. The book involves both algorithms and software tools, and also covers contemporary subjects like smart agents, cognitive aides, and so on. This book is written according to the Joint Directors of Laboratories (JDL) data fusion group model. There are five levels in the JDL model. The first level deals with association, correlation, estimation, and identification in the data domain. The second and third levels perform knowledge-based processing and utilize expert systems. The fourth level is focused on process monitoring and optimization. The fifth level is devoted to human computer interaction. Chapter 1 serves as an overview. Then, Chapter 2 introduces the JDL model and associated algorithms. Chapters 3–6 cover processing techniques at level one. Chapter 7 gives methods at levels two and three. Chapter 8 targets the control of sensor and information resources at level four. Chapter 9 is for data fusion at level five. Chapter 10 discusses implementation of fusion systems. Chapters 11 and 12 describe emerging applications and information management. The book contains 100 equations, 75 illustrations and key references. The typesetting quality is generally excellent but it would be better if some labels in the figures have been put in larger size. Note that the additons in this book include materials on fusion system control, DARPA's TRIP model, and applications in data warehousing, medical equipment, and defense systems. Hall (associate dean for research, Pennsylvania State University School of Information Sciences and Technology) and McMullen (captain, US Air Force) are well known experts in the field, and should be congratulated for accomplishing such an excellent job in summarizing the up-to-date essential ideas and results on multisensor fusion. Overall, the book is very informative and not difficult to read for electrical and computer engineers as well as technical managers. These types of practitioners can gain solid advice from the book regarding selection of data fusion methods, balance of trade-offs among commercial off-the-shelf tools, development of multisensor data fusion systems and their applications to solve real-world problems. However, to build a sophisticated data fusion system one may need other references for more comprehensive mathematical theories, more rigorous statistical treatments, and more technical details. The cited literature in the book represents appropriate pointers for that purpose. A good example for further reading is the Handbook [1]. As an engineer in biomedical imaging, I would like to underline the relevance of this book to research on systems biomedicine. Given the completion of the Human Genome Project [2] and the momentum of the Physiome Project [3], in 2003 the NIH defined its Roadmap to guide our efforts towards most important biomedical research, including studies on biological building blocks, pathways, and networks using molecular libraries, imaging, bioinformatics and computational biology. Consequently, multi-modality signal sensing and imaging plays a critical role, which often necessarily generates heterogeous sparse/huge datasets that are multi-dimensional, multi-spectral, dynamic, noisy and/or incomplete. The data fusion techniques discussed in this book are closely related to that for biomedical image registration/fusion. Major challenges and new opportunities are enormous ahead of us. In that context, I hope this book will have a significant impact on the biomedical imaging areas, and recommend it to biomedical imaging researchers in particular and other interested engineers in general.

The support vector machine (SVM) is an algorithm based on structure risk minimizing principle, having high generalization ability. In the course of multi-sensor information fusion of industrial control, sensor has bigger nonlinearity and fuzzy relation between coefficient and relevant parameter. A kind of model and algorithm of multiple sensor information fusion based on the support vector machine are proposed. The model offered a kind of effective way for little sample, non-linear, high dimension.

Magnetic flux leakage (MFL) inspection robots are widely used in the pipeline inspection industry to obtain MFL information from multisensor sources. Multisensor fusion is required for precise defect characterization. In this article, a multisensor fusion framework is presented to fuse and enhance MFL information for defect characterization. First, a space-vector-based information fusion method is proposed for better reflecting the spatial distribution of real magnetic field, where the direction and amplitude information of magnetic field are considered simultaneously. Second, considering the information incompletion caused by unstable inspection environment and abnormal device, the improved conditional generative adversarial nets are designed to enhance MFL information. These networks can effectively reconstruct multisensor information by refining the generator loss function, so that the enhanced MFL information can be better applied for defect characterization. Experimental results are compared with state of the art in detail, which highlight the superiority of the proposed multisensor fusion method in MFL inspection.

The increasing interest in Electro-encephalogram (EEG)-based stress prediction is driven by the global prevalence of stress. However, current studies predominantly rely on machine learning and deep learning techniques, utilizing extensive EEG data from 8 to 32 channels for stress prediction. In contrast, our research proposes an innovative approach that predicts stress using only 2 EEG channels and focuses on a specific frequency band (beta). The dataset used in this work is collected and pre-processed in a novel approach which is discussed in depth. Moreover, we have transformed the entire system into a TFLite model to enhance portability. Our experimental results, conducted on 10 subjects, demonstrate that our proposed technique achieves a remarkable prediction accuracy of 74%. Notably, this performance is comparable to other models that employ up to 128-channel data and consider multiple frequency bands. Our work lays the foundation for future advancements, with the ultimate goal of developing a portable EEG-based headband featuring only 2 channels. This would enable stress prediction, and the results could be easily accessed through either a mobile or web interface. By streamlining the EEG data acquisition and focusing on a specific frequency band, our approach not only achieves impressive prediction accuracy but also paves the way for the development of more user-friendly and accessible stress prediction technologies. This has the potential to significantly impact stress management and well-being on a global scale.

This study proposed two multisensor fusion methods for segment-based image classification utilizing a region-growing segmentation. The proposed algorithms employ a Gaussian-PDF measure and an evidential measure respectively. In remote sensing application, segment-based approaches are used to extract more explicit information on spatial structure compared to pixel-based methods. Data from a single sensor may be insufficient to provide accurate description of a ground scene in image classification. Due to the redundant and complementary nature of multisensor data, a combination of information from multiple sensors can make reduce classification error rate. The Gaussian-PDF method defines a regional measure as the PDF average of pixels belonging to the region, and assigns a region into a class associated with the maximum of regional measure. The evidential fusion method uses two measures of plausibility and belief, which are derived from a mass function of the Beta distribution for the basic probability assignment of every hypothesis about region classes. The proposed methods were applied to the SPOT XS and ENVISAT data, which were acquired over Iksan area of of Korean peninsula. The experiment results showed that the segment-based method of evidential measure is greatly effective on improving the classification via multisensor fusion.

The mechanical behavior of aqueduct structures exhibits highly complex characteristics during prestress tensioning, making it difficult for the traditional double-control method to accurately predict and real-time control the key stresses. To improve the construction safety of prestressed tensioning and the prediction accuracy of structural prestress responses, this study develops a rapid structural mechanical property prediction method based on machine learning. Taking prestressed aqueducts as the research object, a system of “finite element simulation—sample generation—machine learning prediction” is established. Firstly, multiple groups of tensioning parameter combinations are designed via Latin hypercube sampling, and the stress responses are obtained through finite element analysis to form a high-quality training sample library. Subsequently, critical structural features are extracted based on mesh reconstruction, and stress prediction models are established using the K-Nearest Neighbors (KNN) and Random Forest algorithms respectively; the prediction performance of both models is compared and validated against finite element simulation results. Furthermore, the prediction outputs of the optimal machine learning model were used to analyze the stress distribution and potential stress concentration issues of the structure during the tensioning process. The comparative analysis results indicate that the Random Forest model performs best in terms of stress prediction accuracy and stability, and its prediction results are highly consistent with those of the finite element method. Compared with traditional finite element condition analysis, the machine learning model can complete multi-condition stress prediction in a shorter time. Leveraging its high-efficiency prediction capability, local high-stress areas of the structure in the tensioning construction scheme can be identified, thereby providing effective optimization schemes to improve the stress distribution. The mechanical response prediction method for the prestress tensioning process of aqueducts, with machine learning as the core, constructed in this paper realizes the rapid and reliable prediction of key stresses throughout the entire prestress tensioning process. This method can be applied to assist in optimizing tensioning construction schemes and construction monitoring, providing a practical technical solution for safety control of aqueduct structures during the prestress construction stage.

Bearing is a key element in high-speed electric multiple unit (EMU) and any defect of it can cause huge malfunctioning of EMU under high operation speed. This paper presents a new method for bearing fault diagnosis based on least square support vector machine (LS-SVM) in feature-level fusion and Dempster-Shafer (D-S) evidence theory in decision-level fusion which were used to solve the problems about low detection accuracy, difficulty in extracting sensitive characteristics and unstable diagnosis system of single-sensor in rolling bearing fault diagnosis. Wavelet de-nosing technique was used for removing the signal noises. LS-SVM was used to make pattern recognition of the bearing vibration signal, and then fusion process was made according to the D-S evidence theory, so as to realize recognition of bearing fault. The results indicated that the data fusion method improved the performance of the intelligent approach in rolling bearing fault detection significantly. Moreover, the results showed that this method can efficiently improve the accuracy of fault diagnosis.

In order to improve the dynamic optimization capability and fault-tolerant ability of the information fusion method for multi-sensor system, the theory of Dynamic Bayesian Networks was used to rebuild the conventional Federated Kalman Filter in this paper, and a new kind of active and dynamic information fusion and optimization method for multi-sensor systems under high-dynamic situation was proposed. The simulation results indicated the high dynamic flexibility and fault-tolerant ability of the proposed method.

Recent work has developed maximum likelihood (ML) methods for track-to-track data association and fusion in a multisensor, i.e., more than two sensor, environment. In order to conserve bandwidth, only the state estimates and corresponding covariance matrices are shared amongst the nodes. The fusion engine uses this track information to determine which tracks associate to the same target and then computes a fused track to improve the accuracy of the state estimates. The simplest class of ML methods assumes that the track errors from different sensors are uncorrelated. The more computationally demanding ML methods incorporate the cross-correlations that are due to the common process noise in the kinematic model of the target. In order to account for track correlations in practice, the cross-covariance matrices must be approximated from the single sensor covariance matrices. This paper introduces new methods to approximate the cross-covariance matrices, and these approximations lead to a third class of association and estimation methods. The paper then uses simulations to assess the performance of the different association and estimation techniques. The simulations include results when the sensor tracks are produced by either a Kalman filter or an interacting multiple model (IMM) filter.

This paper designs a fixed-wing UAV postural reference system, which uses magnetometer, accelerometer and gyroscope (MARG) to perform the attitude measurement of high precision. A quaternion is calculated using Gyroscope sensor by means of quaternion algorithm, and we can get another quaternion using Magnetometer and Accelerometer sensor in the same way. The postural data are calculated using both two quaternions by two data fusion methods, adaptive complementary filter and extended Kalman filter, respectively. We compared data precision from the two filter methods in experiments. The results show that both this two fusion methods can meet the required precision of attitude data, in which Kalman filter method is more accurate than adaptive complementary filter although the latter is simpler and more practical than the former.

In this paper, bending test of an inflatable, polyvinylchloride (PVC)-coated fabric tube is carried out to study the deformation, bearing capacity, and wrinkling characteristics. In order to illustrate the effect of hoop stress, two nonlinear stress-strain relationships are measured through the uniaxial tension and the biaxial tension tests. Meanwhile, a computing method based the fiber beam model is developed for the inflatable tube. This method can consider nonlinearity of membrane materials and wrinkling effect easily. The numerical results using the stress-strain relationship considering hoop stress effect are shown to more accurately predict the load-displacement response in bending test. As the test results shown, the cross-section remains basically circular and the strain distribution in no wrinkling region approximately satisfies the plane section assumption before and after wrinkling. Furthermore, the wrinkling load of the inflatable tube in bending test is compared with the numerical results. Lastly, wrinkling expansion process and wrinkling angle at ultimate load are detailed.