A two-phase soft sensor modeling framework for quality prediction in industrial processes with missing data

Soft sensor modeling plays a crucial role in industrial processes by enabling real-time prediction of product quality through the use of process variables that are difficult to measure directly. However, traditional soft sensor models often struggle to effectively capture the complex temporal dynamics and deep interdependencies among variables in high-dimensional process data.This paper proposes a novel Indirect Core Allocation Model (ICAM) to address this issue. ICAM efficiently fuses and processes multivariable process data through core aggregation and allocation. The model leverages an indirect attention mechanism to integrate information from various channels with linear complexity, and dynamically selects key process variables through random pooling. This process generates a global indirect core representation, which is then efficiently allocated to the embedding sequences of original samples, facilitating the sharing and collaborative optimization of process variable information. Specifically, ICAM balances the exploration and exploitation of information by dynamically selecting key variables during the training phase. During the testing phase, a probability-based weighted averaging method is employed to ensure the stability and consistency of the prediction results. To verify the effectiveness of ICAM, we conducted modeling experiments on penicillin fermentation, hot rolling, and injection molding industrial process datasets with high data dimensionality. The experimental results demonstrate that ICAM outperforms other advanced models in terms of prediction accuracy and shows superior performance in industrial process quality prediction. Our data and code will be published at https://github.com/pisa-lut/ICAM.

High-precision prediction of material properties is an advanced direction of hot rolled product technology in manufacturing processes. However, complex disturbance usually produces wrong or missing data in the acquisition and entry stages. The long process and the diverse operating conditions bring great challenges to constructing soft sensors for the hot rolling process. To address this problem, a new time-series soft sensor model framework is proposed for mechanical properties prediction. Firstly, in order to process the raw dataset, the isolation forest algorithm is used to remove the outliers, and the random forest algorithm is used to fill in missing values. Secondly, the auxiliary variables are extracted by the combination of the XGBoost algorithm and prior mechanistic knowledge. Then, a DA-CBGRU-Seq2Seq time-series model based on a two-stage attention mechanism is designed for soft sensor modeling. In the encoder stage, external factors are introduced by an entity embedding layer. A combination structure of bidirectional gated recurrent units and convolutional layers is designed for extracting and learning the input features adaptively with the input attention mechanism. In the decoder stage, the long-term time dependence of the time series is learned by the temporal attention mechanism. Finally, the reliability of the proposed sensor model is compared and verified by the hot rolling process. The results indicate that the proposed method has an accuracy of over 95% in predicting the mechanical properties of a real hot-rolling process dataset, and has better performance compared to other comparative soft sensor prediction methods.

In industrial processes, dynamic changes are one of the factors restricting the performance of soft sensor models. Meanwhile, the inconsistency of sensor sampling rates often leads to the problem of mismatch between process variables and quality variables. This paper proposes a semi-supervised soft sensor modeling method based on sample convolution and interactive networks (SCINet). To extract the dynamic information of industrial processes more fully, an unsupervised time series dynamic feature extractor was designed based on SCINet and an autoencoder, and the feature extractor was trained using complete data. The dynamic features encoded by the dynamic feature extractor were transferred to the eXtreme Gradient Boosting (XGBoost) ensemble model with strong generalization ability. The semi-supervised soft measurement model SSCI-XGBoost was established. The effectiveness of dynamic feature transfer and model performance improvement was verified on the industrial process dataset.

BackgroundAiming at the problem that traditional transfer methods are prone to lose data information in the overall domain-level transfer, and it is difficult to achieve the perfect match between source and target domains, thus reducing the accuracy of the soft sensor model.MethodsThis paper proposes a soft sensor modeling method based on the transfer modeling framework of substructure domain. Firstly, the Gaussian mixture model clustering algorithm is used to extract local information, cluster the source and target domains into multiple substructure domains, and adaptively weight the substructure domains according to the distances between the sub-source domains and sub-target domains. Secondly, the optimal subspace domain adaptation method integrating multiple metrics is used to obtain the optimal projection matrices Ws and Wt that are coupled with each other, and the data of source and target domains are projected to the corresponding subspace to perform spatial alignment, so as to reduce the discrepancy between the sample data of different working conditions. Finally, based on the source and target domain data after substructure domain adaptation, the least squares support vector machine algorithm is used to establish the prediction model.ResultsTaking Pichia pastoris fermentation to produce inulinase as an example, the simulation results verify that the root mean square error of the proposed soft sensor model in predicting Pichia pastoris concentration and inulinase concentration is reduced by 48.7% and 54.9%, respectively.ConclusionThe proposed soft sensor modeling method can accurately predict Pichia pastoris concentration and inulinase concentration online under different working conditions, and has higher prediction accuracy than the traditional soft sensor modeling method.

Soft Sensor for Melt Index Prediction Based on Long Short-Term Memory Network

Soft sensor model development in multiphase/multimode processes based on Gaussian mixture regression

Data‐driven soft sensor models have been extensively utilized in industrial processes. Batch processes are usually employed to manufacture low‐volume and high value‐added products in chemical, materials, and pharmaceutical industries. The most distinctive features of batch process lie in nonlinear, repetition, and slow time varying characteristics. In this paper, a data‐driven soft sensor modelling method based on linear slow feature analysis (LSFA) and least squares support vector regression (LSSVR) is proposed. In this method, LSFA was used to effectively capture the driving force behind the data features that change as slowly as possible. Then, a LSSVR model was constructed between the extracted slow feature variables and quality variables. Finally, a numerical example, industrial penicillin fermentation processes, and cobalt oxalate synthesis process were utilized to confirm the prediction accuracy and model reliability of the proposed approach.

Improving the robustness of the soft sensor model of industrial processes is an important yet challenging problem for a large amount of noise interference and missing data in practical industrial data. In this article, an adversarial training-based deep supervised variational autoencoder (Adv-DSVAE) is proposed to enhance the performance of industrial soft sensor models. Specifically, a supervised variational autoencoder (SVAE) is first designed to extract the quality-relevant feature representation. Then, a deep SVAE (DSVAE) model is constructed by stacking the hidden features extracted by SVAE, such that a high-level output-related feature representation can be captured. In this way, the missing data situation can be handled by the probabilistic latent feature representation extracted in DSVAE. To improve the robustness of a DSVAE-based soft sensor model, an adversarial training method is designed, in which adversarial examples are generated by adding perturbations to the last hidden feature of DSVAE, such that the model can perform well on both clean and perturbed feature representations. We further provide theoretical convergence analysis for the proposed Adv-DSVAE to guarantee its successful practical application. The ablation studies confirm that industrial quality prediction using the adversarial training strategy can ensure better robustness. Case studies on both the debutanizer column process and the real-world aluminum electrolysis process validate the superiority of Adv-DSVAE.

Soft sensing technique has been extensively used to predict key quality variables in industrial systems. However, due to the difficulty of quality variable acquisition, only limited labeled data samples are available, and a large number of unlabeled ones are discarded. This raises a big challenge to build a high-quality soft sensor model. In order to furthest exploit information contained in both the labeled and unlabeled data, this paper proposes a principal component-based semi-supervised extreme learning machine (referred to as PCSELM) model. Through this model, extracting latent features and learning nonlinear input-output relationship can be simultaneously performed. In this way, unlabeled samples are utilized efficiently for feature representation and model accuracy improvement. Moreover, mixed regularizations are employed to work in conjunction with the PCSELM to obtain high generality and flexibility. We also derive an efficient parameter learning algorithm with theoretically guaranteed convergence. Comprehensive experiments are conducted via an industrial process. Comparison results illustrate that the proposed PCSELM outperforms other representative semi-supervised algorithms. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —Industrial processes in general incorporate unlabeled samples which are ubiquitous in real world applications. The focus of this paper is to develop a semi-supervised soft sensor model (PCSELM) that is capable to learn the nonlinear features and regression relationship efficiently with both the labeled and unlabeled samples. The proposed model can automatically implement the feature representation and the input-output relationship description. In addition, we introduce mixed norms for the model objective function to improve the final prediction performance and generalization. A feasible model optimization technique with proved convergence is also derived. Experimental results based on a real industrial dataset manifest that PCSELM achieves better prediction accuracy than its peers.

Soft sensors development for industrial reactive distillation processes under small training datasets

Online quality prediction for cobalt oxalate synthesis process using least squares support vector regression approach with dual updating

Missing data is a very common challenge in health monitoring systems and one reason for that is that they are largely dependent on different types of sensors. A critical characteristic of the sensor-based prediction systems is their dependency on hardware, which is prone to physical limitations that add another layer of complexity to the algorithmic component of the system. For instance, it might not be realistic to assume that the prediction model has access to all sensors at all times. This can happen in the real-world setup if one or more sensors on a device malfunction or temporarily have to be disabled due to power limitations. The consequence of such a scenario is that the model faces "missing input data" from those unavailable sensors at the deployment time, and as a result, the quality of prediction can degrade significantly. While the missing input data is a very well-known problem, to the best of our knowledge, no study has been done to efficiently minimize the performance drop when one or more sensors may be unavailable for a significant amount of time. The sensor failure problem investigated in this paper can be viewed as a spatial missing data problem, which has not been explored to date. In this work, we show that the naive known methods of dealing with missing input data such as zero-filling or mean-filling are not suitable for senors-based prediction and we propose an algorithm that can reconstruct the missing input data for unavailable sensors. Moreover, we show that on the MobiAct, MotionSense, and MHEALTH activity classification benchmarks, our proposed method can outperform the baselines by large accuracy margins of 8.2%, 15.1%, and 11.6%, respectively.

A soft sensor modeling method with dynamic time-delay estimation and its application in wastewater treatment plant

In recent years, deep learning has been applied to build soft sensor models. Compared to the classical latent variable models, the model based on deep learning has a good performance in tackling nonlinearity in process data. However, static soft sensor based on deep neural network (SSSDNN) fails to take into account the dynamic characteristics, which are unavoidable in some applications. To improve the performance of the soft sensor, a novel dynamic soft sensor model based on impulse response template and deep neural network (DSSDNN) is proposed by means of Wiener structure, and an iterative algorithm will be used to train this novel model. A case study based on real debutanizer column data demonstrates the desirable prediction performance of DSSDNN and shows that the DSSDNN is able to obtain better approximation accuracy than the static soft sensor model in nonlinear processes with dynamic characteristics.

Dynamic historical information incorporated attention deep learning model for industrial soft sensor modeling