A Deep Reinforcement Learning Approach for Fault Detection in Continuous Chemical Processes

Fault detection and diagnosis in process data using one-class support vector machines

Multiscale monitoring of industrial chemical process using wavelet-entropy aided machine learning approach

The fault detection and diagnosis of continuous process is very important for the production safety and product quality. Owing to its no need to know much about the process mechanism and exact process model ,the data driven method, typically the principal component analysis (PCA) has attracted much attention by chemical researchers for monitoring process. PCA is powerful in fault detection ,however, it has difficulties in diagnosing fault correctly in complex process. In this paper, the multiblock principal component analysis (MBPCA) is applied for fault detection and diagnosis in continuous process, which uses the integral PCA to detect fault and block contribution and variables contribution to diagnose fault. The simulations on the Tennessee Eastman process show that the proposed method can not only detect fault quickly ,but also find the fault location exactly.

Reinforcement learning (RL), a promising framework for data-driven decision making in an uncertain environment, has successfully been applied in many real-world operation and control problems. However, the application of RL in a large-scale decentralized multi-agent environment remains a challenging problem due to the partial observability and limited communications between agents. In this paper, we develop a model-based kernel RL approach and a model-free deep RL approach for learning a decentralized, shared policy among homogeneous agents. By leveraging the strengths of both these methods, we further propose a novel deep ensemble multi-agent reinforcement learning (MARL) method that efficiently learns to arbitrate between the decisions of the local kernel-based RL model and the wider-reaching deep RL model. We validate the proposed deep ensemble method on a highly challenging real-world air traffic control problem, where the goal is to provide effective guidance to aircraft to avoid air traffic congestion, conflicting situations, and to improve arrival timeliness, by dynamically recommending adjustments of aircraft speeds in real-time. Extensive empirical results from an open-source air traffic management simulation model, developed by Eurocontrol and built on a real-world data set including thousands of aircrafts, demonstrate that our proposed deep ensemble MARL method significantly outperforms three state-of-the-art benchmark approaches.

Fault detection and diagnosis for non-linear processes empowered by dynamic neural networks

The practical application of traditional data‐driven techniques for process monitoring encounters significant challenges due to the inherent nonlinear and dynamic nature of most industrial processes. Aiming at the problem of nonlinear dynamic process monitoring, a novel fault detection method based on dynamic kernel principal component analysis combined with weighted structural difference (DKPCA‐WSD) is proposed in this paper. Initially, the proposed method leverages a sophisticated nonlinear transformation to project the augmented matrix of the original input data into a high‐dimensional feature space, thereby facilitating the establishment of a DKPCA model. Subsequently, the WSD statistic is computed, utilizing a widely known sliding window technique, to quantify the mean and standard deviation differences across data structures. Ultimately, the WSD statistic is utilized for fault detection, completing the process monitoring task. By integrating the capability of DKPCA to capture nonlinear dynamic characteristics with the effectiveness of the WSD statistic in mitigating the impact of non‐Gaussian data distributions, DKPCA‐WSD significantly enhances the monitoring performance of traditional DKPCA in nonlinear dynamic processes. The proposed method is evaluated through a numerical case exhibiting nonlinear dynamic behaviors and a simulation model of a continuous stirred tank reactor. A comparative analysis with conventional methods, including principal component analysis (PCA), dynamic principal component analysis, KPCA, PCA similarity factor (SPCA), DKPCA, and moving window KPCA (MWKPCA), demonstrates that DKPCA‐WSD outperforms traditional fault detection techniques in nonlinear dynamic processes, offering a substantial improvement in monitoring performance.

The Process Chemometrics Approach to Process Monitoring and Fault Detection

Comparison Between Statistical and Observer-Based Approaches for Fault Detection and Isolation in a Chemical Process

This paper presents a fault detection and diagnosis approach based on Mahalanobis distance (MD) for monitoring continuous processes. The Mahalanobis distance is first used to detect sensor faults, and its control limit is determined by the empirical method via specification a significance level. The main idea behind Mahalanobis distance is using the whole data information instead of dimensionality techniques such as partial least squares, canonical variate analysis, and principal component analysis. The data reflected by the detection index, MD, is analysed by a fault diagnosis method, i.e, contributions plots. An integrated approach for fault detection and diagnosis is presented and tested by two simulation studies, Monte Carlo simulation data and continuous stirred tank reactor chemical process(CSTR). The results illustrate the reliability of the proposed approach in detecting and diagnosing sensor faults.

Recently, principal component analysis (PCA) and independent component analysis (ICA) have attracted increasing attention for fault detection in industrial processes. However, industrial processes usually have multimodal characteristics, in which case, the performance of traditional PCA and ICA methods is poor. A fault detection method based on the local entropy double subspace (LEDS) is proposed that effectively supervises the multimodal process with multiple steady modes. The proposed method uses the local probability density (LPD) to eliminate the multimodal characteristics of the data. Furthermore, local entropy (LE) is applied to mine the statistical information associated with the process data. The process variables are classified into Gaussian and non-Gaussian subspaces by the Kolmogorov–Smirnov (KS) test. PCA and ICA models are built in Gaussian and non-Gaussian subspaces, respectively, for fault detection. A Bayesian decision method is employed to transform the detection results into fault probabilities, and the detection results are combined with the final statistical information. This method effectively eliminates the multicentre structure of the data and improves the fault detection performance of traditional PCA and ICA. The proposed method is applied to the Tennessee Eastman multimodal process. The simulation results show that the fault detection rate of LEDS is better than the rates of traditional methods when the false alarm rate is low for most faults.

Reinforcement Learning (RL) is a machine learning methodology that develops the capability to make sequential decisions in intricate issues using trial-and-error techniques. RL has become increasingly prevalent for decision-making and control tasks in diverse fields such as industrial processes, biochemical systems and energy management. This review paper presents a comprehensive examination of the development, models, algorithms and practical uses of RL, with a specific emphasis on its application in process control. The study examines the fundamental theories, methodology and applications of RL, classifying them into two categories: classical RL such as such as Markov decision processes (MDP) and deep RL viz., actor critic methods. RL is a topic of discussion in multiple process industries, such as industrial chemical process control, biochemical process control, energy systems, wastewater treatment and the oil and gas sector. Nevertheless, the paper also highlights challenges that hinder its larger acceptance, including the requirement for substantial computational resources, the complexity of simulating real-world settings and the challenge of guaranteeing the stability and resilience of RL algorithms in dynamic and unpredictable environments. RL has demonstrated significant promise, but more research is needed to fully integrate it into industrial and environmental systems in order to solve the current challenges. Abbreviations: AC: Actor critic; AI: Artificial intelligence; ANN: Artificial neural networks; A3C: Asynchronous advantage actor critic; CRL : Classical Reinforcement learning; CV : Controlled variable; DDPG : Deep deterministic policy gradient; DQN: Deep Q network; DRL: Deep reinforcement learning; DP: Dynamic programming; FOMDP: Fully observable Markov decision process; GRU: Gated recurrent unit; LQR: Linear quadratic regulator; LSTM: Long short-term memory; ML: Machine learning; MV : Manipulated variable; MC: Monte Carlo; MDP: Markov decision process; MPC: Model predictive controller; MIMO: Multi input multi output; PG: Policy gradient; PID: Proportional integral derivative; PPO: Proximal policy optimisation; RL: Reinforcement learning; PPO: Proximal policy optimisation; SAC: Soft actor critic; SISO: Single input single output; TD: Temporal difference; TRPO: Trust region policy optimisation; TD3: Twin delayed deep deterministic policy gradient.

Modified performance-enhanced PCA for incipient fault detection of dynamic industrial processes

Fault detection in linear systems is a fairly matured area where the well-known principal component analysis (PCA) and its variants are widely used. However, a large class of non-linear systems exist, especially chemical processes, on which such techniques cannot be applied. The present work aims at demonstrating the application of self-organising maps (SOM) for fault detection in non-linear processes. SOM belongs to the class of unsupervised and competitive learning algorithms and it is highly capable of handling nonlinear relationships. Application of SOM to fault detection involves generation of a reference template for the process under fault-free conditions. Online fault detection is performed by generating a new template using a windowing of the data, which is compared with the reference template using a novel metric based on the node weights obtained from SOM to detect possible faults in the process. Simulation studies on two non-linear systems, namely, (1) continuously stirred tank reactor (CSTR) and (2) bioreactor process demonstrate the practicality and utility of the proposed method.

A Data-Driven Fault Detection and Diagnosis by NSGAII-t-SNE and Clustering Methods in the Chemical Process Industry

Path planning refers to that the mobile robot can obtain the surrounding environment information and its own state information through the sensor carried by itself, which can avoid obstacles and move towards the target point. Deep reinforcement learning consists of two parts: reinforcement learning and deep learning, mainly used to deal with perception and decision-making problems, has become an important research branch in the field of artificial intelligence. This paper first introduces the basic knowledge of deep learning and reinforcement learning. Then, the research status of deep reinforcement learning algorithm based on value function and strategy gradient in path planning is described, and the application research of deep reinforcement learning in computer game, video game and autonomous navigation is described. Finally, I made a brief summary and outlook on the algorithms and applications of deep reinforcement learning.