Deep reinforcement learning based preventive maintenance policy for serial production lines

Purpose Making decisions on preventive maintenance (PM) policy and buffer sizing, as is often studied, may not result in overall optimization. The purpose of this paper is to propose a joint model that integrates PM and buffer sizing with consideration of quality loss for a degenerating system, which aims to minimize the average operation cost for a finite horizon. The opportunistic maintenance (OM) policy which could increase the output and decrease the cost of the system is also explored. Design/methodology/approach A joint PM and buffer size model considering quality loss is proposed. In this model, the time-based PM and the condition-based PM are taken on the upstream and the downstream machine, respectively. Further, the OM policy based on the theory of constraints (TOC) is also considered. An iterative search algorithm with Monte Carlo is developed to solve the non-linear model. A case study is conducted to illustrate the performance of the proposed PM policies. Findings The superiority of the proposed integrated policies compared with the separate PM policy is demonstrated. Effects of the policies are testified. The advantages of the proposed TOC-based OM policy is highlighted in terms of low-cost and high-output. Originality/value Few studies have been carried out to integrate decisions on PM and buffer size when taking the quality loss into consideration for degenerating systems. Most PM models treat machines equally ignoring the various roles of them. A more comprehensive and integrated model based on TOC is proposed, accompanied by an iterative search algorithm with Monte Carlo for solving it. An OM policy to further improve the performance of system is also presented.

Optimal preventive maintenance policy for leased equipment using failure rate reduction

ABSTRACTA framework for reliability and maintenance analysis of job shop manufacturing systems is proposed in this paper. An efficient preventive maintenance (PM) policy in terms of failure effects analysis (FEA) is proposed. Subsequently, reliability evaluation and component importance measure based on FEA are performed under the PM policy. A job shop manufacturing system is applied to validate the reliability evaluation and dynamic maintenance policy. Obtained results are compared with existed methods and the effectiveness is validated. Some vague understandings for issues such as network modelling, vulnerabilities identification, the evaluation criteria of repairable systems, as well as PM policy during manufacturing system reliability analysis are elaborated. This framework can help for reliability optimisation and rational maintenance resources allocation of job shop manufacturing systems.

A special game against nature is examined in this paper, in which nature controls timings of repairable and non-repairable failures of a system of equipment, and preventive maintenance, repairs and preventive replacement policies are the countermeasures. We optimize the imperfect preventive maintenance policy for a multi-unit system with different initial virtual age units. We develop a binary integer programming problem, where the management decides on the optimal preventive maintenance policy to minimize the total expected maintenance cost not to exceed a given budget. The mathematical formulation is developed for a multi-unit system, considering different preventive maintenance levels for each unit. Numerical examples with sensitivity analysis are performed to illustrate the performance and efficiency of the proposed model. Also, the model is examined for a real case study from railway industry. The results determine the optimal preventive maintenance policy and provide managerial insights based on computational analysis.

PurposeWhile steady-state analysis is useful, it does not consider the inherent transient characteristics of repairable systems' behavior, especially in systems that have relatively short life spans, or when their transient behavior is of special concern such as the motivating example used in this paper, military systems. Therefore, a maintenance policy that considers both transient and steady-state availability and aims to achieve the best trade-off between high steady-state availability and rapid stabilization is essential.Design/methodology/approachThis paper studies the transient behavior of system availability under the Kijima Type II virtual age model. While such systems achieve steady-state availability, and it has been proved that deploying preventive maintenance (PM) can significantly improve its steady-state availability, this improvement often comes at the price of longer and increased fluctuating transient behavior, which affects overall system performance. The authors present a methodology that identifies the optimal PM policy that achieves the best trade-off between high steady-state availability and rapid stabilization based on cost-availability analysis.FindingsWhen the proposed simulation-based optimization and cost analysis methodology is applied to the motivating example, it produces an optimal PM policy that achieves an availability–variability balance between transient and steady-state system behaviors. The optimal PM policy produces a notably lower availability coefficient of variation (by 11.5%), while at the same time suffering a negligible limiting availability loss of only 0.3%. The new optimal PM policy also provides cost savings of about 5% in total maintenance cost. The performed sensitivity analysis shows that the system's optimal maintenance cost is sensitive to the repair time, the shape parameter of the Weibull distribution and the downtime cost, but is robust with respect to changes in the remaining parameters.Originality/valueMost of the current maintenance models emphasize the steady-state behavior of availability and neglect its transient behavior. For some systems, using steady-state availability as the sole metric for performance is not adequate, especially in systems that have relatively short life spans or when their transient behavior affects the overall performance. However, little work has been done on the transient analysis of such systems. In this paper, the authors aim to fill this gap by emphasizing such systems and applications where transient behavior is of critical importance to efficiently optimize system performance. The authors use military systems as a motivating example.

Preventive maintenance (PM) is a maintenance program with activities initiated at predetermined intervals, or according to prescribed criteria, and intended to reduce the probability of failure, or the degradation of the functioning of an item. In the literature, a number of PM models have been introduced to depict the effectiveness of PM. Based on these models, approaches to scheduling PM policies have been considerably studied. This paper attempts to review existing PM models, and investigate their inter-relationships. We then categorize these models into three classes: linear, nonlinear, and a hybrid of both. These three PM model classes depict the relationships of the hazard functions before, and after a PM. Possible extensions to these three PM models are discussed. The statistical properties for models are derived, and approaches to optimizing the PM policy are given.

Most preventive maintenance (PM) models require, as inputs, information on the behaviour of the equipment under failure-only maintenance (FM). Once a schedule of PM has been applied, data arising from failures are affected by the PM. Over life the behaviour that would occur under FM changes. But PM also tends to delay aging processes. This paper examines: 1) how data collected under four kinds of PM policy can be modified to re-assess the FM characteristics, and 2) how to tell whether an apparent change in them is important. The approach takes account of engineering as well as statistical factors. It concludes that: 1. Estimation of distributions and costs as they would be under FM from data collected under PM cannot be accurate but a suboptimal policy can be worse. 2. A different type of PM policy might be required following changes to costs and/or distribution. The optimum under one model might not be the overall optimum. 3. The solutions suggested in the paper remain unproved due to the ``data problem''. Mathematical research alone can never produce workable procedures. A viable methodology will need years of experimentation with real systems. 4. The potential savings are huge.

This paper proposes a statistically planned and individually improved predictive maintenance (SPII PdM) policy for a batch of single-unit systems. The SPII PdM policy simultaneously takes advantage of 1) the capability of classical statistical lifetime distribution based (SLD-based) preventive maintenance (PM) policies for long-term planning, and 2) the capability of PdM techniques for predicting the residual life (RL) of an individual system. Within the framework of a classical SLD-based PM policy aiming at maximizing long-term average availability, two lifetime margins are proposed in the decision making process to further improve the availability of some (but not all) individuals. A numerical experiment based on a typical degradation model is used to compare the performance of the proposed policy with that of the classical SLD-based PM policy. The comparison results show that higher average availability is achieved in the SPII PdM policy with a decent RL prediction model. For practical implementation, the current study demonstrates the possibility of partially applying the emerging PdM techniques in the widely used SLD-based PM policy in a (approximately) theoretically effective manner.

Simultaneous control of production, preventive and corrective maintenance rates of a failure-prone manufacturing system

As an airframe has finite lifetime and has to be designed lightweight, the maintenance of airframe is indispensable to operate aircraft without any serious troubles. After an airframe begins to operate, it suffers stresses and the stress causes the damage such as cracks of the airframe. Cracks grow with operation time and cause catastrophic phenomenon such as the mid-air disintegration when they become greater than a critical size. So, the managerial crack size is prespecified and Preventive Maintenance (PM) undergoes when the inspected crack size exceeds it. In this paper, optimal PM policies of airframe crack failure are discussed. Airframe states are represented as the Markov renewal process, and one-step transition probabilities are discussed. The total expected cost from the start of operation to the end by failure is defined and the optimal PM policies which minimize it is discussed.

Dynamic Bayesian Network Approach to improve the Preventive Maintenance of Multi-State Systems

The purpose of this study is to propose and model periodic and sequential preventive maintenance policies for a system that performs various missions over a finite planning horizon. Each mission can have different characteristics that depend on operational and environmental conditions. These proposed preventive maintenance policies are defined and modeled mathematically. The study of these two policies is based on a dynamic system failure law that takes into account the different missions performed. The first step is to determine the optimal business plan to achieve, i.e. the set of missions to perform in order to maximize the profit of missions minus maintenance costs. Thus, for each plan, we determine the maintenance planning considering two policies. The first preventive maintenance policy is periodic and the objective is to determine the optimal number of preventive maintenance to achieve. For the second policy, namely sequential, we calculate the optimal number of preventive maintenance intervals and the duration of these different intervals.

Proactive preventive maintenance policy for buffered serial production systems based on energy saving opportunistic windows

In recent decades, philosophy behind maintenance has varied consistently due to the changes in complexity of designs, advances in automation and mechanization, adaptation to the fast growing market demand, commercial computation in the sectors, and environmental issues. In mid-forties, simplicity of designs, limited maintenance opportunities, and immaturity of trade culture made enough to perform only fix it when it broke approach, i.e. corrective maintenance, after failures. Last quarter of the 21th century made essential to constitute more conservative and preventive maintenance policies in order to ensure safety, reliability, and availability of systems with longer lifetime and cost effectiveness. Preventive maintenance can provide an economic saving more than 18% of operating cost of systems. In this basis, various stochastic models were proposed as a tool to constitute a maintenance policy to measure system availability and to obtain optimal maintenance periods. This paper presents a general perspective on common stochastic models in maintenance planning such as Homogenous Poisson Process, Non-Homogenous Poisson Process, and Imperfect Maintenance. The paper also introduces two common maintenance policies, block and age replacement policy, using these stochastic models.

A deep reinforcement learning based algorithm for a distributed precast concrete production scheduling