Optimal replacement policy for the case where the damage process is a one-sided Lévy process

It is well-known in the standard age replacement policy that a finite preventive replacement time does not exist when the failure time is exponential and the optimal preventive replacement time is nonrandom. It is shown that when the failure time is exponential, a finite time exists by introducing the shortage and excess costs. In addition, the random age replacement is proposed and similar discussions are made. Furthermore, the periodic and random inspection policies are taken up, and their optimal policies are shown to correspond theoretically to those of the age replacement ones. It is shown finally that when the random inspection cost is the half of the periodic one, two expected costs are almost the same.

Maintenance optimization of a production system with buffer capacity

Optimal replacement policy for a deteriorating system with increasing repair times

Condition-based maintenance using the inverse Gaussian degradation model

In this paper, we propose a new model for availability maximization under partial observations for maintenance applications. Compared with the widely studied cost minimization models, few structural results are known about the form of the optimal control policy for availability maximization models. We consider a failing system with unobservable operational states. Only the failure state is observable. System deterioration is driven by an unobservable, continuous-time homogeneous Markov process. Multivariate condition monitoring data which is stochastically related to the unobservable state of the system is collected at equidistant sampling epochs and is used to update the posterior state distribution for decision making. Preventive maintenance can be carried out at any sampling epoch, and corrective maintenance is carried out upon system failure. The objective is to determine the form of the optimal control policy that maximizes the long-run expected average availability per unit time. We formulate the problem as an optimal stopping problem with partial information. Under standard assumptions, we prove that a control limit policy is optimal. A computational algorithm is developed, illustrated by numerical results.

We study the optimal behavior of an investor who is forced to withdraw funds continuously at a fixed rate per unit time (e.g., to pay for a liability, to consume, or to pay dividends). The investor is allowed to invest in any or all of a given number of risky stocks, whose prices follow geometric Brownian motion, as well as in a riskless asset which has a constant rate of return. The fact that the withdrawal is continuously enforced, regardless of the wealth level, ensures that there is a region where there is a positive probability of ruin. In the complementary region ruin can be avoided with certainty. Call the former region the danger-zone and the latter region the safe-region. We first consider the problem of maximizing the probability that the safe-region is reached before bankruptcy, which we call the survival problem. While we show, among other results, that an optimal policy does not exist for this problem. we are able to construct explicit ∊-optimal policies, for any ∊ > 0. In the safe-region, where ultimate survival is assured, we turn our attention to growth. Among other results, we find the optimal growth policy for the investor, i.e., the policy which reaches another (higher valued) goal as quickly as possible. Other variants of both the survival problem as well as the growth problem are also discussed. Our results for the latter are intimately related to the theory of Constant Proportions Portfolio Insurance.

Tangential hand velocity profiles of rapid human arm movements often appear as sequences of several bell-shaped acceleration-deceleration phases called submovements or movement units. This suggests how the nervous system might efficiently control a motor plant in the presence of noise and feedback delay. Another critical observation is that stochastic-ity in a motor control problem makes the optimal control policy essentially different from the optimal control policy for the deterministic case. We use a simplified dynamic model of an arm and address rapid aimed arm movements. We use reinforcement learning as a tool to approximate the optimal policy in the presence of noise and feedback delay. Using a simplified model we show that multiple submovements emerge as an optimal policy in the presence of noise and feedback delay. The optimal policy in this situation is to drive the arm's end point close to the target by one fast submovement and then apply a few slow submovementsto accurately drive the arm's end point into the target region. In our simulations, the controller sometimes generates corrective submovements before the initial fast submovement is completed, much like the predictive corrections observed in a number of psychophysical experiments.

This paper investigates the optimal state-dependent replacement policy for semi-Markovian deteriorating systems. It is assumed that the replacement time, replacement cost, operating cost rate and sojourn time distributions are all state-dependent. The optimization criterion is to minimize the expected long-run cost per unit time. Under some sufficient conditions on the cost and time parameters, we show that the optimal policy is of control limit type and has monotonic properties which facilitate the derivation of the optimal policy. Finally, a numerical example is given to illustrate the structure of the optimal policy.

We consider a physical asset consisting of complex systems, where the systems may require upgrades during the lifetime of the asset. In practice, the asset owner and system supplier can make the upgrade decisions together, requiring a decision-support model that can be jointly used to optimize the total benefit for both parties. Motivated by a real-life use case including an asset owner and a system supplier, we build a continuous-time model to optimize the upgrade decisions of a system during the fixed lifetime of the asset. In our model, we capture the key critical factors that drive the upgrade decisions: increasing functionality requirements due to evolving technology, age-dependent maintenance costs, a predetermined overhaul plan of the asset, and the lifetime of the asset. A system upgrade is less costly if it is executed jointly with an asset overhaul. We first analyze the case with no additional cost of upgrading outside an overhaul. We analytically characterize the structure of the optimal upgrade policy under various realistic assumptions that lead to different types of cost functions. We then use these results as a building block to characterize the optimal policy for a generalized cost function. When there is a penalty for upgrading outside an overhaul moment, we propose a dynamic programming approach that efficiently determines the optimal upgrade policy by using our analytical results. We also prove that as this penalty increases, the optimal policy can only change to one where the number of upgrades not jointly executed with overhauls is reduced. However, the optimal number of upgrades is a nonincreasing function of this penalty. Also, surprisingly, more overhauls may lead to a smaller number of upgrades under the optimal policy. Funding: This publication is part of the project “Maritime Remote Control Tower for Service Logistics Innovation (MARCONI)” (project 439.18.309) of the research program “Integrator-Logistics as Enabler for Enhancing Society,” which is (partly) financed by the Dutch Research Council (NWO).

We consider the economic behavior of a single-channel queuing system operating with the following cost structure: a server start-up cost, a server shut-down cost, a cost per unit time when the server is turned off, a cost per unit time when the server is turned on, and a holding cost per unit time spent in the system for each customer. For the undiscounted, infinite-horizon problem, bounds are obtained for the cost rate and optimal policy when the interarrival time distribution is a member of the class of increasing-failure-rate distributions. Some of the bounds hold for all interarrival-time distributions.

In this paper, a cold standby repairable system consisting of two identical components and one repairman is studied. Assume that each component after repair is not “as good as new”, by using a geometric process, we consider two kinds of repair replacement policy, one based on the working age T of component 1 under which the system is replaced when the working age of component 1 reaches T, and the other based on the failure number N of component 1 under which the system is replaced when the failure number of component 1 reaches N. Our problem is to choose optimal replacement policies T* and N* respectively such that the long-run average cost per unit time of the system is minimized. And we can prove under some mild conditions that the optimal policy N* is better than the optimal policy T*. Finally, a numerical example for policy N is given.

본 논문에서는 수리가 가능한 시스템에 대하여 혼합보증이 종료된 이후의 교체모형과 예방보전모형을 고려하는데, 만약 보증기간이 종료된 이후에 시스템에 고장이 발생하면 최소수리를 수행한다. 최적의 교체정책과 최적의 예방보전정책을 결정하기 위한 기준으로는 기대비용과 기대비가동시간에 근거한 총밸류함수를 사용한다. 그리고 시스템의 고장시간이 와이블분포를 할 때 수치적 예를 통해서 제안된 최적의 교체정책 및 예방보전정책을 자세히 설명하고자 한다. This paper considers the replacement model and the preventive maintenance model following the expiration of combination warranty for a repairable system. If the system fails after the combination warranty is expired, then it is minimally repaired at each failure. The criterion used to determine the optimal replacement policy and the optimal preventive maintenance policy is the overall value function based on the expected cost rate per unit time and the expected downtime per unit time. The numerical examples are presented for illustrative purpose when the failure time follows a Weibull distribution.

In this paper, a reinforcement learning method called Q-learning is applied to find a near optimal control policy for controlling power system stabilizers (PSS). The single agent approach is used, but the design procedure can be expanded to a multi-agent system. The objective of the control policy is to enhance the stability of a multi-machine power system by increasing the damping ratio of the least damped modes. By learning a near optimal policy, not only the design parameters of the PSSs are simultaneously optimized, but also the agents can track the system changes and update the parameters of PSSs. The off-line mode of operation is used in this paper after achieving the near optimal policy. The validity of proposed method has been tested on a 2 area, 4 machines power system.

Optimal circular economy index policy in a production system with carbon emissions

The problem addressed in this study is that of determining how to allocate the workstation processing and buffering capacity in a capacitated re-entrant line to the job instances competing for it, in order to maximize its long-run/steady-state throughput, while maintaining the logical correctness of the underlying material flow, i.e., deadlock-free operations. An approximation scheme for the optimal policy that is based on neuro-dynamic programming theory is proposed, and its performance is assessed through a numerical experiment. The derived results indicate that the proposed method holds considerable promise for providing a viable, computationally efficient approach to the problem and highlight directions for further investigation. Note to Practitioners-Sequencing and scheduling problems arising in the context of contemporary manufacturing environments are known to be extremely hard. For this reason, in most practical situations, these problems have been resolved through the application of a number of heuristics-i.e., rules of thumb that are expected to provide reasonable performance. Things are complicated even further in the automated versions of these environments, since the applied sequencing and scheduling logic must guarantee, in addition to good performance, logically correct and smooth operation. Both the logical and the performance-oriented control problem of flexibly automated production systems can be-and have been-addressed through formal systems theory. However, a challenging remaining problem is the approximation of the derived optimal policies in a way that will maintain near optimality, and at the same time, it will be computationally tractable in the context of the real-world applications. Our past work has addressed this approximation problem primarily with respect to the optimal logical control policy. The work presented in this paper undertakes the complementary problem of approximating the optimal scheduling policy. To this end, we employ some recently emerged results from a field known as neuro-dynamic programming (which is essentially a systematic approximation framework for dynamic programming). Our results indicate that the proposed approximation framework holds considerable promise toward developing a systematic analytical methodology for deriving near-optimal and logically correct scheduling policies for flexibly automated production systems. More specifically, it is shown that, when applied to some prototypical problems concerning the scheduling of re-entrant lines with finite buffering capacity at their workstations, the proposed approximation framework: 1) effectively integrates past results concerning the logical control of these environments and 2) the obtained performance is consistently superior to the performance provided by the typically used heuristics.