Process Adjustment for Assemblies

A state-space process-control model involving adjustment error and deterministic drift of the process mean is presented. The optimal adjustment policy is developed by dynamic programming. This policy calls for a particular adjustment when a Kalman-filter estimator is outside a deadband defined by upper and lower action limits. The effects of adjustment cost, adjustment variance, and drift rate on the optimal policy are discussed. The optimal adjustment policy is computed for a real machining process, and a simulation study is presented that compares the optimal policy to two sensible suboptimal policies.

Minimizing Risk Models in Markov Decision Processes with Policies Depending on Target Values

Heterogeneous traffic types (interactive messages, file transfers, facsimile, etc.) compete for access to a shared resource (satellite channel, bus, etc.). If a packet is not immediately given access to the resource, it waits in a buffer. We assume that the buffer capacity is infinite, that the traffic types arrive according to independent Poisson processes and that the period over which a packet occupies the resource has an arbitrary type-dependent distribution. The traffic types are partitioned into two groups: interactive and noninteractive. The average delay of each of the interactive types is required to below given thresholds. The problem of finding an optimal scheduling policy that minimizes a linear combination of the average delays for the noninteractive types while meeting the design constraints is considered. Simple necessary and sufficient conditions are derived for the existence of a policy that satisfies the constraints. An algorithm is given that decomposes the traffic types into an ordered arrangement of groups, and the existence of a policy that gives strict priority accordingly is established. Under weak conditions on the costs and rates, it is shown that all optimal policies must have this structural property. Sensitivity and aggregation analyses are given. Finally, exploiting the above decomposition, an optimal policy is constructed and is shown to have many appealing properties.

The time needed to complete a software development project is hard to estimate in practice. In addition to the uncertainty about the duration of the individual development steps, software projects often suffer from unexpected rework and delays. As a result, project scheduling remains a difficult task for the managers of software projects. In this paper, we compute optimal scheduling strategies for a set of sample software projects. The computations are based on a stochastic Markov decision model for software projects which focuses on capturing the feedback between concurrent development activities. The optimal strategies are computed using a variant of the value iteration algorithm from stochastic dynamic programming. Since the underlying process model is stochastic, the strategies are stochastically optimal, that is, they minimize the expected project duration. The ultimate goal of this research is to develop guidelines for managers how to schedule their software projects under uncertainty in the best possible way. The sample projects are similar, but differ in certain characteristics of the project or product, such as the strength of the coupling between the components or the degree of specialization of the teams on the tasks. By using a set of related projects, we can study how the project characteristics influence the optimal scheduling decisions in a project. In addition to computing the optimal scheduling policies for the sample projects, we use extensive simulations to compare the performance of the optimal policy against the so-called list policies for each given setting. List policies are a simple, but commonly used class of scheduling policies. For our sample projects, we find that the best list policy in general is not optimal. The performance gap is the larger the higher the degree of specialization of the teams is. On the other hand, the stronger the coupling between the components, the smaller is the improvement which the optimal policy achieves over the best list policy.

Management Insights

Optimal risk adjustment with adverse selection and spatial competition

We consider a dynamic pricing model for selling a given stock of a perishable product over a finite time horizon. Customers, whose reservation price distribution changes over time, arrive according to a nonhomogeneous Poisson process. We show that at any given time, the optimal price decreases with inventory. We also identify a sufficient condition under which the optimal price decreases over time for a given inventory level. This sufficient condition requires that the willingness of a customer to pay a premium for the product does not increase over time. In addition to shedding managerial insight, these structural properties enable efficient computation of the optimal policy. Numerical studies are conducted to show the revenue impact of dynamic price policies. Price changes are set to compensate for statistical fluctuations of demand and to respond to shifts of the reservation price. For the former, our examples show that using optimal dynamic optimal policies achieves 2.4–7.3% revenue improvement over the optimal single price policy. For the latter, the revenue increase can be as high as 100%. These results explain why yield management has become so essential to fashion retailing and travel service industries.

This paper investigates the optimal state-dependent replacement policy for semi-Markovian deteriorating systems. It is assumed that the replacement time, replacement cost, operating cost rate and sojourn time distributions are all state-dependent. The optimization criterion is to minimize the expected long-run cost per unit time. Under some sufficient conditions on the cost and time parameters, we show that the optimal policy is of control limit type and has monotonic properties which facilitate the derivation of the optimal policy. Finally, a numerical example is given to illustrate the structure of the optimal policy.

In this paper, we use the event-based optimization framework to study the admission control problem in an open Jackson network. The external arriving customers are controlled by an admission controller. The controller makes decision only at the epoch when an event of customer arrival happens. Every customer in the network obtains a fixed reward for each service completion and pays a cost with a fixed rate during its sojourn time (including waiting time and service time). The complete information of the system state is not available for the controller. The controller can observe only the number of total customers in the network. Based on the property of closed form solution of Jackson networks, we prove that the system performance is monotonic with respect to the admission probabilities and the optimal control policy has a threshold form. That is, when the number of total customers is smaller than a threshold, all of the arriving customers are admitted; otherwise, all are rejected. The sufficient condition and necessary condition of optimal policy are also derived. Furthermore, we develop an iterative algorithm to find the optimal policy. The algorithm can be executed based on a single sample path, which makes the algorithm online implementable. Simulation experiments are conducted to demonstrate the effectiveness of our approach.

Stochastic dynamic programming (SDP) with a conditional value-at-risk (CVaR) criterion for management of storm-water

The cost-effectiveness and value of additional information about a health technology or program may change over time because of trends affecting patient cohorts and/or the intervention. Delaying information collection even for parameters that do not change over time may be optimal. We present a stochastic dynamic programming approach to simultaneously identify the optimal intervention and information collection policies. We use our framework to evaluate birth cohort hepatitis C virus (HCV) screening. We focus on how the presence of a time-varying parameter (HCV prevalence) affects the optimal information collection policy for a parameter assumed constant across birth cohorts: liver fibrosis stage distribution for screen-detected diagnosis at age 50. We prove that it may be optimal to delay information collection until a time when the information more immediately affects decision making. For the example of HCV screening, given initial beliefs, the optimal policy (at 2010) was to continue screening and collect information about the distribution of liver fibrosis at screen-detected diagnosis in 12 years, increasing the expected incremental net monetary benefit (INMB) by $169.5 million compared to current guidelines. The option to delay information collection until the information is sufficiently likely to influence decisions can increase efficiency. A dynamic programming framework enables an assessment of the marginal value of information and determines the optimal policy, including when and how much information to collect.

Corporate income tax constitutes a significant portion of financial costs in business operations. A typical tax system is characterized by the tax structure, accounting method, and loss carryover rule. In this article, we study the optimal inventory policies under taxation and accrual accounting. We consider a lost-sales periodic-review inventory system under a proportional tax with loss carryforward. We formulate the problem as a cyclic stochastic dynamic program with multiple accounting periods, each of which consists of multiple review periods. We show that an income-dependent basestock policy is optimal. We find that tax function convexity and loss carryforward may introduce conflicting incentives into inventory decisions: the former drives risk-averse decisions whereas the latter induces risk-seeking behavior. We identify two intertemporal effects of taxation, the intra-accounting-period effect and the inter-accounting-period effect, both inducing smaller basestock levels. We further extend the analysis to the progressive tax, backordering system, and cash accounting method. Our numerical study examines the effects of various characteristics of taxation on the optimal policies. Our results demonstrate that ignoring taxation in inventory decisions may lead to significant losses especially when the tax rate and demand uncertainty are sufficiently high, which reveals the value of tax considerations in inventory management.

Optimal Guidelines for Screening Mammography Based on Individual Risk Factors

Optimal transmission policies for a two-user multiple access broadcast channel with binary feedback are investigated. The system is modeled as a dynamic team. Conditioned on the channel feedback, the past history of buffer states is redundant to each user. Once this redundant data is removed, the information structure, although non-classical, satisfies the sufficient conditions of Mahajan, Nayyar, and Teneketzis, 2008 for a dynamic team problem to be tractable. Using the idea of a virtual coordinator of Mahajan, Nayyar, and Teneketzis, a dynamic programming decomposition is presented. This dynamic program is defined over a countable state space and a finite action space. When the arrival rates of both users are symmetric, an optimal policy is identified by solving the dynamic program analytically. This policy matches the optimal window protocol proposed by Hluchyj and Gallager, 1981. Thus, this paper presents the an example of a non-trivial dynamic team with non-linear dynamics where an exact analytic solution is obtained.

Problem definition: Production systems deteriorate stochastically due to use and may eventually break down, resulting in high maintenance costs at scheduled maintenance moments. This deterioration behavior is affected by the system’s production rate. Although producing at a higher rate generates more revenue, the system may also deteriorate faster. Production should thus be controlled dynamically to tradeoff deterioration and revenue accumulation in between maintenance moments. We study systems for which the relation between production and deterioration is known and the same for each system and systems for which this relation differs from system to system and needs to be learned on-the-fly. The decision problem is to find the optimal production policy given planned maintenance moments (operational) and the optimal interval length between such maintenance moments (tactical). Methodology/results: For systems with a known production-deterioration relation, we cast the operational decision problem as a continuous time Markov decision process and prove that the optimal policy has intuitive monotonic properties. We also present sufficient conditions for the optimality of bang-bang policies, and we partially characterize the structure of the optimal interval length, thereby enabling efficient joint optimization of the operational and tactical decision problem. For systems that exhibit variability in their production-deterioration relations, we propose a Bayesian procedure to learn the unknown deterioration rate under any production policy. Numerical studies indicate that on average across a wide range of settings (i) condition-based production increases profits by 50% compared with static production, (ii) integrating condition-based production and maintenance decisions increases profits by 21% compared with the state-of-the-art sequential approach, and (iii) our Bayesian approach performs close, especially in the bang-bang regime, to an Oracle policy that knows each system’s production-deterioration relation. Managerial implications: Production should be adjusted dynamically based on real-time condition monitoring and the tactical maintenance planning should anticipate and integrate these operational decisions. Our proposed framework assists managers to do so optimally. Funding: This work was supported by the Nederlandse Organisatie voor Wetenschappelijk Onderzoek [Grant 439.17.708]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/msom.2022.0473 .