Noncyclic Scheduling of Cluster Tools With a Branch and Bound Algorithm

Traditional rule-based cluster tool scheduling in semiconductor manufacturing faces significant limitations, including inflexibility, reliance on domain-specific expertise, and suboptimal performance in dynamic and complex environments. These methods often struggle to adapt to varying process conditions and equipment configurations, which are common in modern fabrications (fabs). Furthermore, previous research has typically relied on simplified simulators of cluster tools, failing to capture the full complexity of real-world semiconductor manufacturing equipment. To address these challenges, this study examines the potential of deep reinforcement learning (DRL) for optimizing cluster tool scheduling. This research presents a comprehensive simulation environment that models a cluster tool system, including both vacuum (VTM) and atmospheric (ATM) robots. This study progressively evaluates DRL agents, starting with a single-agent deep Q-network (DQN) and advancing to a multi-agent DQN (MADQN) framework to schedule the combined VTM-ATM system. Experimental results demonstrate that the proposed DRL agents consistently outperform traditional rule-based methods in terms of productivity and adaptability. In the complex multi-agent environment, the MADQN agent demonstrated robust performance across all tested configurations, achieving a productivity improvement of up to 8.9% over standard rule-based scheduling methods. These findings highlight the potential of DRL to overcome the limitations of existing scheduling methods and significantly enhance productivity in semiconductor manufacturing.

Timed Petri nets (TPNs) have been widely used for modeling discrete-event systems of diverse manufacturing and service industries. In this article, we introduce a reachability tree-based optimization algorithm to optimize cyclic schedules of TPNs. In particular, we focus on a special class of cyclic schedules that are referred to as one-cyclic schedules, i.e., the algorithm efficiently finds the optimal one-cyclic transition firing schedule of a TPN. The proposed scheduling method can be robustly applied and extended to a number of different scheduling models since the methodology is not bounded to a specific domain. To enhance the computational performance, we establish a set of transition ordering constraints that can reduce the tree size during the search procedure. We evaluate the computational efficiency of the suggested algorithm by examining robotized manufacturing systems where one-cyclic schedules are popularly being used. It is numerically shown that the proposed algorithm is computationally more efficient than the previously studied Petri net-based optimization methods. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —Resource scheduling is one of the most important managerial issues in diverse industrial systems. An optimal scheduling method for a certain industrial system is often locally developed by utilizing domain-specific operational properties. Although such domain-dependent knowledge can contribute to enhancing the computational efficiency of an optimization method, such an approach has a weak point that the method might not be applicable to scheduling problems of different industrial fields. Our motivation is to develop an algorithm for optimizing steady-state schedules that can be robustly applied for various types of discrete-event systems. The algorithm is developed on the basis of the Petri net modeling framework as it is widely being used for describing cyclic behaviors of diverse manufacturing systems, service systems, and social systems. It is experimentally shown that the proposed algorithm is computationally efficient compared with the existing cyclic scheduling methods.

Recently, semiconductor manufacturing fabs tend to reduce the wafer lot size, down to just a few. Consequently, the wafer recipe or wafer flow pattern changes frequently. For such problems, it is impossible to apply conventional prevalent cyclic scheduling methods that repeat processing of wafers in an identical cyclic tool operation sequence. We therefore consider the noncyclic scheduling problem of single-armed cluster tools that process wafers with different recipes. Our proposed method is to transforms the problem into a multiobjective problem by considering the ready times of the resources as objectives to minimize. Only feasible states are generated based on the initial state of the system. These feasible states form a multiobjective shortest path problem and give us as an upper bound for the number of states, where , and are the number of different wafer recipes, wafers, and processing chambers. We solve this problem with implicit enumeration by making our scheduling decisions based on the Pareto optimal solutions for each state. The experimental results show that the proposed algorithm can quickly solve large sized problems including ones with arbitrary initial tool states, changing recipes, reentrant wafer Ωows, and parallel chambers. Note to Practitioners-The scheduling method developed in this paper is designed for scheduling robots used in semiconductor production. The method is able to efficiently represent the state of the manufacturing system and uses this to find an optimal schedule to maximize productivity. The same approach can be used for other manufacturing systems with discrete events. The main advantage of this method is that it can fast find an optimal schedule based on the current state of the system. A long-time horizon can be used because of the high efficiency of the algorithm with respect to the number of jobs. However, the method is best suited for manufacturing systems with few buffers and limited degrees of freedom, in other words highly interconnected systems.

Most cluster tool scheduling studies assume identical access times between chambers or do not discuss the impact of access times. However, the optimal scheduling rule and the cycle time can depend on the access times or physical configuration of parallel chambers. Therefore, we examine cyclic scheduling problems for cluster tools that have nonidentical access times. We first develop Petri net models for tool behaviors and analyze the cycle time by identifying the workloads of the process steps. We prove that the conventional backward and swap sequencing strategies are still optimal for single-armed and dual-armed cluster tools, respectively, when a process step is the bottleneck and the tool repeats a minimal cyclic work cycle. We also present a closed-form formula for the cycle time and identify a coprime condition on the number of parallel chambers for which the cycle time is independent of the order of using parallel chambers. Finally, we develop a mixed integer programming model for cases in which the coprime requirement is not satisfied.

Mapping high-level program specifications onto microprogrammable pipelines raises complex scheduling problems. This paper studies the problem of optimizing the throughput of microprograms for recurrent vector loops on a quite general architecture model, taking into account many resource requirements, in particular register allocation. For a given architecture and a given loop specification, it is shown that the feasible microprograms are solutions of a cyclic scheduling problem modelled with a Timed Petri net. A resolution graph that represents the behaviour of the net is built. Critical cycles of this graph are then used to build cyclic optimal solutions.

This paper proposes and analyzes an efficient robot task sequence for a flexible assembly system that consists of multiple 3D printers, post-processing, assembly, inspection machines, and a material handling robot. The flexible assembly system, which has been built in many large cities in Korea, has been designed to fabricate customized products for start-ups or individuals. We consider cyclic scheduling of the system in which the robot repeats a certain sequence to produce identical products. We propose a robot task sequence to minimize the cycle time. The system behavior with the proposed robot task sequence is modeled with a timed event graph (TEG), and the optimality of the sequence is proved by analyzing the circuits in the TEG. The results are extended for producing multiple types of products simultaneously.

Timed Petri nets can be used to model and analyse scheduling problems. To support the modelling of scheduling problems, we provide a method to map tasks, resources and constraints onto a timed Petri net. By mapping scheduling problems onto Petri nets, we are able to use standard Petri net theory. In this paper we will show that we can use Petri net based tools and techniques to find conflicting and redundant precedences, upper- and lower-bounds for the makespan, etc. This is illustrated by a Petri net based analysis of the notorious 10×10 problem due to Fisher & Thompson (1963)

In the above paper by Zuberek, conflict-free Petri nets with deterministic timing are used for modeling cluster tools. Performance analysis of the models is based on P-invariants. This comment tries to clarify some statements in the paper and points out a more efficient performance-analysis approach based on linear programming, which is of polynomial time complexity.

Cyclic scheduling and operation of a residency time- constrained single-arm cluster tool with failure-prone process modules are highly challenging. In some cases, when a fail- ure occurs, there still exists a feasible cyclic schedule for the performance-degraded tool. In other cases, such a schedule no longer exists. For the latter, it is highly desired to respond to a process module failure properly such that the tool can continue working and the wafers in the tool can be completed in a feasi- ble way. This work is the first one to study this important issue. The idea is to apply Petri nets to describe the dynamic behav- ior of a single-arm cluster tool. With the developed Petri net model, this paper formulates failure response policies to control the cluster tool such that it can keep working without violat- ing any residency time constraint. The failure response policies are implemented via efficient real-time control laws. Illustrative examples are presented to show their usage. Index Terms—Wafer fabrication, Cluster tools, Petri net, Scheduling, Failure response.

Integrated processing tools are becoming increasingly prevalent in modern fabs. Integrated tools consist of several process modules connected around a central handler such that the modules can process several wafers from the same lot simultaneously. Lithography cells and vacuum cluster tools are examples of integrated processors. Simple, intuitive models of the cycle time, throughput, and wafer cost of integrated tools are introduced. The models use two measurable parameters that aggregate tool operations: the incremental cycle time is the average increase in cycle time resulting from a lot size increment of one wafer and the fixed cycle time is the portion of cycle time that is independent of lot size. One example shows how an empirical analysis of a commercial CVD cluster tool was used with the models to identify opportunities to increase tool throughput. In a second example, analytical models of the incremental and fixed cycle times are used to provide insight into the effects of tool configuration on performance. Finally, simulations of single integrated tools are used to show some limitations of the model.

Scheduling a cluster tool with wafer residency time constraints is challenging and important in wafer manufacturing. With a backward strategy, the scheduling problem of such singlerobot-arm cluster tools is well-studied in the literature. It is much more challenging to schedule a more general case whose optimal scheduling strategy is not limited to the backward one. This work uses a timed Petri net (PN) to model the dynamic behavior of the system and presents a method to determine the optimal scheduling strategy for the system. Based on its PN model and the obtained strategy, it reveals that the key issue to schedule such a tool is to determine when and how long the robot should wait for. Based on this finding, this work establishes for the first time the necessary and sufficient conditions regarding the existence of an optimal and feasible one-wafer cyclic schedule for singlerobot-arm cluster tools. It then formulates a computationally efficient linear program to find it if existing, and finally gives industrial examples to show the application and power of the proposed method.

Scheduling of multicluster tools has received much attention due to its complexity of interaction among cluster tools. A cluster tool comprises several modules such as processing modules, a transfer module with a single or dual-armed handling robot, and loadlock modules. A multicluster tool comprises two, three, or more cluster tools that are interconnected with each other. In this paper, we propose a deadlock prevention policy for noncyclic scheduling of dual-armed multicluster tools. The proposed deadlock prevention policy is effective for generating an efficient schedule for two or three-connected multicluster tools with a single or a dual-path flow. This is proven by analyzing the strict minimal siphon of the Petri net model. The performance of the proposed method is compared with that of a maximal permissible deadlock prevention policy. The results demonstrate that the proposed method is computationally efficient for larger state spaces and more suitable than conventional deadlock prevention policies for generating effective schedules for noncyclic scheduling of multicluster tools. Note to Practitioners —Scheduling of multicluster tools has recently attracted attention in the growing semiconductor manufacturing industry. Most conventional studies on multicluster tools focus on cyclic scheduling to minimize cycle time with swap strategy. The challenge addressed by this paper is to derive a deadlock-free schedule for multicluster tools with noncyclic operations. We propose a simple deadlock prevention policy that can be applied to large-scale timed Petri net (PN) model of general multicluster tools without enumeration of siphon computations or solution of mixed integer programming. A timed PN model is developed for noncyclic operations of dual-armed multicluster tools with a dual ath. The computational results show that the approximately 10% of the total throughput of the proposed method is better than those of the conventional methods. It enables more efficient operations for noncyclic scheduling under transient periods including cleaning operations.

Cluster tools are closed mini production environments that are especially used in the semiconductor industry. They consist of multiple processing chambers. In contrast to single processing tools they can handle multiple lots with different process characteristics at the same time. The major challenge for scheduling cluster tools is the hard to predict process times of even comparable lots as they strongly depend on a changing product mix over the tool. In this paper a mathematical model for the external scheduling of a cluster tool workstation is presented. The goal is to minimize the weighted cycle time by accounting for the changing cycle times of each lot. A MIP model assigns lots to tools of a workstation and determines for each lot the chambers, which are used during the processing steps inside the cluster tool. Finally the developed method is compared with a simulation that uses dispatching strategies.

Lagrangian Relaxation Technique for Solving Scheduling Problems by Decomposition of Timed Petri Nets

The dynamic scheduling problem (DSP) in unreliable flexible manufacturing systems (UFMSs) with concurrency, conflicts, resource sharing, and sequential operations is a complex optimization problem that requires the use of efficient solution methodologies. The effectiveness of scheduling UFMSs relies on the quality of equipment maintenance. Currently, UFMSs with consistently large queues of parts awaiting service employ a repair-after-failure approach as a standard maintenance procedure. This method may require unexpected resources, incur costs, consume time, and potentially disrupt the operations of other UFMSs, either partially or fully. This study suggests using a predictive maintenance (PdM) strategy that utilizes the Internet of Things (IoT) to predict and avoid early mechanical equipment failures before they happen in UFMSs, thereby reducing unplanned downtime and enhancing reliability. Therefore, the objective of this paper is to construct timed Petri net (TPN) models using the IoT for the PdM configuration of mechanical equipment in the dynamic scheduling problem of UFMSs. This necessitates that users represent the specific problem using TPNs. The process of PN modeling requires the utilization of domain knowledge pertaining to the target problems as well as to machine information. However, it is important to note that the modeling rules for PNs are straightforward and limited in number. Consequently, the TPN model is applied to generate and formulate mixed-integer linear programming (MILP) instances accurately. This is done to identify the optimal production cycle time, which may be implemented in real-life scenarios. Several UFMS instances are used to demonstrate the applications and effectiveness of the proposed method. The computational results demonstrate that the proposed method shows superior solution quality, effectively solves instances for a total of 10 parts and 6 machines, and achieves a solution in a reasonable CPU time.