The Multiple Objectives Flexible Jobshop Scheduling Using Reinforcement Learning

Job shop scheduling, as one of the classical scheduling problems, has been widely studied in literatures, and proved to be mostly NP-hard. Although it is extremely difficult to solve job shop scheduling with no-wait constraint to optimality, the two-machine no-wait job shop scheduling to minimise makespan could be solvable in polynomial time when each job has exactly two equal length operations (proportionate job shop). In the present paper, an extension is attempted by considering a proportionate flexible two-stage no-wait job shop scheduling problem with minimum makespan, and a set-covering formulation is put forward which contains a master problem and a pricing problem. To solve this problem, a column generation (CG)-based approach is implemented. In comparison, a mixed integer programming model is constructed and optimised by Cplex. A series of randomly generated numerical instances are calculated. And the testing result shows that the mixed integer model handled by Cplex can only solve small scale cases, while the proposed CG-based method can conquer larger size problems in acceptable time.

The classical job shop scheduling (JSS) problem can be extended by allowing processing of an operation by any machine from a given set. This type of scheduling is known as flexible job shop scheduling (FJSS) problem. It incorporates all the difficulties and complexities of its predecessor classical problem. However, it is more complex as it is required to determine the assignment of operations to the machine. Swarm intelligence techniques proved their effectiveness in solving a wide range of complex NP-Hard real world problems. One of these techniques is the meerkat clan algorithm (MCA) that has been successfully applied to various optimization problems. This paper presents a modified MCA for solving the FJSS problem. The modification is based on using harmony search (HS). The introduction of HS provides more exploitation and intensification. HS generates various solutions, which are provided to the MCA. As a result, the exploitation of the local optimum is increased, which in turn increases the convergence rate. The experimental results show that the improved method achieves higher quality schedules. Additionally, the convergence rate is speeded up compared with the standalone algorithm. This gives the proposed method the superiority over the original algorithm.

Flexible Job Shop Scheduling (FJSS) problem has many real-world applications such as manufacturing and cloud computing, and thus is an important area of study. In real world, the environment is often dynamic, and unpredicted job orders can arrive in real time. Dynamic FJSS consists of challenges of both dynamic optimisation and the FJSS problem. In Dynamic FJSS, two kinds of decisions (so-called routing and sequencing decisions) are to be made in real time. Dispatching rules have been demonstrated to be effective for dynamic scheduling due to their low computational complexity and ability to make real-time decisions. However, it is time consuming and strenuous to design effective dispatching rules manually due to the complex interactions between job shop attributes. Genetic Programming Hyper-heuristic (GPHH) has shown success in automatically designing dispatching rules which are much better than the manually designed ones. Previous works only focused on standard job shop scheduling with only the sequencing decisions. For FJSS, the routing rule is set arbitrarily by intuition. In this paper, we explore the possibility of evolving both routing and sequencing rules together and propose a new GPHH algorithm with Cooperative Co-evolution. Our results show that co-evolving the two rules together can lead to much more promising results than evolving the sequencing rule only.

Scheduling for the flexible job shop is very important in both fields of production management and combinatorial optimization. However, when we attempt to formulate job shop scheduling problems which closely describe and represent the real world problem, various factors involved are often imprecisely or ambiguously known to the analyst. Also, in the real situations, machines are not continuously available due to preventive maintenance activity. Besides, in the production system, there are many users and machines in various priorities. Often, user satisfactions, machine stability and schedule itself are in conflict. By considering the imprecise or fuzzy nature of the data in real world problem, job shop scheduling with uncertain processing time and machine available constraint is introduced. Under the fuzzy circumstance, we attempt to make a feasible solution for flexible job shop scheduling problem (FJSSP) based on a hybrid optimization approach with multi-objective which not only to minimize the makespan, but also maximize the user satisfaction and machine stability. In the experiment, we make the comparisons with some other algorithms, such as genetic algorithm, to demonstrate the hybrid approach is much better on efficient search and stability with three benchmark examples. Sensitivity analysis is conducted to study the impact, in term of total objective, when the proportion of machine stability is varied.

A Petri net with controller is used to model discrete events in flexible job shop scheduling, the objective of scheduling problems is to minimize make-span, the scheduling results is obtained based on genetic and Tabu Search (TS) algorithm. It is proved capable of providing optimized schedule to the job-shop where the machine tool and manpower resources are both constrained. After crossover and mutation operations, an optimal or suboptimal scheduling plan can be found. The result of the test shows that this method is feasible and efficient. Job shop scheduling is the primary content and critical technology of the production management of CIMS, in the past during 30 years, a lot of researchers and results focus on it, but most of them deal with single objective and resource scheduling problems which are fixed processing route and constrained by machine, and production period is its optimal objective. Nevertheless, the production styles are variable and processing route is determined under the special condition. Moreover, job shop scheduling problem is the most difficult combined constraints optimal problem and typical NP hard problem which can't be solved in polynomial time by an effective algorithm. Up to date, there are some traditional algorithms to solve JSP problem (1-2). About paper (3), Petri net is used to construct JSP model and L1 which combines branch and bound algorithm with dynamic program is used to search its optimal solution. And paper (4), the job sequence is assigned by genetic code firstly, and then filtered beam search which based on breadth-first algorithm is adopted to search. In recent years, the Petri net as a discrete event dynamic system modeling and analysis tool(5), has been successfully applied to the flexible manufacturing system modeling, analysis and control. Some researchers put forward to be based on Petri network scheduling method, but only on the production line itself can not reflect the modeling, as in the production line of external control scheduling strategy influence. A hybrid algorithm is proposed to solve dual-resource scheduling problem in flexible production environment, and Petri net model with a controller, which can effectively model control. It combines the advantage of global search ability of GA with the self-adaptive merit of Tabu Search (TS) and improves its convergence. It is proved capable of global and part searching ability and efficient.

Dynamic scheduling is a complicated problem with high research value in the production field, because it requires a fast response to the dynamic disturbance event. In this paper, knowledge graph is combined with reinforcement learning to solve this problem. In the proposed method, knowledge graph model is constructed by a series of steps, including ontology-based knowledge modeling, knowledge extraction, and knowledge fusion. In the knowledge graph model, the state information of the shop floor is extracted from the production data, including the information about machine, job, and process. On this basis, reinforcement learning model is designed, which is composed of machine assignment agent and job sequence agent. Thus, the real-time scheduling decisions can be made rapidly under the dynamic production environment. The effectiveness of the proposed method is verified on a case study of aviation component production workshop. Compared with other methods, the proposed method obtains the best results.

Production scheduling is a critical requirement in today’s production systems. The production scheduling problem focuses on the rational allocation of production resources, determining the processing order of production tasks and operations, satisfying production constraints arising from the production environment such as the number of machines and task duration, and achieving optimal production goals, such as completing all tasks in the fastest way possible. The Flexible Job Shop Scheduling (FJSP) problem is a classical production scheduling issue, an extension of the Job Shop Scheduling (JSP) problem. The JSP problem involves allocating a set of jobs to a set of machines, where each job consists of multiple operations that must adhere to a precedence order. Each machine can process only one operation at a time for each job, and the time required for each operation on the corresponding machine is predetermined. The challenge in the JSP problem is to determine an appropriate order for processing operations or assignments, based on the production objective. FJSP represents a more complex and flexible variant of this problem, where at least one operation of a task can be performed on a set of alternative machines. Given that FJSP problems are NP-hard, solutions tend to be complex, varied, and often have limited applicability. Many FJSP scenarios require targeted modeling. In this paper, I propose a modeling approach for the static scheduling problem of job shops, considering low flexibility and demonstrating its usability by comparing it with other scheduling approaches, such as those based on genetic algorithm modeling. The paper is organized as follows:The first part of this paper offers a brief introduction to the FJSP problem, including background, research motivation, and main contributions, the second part provides a literature review,including a brief review and introduction of recent solutions to the FJSP problem, the third part describes a specific modeling approach, the fourth part details the test experiments and results of the modeling approach and includes comparisons with some other approaches, the fifth part analyzes the results from the fourth part and discusses the advantages and disadvantages of the modeling approach relative to others, and the sixth part concludes with a summary of findings and recommendations.

The Flexible Job Shop Scheduling Problem (FJSP) is an extension of the classical job shop scheduling problem, which is characterized by the fact that each process can be processed on multiple candidate machines, and needs to solve the two subproblems of machine allocation and process sequencing simultaneously. Since FJSP is an NP-hard problem, its complexity and multi-objective characteristics make the traditional exact methods inefficient. At the same time, the existing intelligent optimization algorithms are prone to falling into local optimums, which makes it difficult to balance global exploration and local exploitation capabilities. To this end, this study proposed a Levy flight-based Harmony Search algorithm (LHS), which effectively avoids premature convergence by dynamically and adaptively adjusting the Harmony Memory Considering Rate (HMCR), the probability of Pitch Adjusting Rate (PAR), and the arbitrary distance Bandwidth(BW), and by introducing a Levy flight mechanism to perturb the parameters to broaden the search space and enhance the diversity of the population. We validate the experiments using 8 × 8, 10 × 10, and 10 benchmark instances proposed by Brandimarte in the literature, and the experimental results show that the Harmony search algorithm based on Levy flight outperforms the other comparative algorithms in terms of the solution quality and the convergence speed, demonstrating the effectiveness of its solution for FJSP.

The Job Shop Scheduling Problem (JSSP) attracts many researchers due to its combinatorial nature and its discovery in numerous practical applications. This type of problem is characterized by high computational complexity; therefore, solving large-sized problems is not accessible with exact optimization methods. Very often, real JSSP problems can be presented as Flexible Job Shop Scheduling Problems (FJSSP). For these problems, there are single-criterion and multi-criteria mathematical models. On the other hand, the ways to solve this type of problems include exact methods and heuristic or metaheuristic algorithms. This paper the aim to review the progress of research in the field of solving FJSSP over the last 10 years, as well as to show current trends for future scientific developments in this area.

With the increasing demand for hospital services amidst the COVID-19 pandemic, allocation of limited public resources and management of healthcare services are of paramount importance. In the field of patient flow scheduling, previous research primarily focused on classical-based objective functions, while ignoring environmental-based objective functions. This study presents a flexible job shop scheduling problem to optimize patient flow and, thereby, minimize the total carbon footprint, as the sustainability-based objective function. Since flexible job shop scheduling is an NP-hard problem, a metaheuristic optimization algorithm, called Chaotic Salp Swarm Algorithm Enhanced with Opposition-Based Learning and Sine Cosine (CSSAOS), was developed. The proposed algorithm integrates the Salp Swarm Algorithm (SSA) with chaotic maps to update the position of followers, the sine cosine algorithm to update the leader position, and opposition-based learning for a better exploration of the search space. generating more accurate solutions. The proposed method was successfully applied in a real-world case study and demonstrated better performance than other well-known metaheuristic algorithms, including differential evolution, genetic algorithm, grasshopper optimization algorithm, SSA based on opposition-based learning, quantum evolutionary SSA, and whale optimization algorithm. In addition, it was found that the proposed method is scalable to different sizes and complexities.

Optimizing Automated Manufacturing Processes Using a Hybrid BRKGA Algorithm: A Case Study on Flexible Job-Shop Scheduling

The classic job shop scheduling problem mainly focuses on one kind of manufacturing resources, such as machine tools, and etc. But the job shop scheduling in practical production activities always needs to consider the constraints of different manufacturing resources. In this paper, a double-resource flexible job shop scheduling problem (DFJSSP) is presented. Both machines and workers are considered in the process of job shop scheduling in this DFJSSP. And a genetic algorithm (GA) is used to solve this problem, in which a new well designed three-layer chromosome encoding method has been adopted and some effective crossover and mutation operators are designed. Finally, a case study is used to validate the method.

An effective job shop scheduling (JSS) in the manufacturing industry is helpful to meet the production demand and reduce the production cost, and to improve the ability to compete in the ever increasing volatile market demanding multiple products.In so many combinatorial optimization problems, job shop scheduling problems have earned a reputation for being difficult to solve. Job-shop scheduling is essentially an ordering problem. A new encoding scheme for a classic job-shop scheduling problem is presented. The aim is to find an allocation for each job and to define the sequence of jobs on each machine so that the resulting schedule has a minimal completion time.Genetic algorithm that has demonstrated considerable success in providing efficient solutions to many non polynomial-hard optimization problems is used to solve job-shop scheduling problem. The schedules given by genetic algorithms are constructed using a priority rule and under several constraints. After a schedule is obtained a checking operation is applied to ensure that the solution is feasible. The approach is tested on a set of instances. The results validate the effectiveness of the algorithm.

Solving job shop scheduling problems (JSSPs) with a fixed strategy, such as a priority dispatching rule, may yield satisfactory results for several problem instances but, nevertheless, insufficient results for others. From this single-strategy perspective finding a near optimal solution to a specific JSSP varies in difficulty even if the machine setup remains the same. A recent intensively researched and promising method to deal with difficulty variability is Deep Reinforcement Learning (DRL), which dynamically adjusts an agent's planning strategy in response to difficult instances not only during training, but also when applied to new situations. In this paper, we further improve DLR as an underlying method by actively incorporating the variability of difficulty within the same problem size into the design of the learning process. We base our approach on a state-of-the-art methodology that solves JSSP by means of DRL and graph neural network embeddings. Our work supplements the training routine of the agent by a curriculum learning strategy that ranks the problem instances shown during training by a new metric of problem instance difficulty. Our results show that certain curricula lead to significantly better performances of the DRL solutions. Agents trained on these curricula beat the top performance of those trained on randomly distributed training data, reaching 3.2% shorter average makespans.

Job shop scheduling is an important decision process in contemporary manufacturing systems. Most existing research has been focused on the minimization of makespan (i.e. the completion time of all jobs). However, due date related performances are actually crucial for maintaining a high service reputation in the competitive market. In this paper, we propose a novel particle swarm optimization (PSO) algorithm for solving the job shop scheduling problem with the objective of minimizing total weighted tardiness. A local search procedure based on processing time perturbations is designed and embedded into the framework of PSO. The computational experiments for a wide range of test instances show that the hybrid PSO converges fast to high-quality solutions.