A Memetic Algorithm With Reinforcement Learning for Sociotechnical Production Scheduling

Study on deep reinforcement learning techniques for building energy consumption forecasting

This paper presents a multi-objective mathematical model for a flexible job shop scheduling problem (FJSSP) with fuzzy processing times, which is solved by a hybrid intelligent algorithm (HIA). This problem contains a combination of a classical job shop problem with parallel machines (JSPM) to provide flexibility in the production route. Despite the previous studies, the number of parallel machines is not pre-specified in this paper. This constraint with other ones (e.g., sequence-dependent setup times, reentrant workflows, and fuzzy variables) makes the given problem more complex. To solve such a multi-objective JSPM, Pareto-based optimization algorithms based on multi-objective meta-heuristics and multi-criteria decision making (MCDM) methods are utilized. Then, different comparison metrics (e.g., quality, mean ideal distance, and rate of achievement simultaneously) are used. Also, this paper includes two major phases to provide a new model of the FJSSP and introduce a new proposed HIA for solving the presented model, respectively. This algorithm is a hybrid genetic algorithm with the SAW/TOPSIS method, namely HGASAW/HGATOPSIS. The comparative results indicate that HGASAW and HGATOPSIS outperform the non-dominated sorting genetic algorithm (NSGA-II) to tackle the fuzzy multi-objective JSPM.

Deep reinforcement learning (DRL) is one of the emerging areas of machine learning which focuses on maximized rewards. DRL is a type of machine learning that combines reinforcement learning and deep learning. It uses a series of algorithms to enable an agent to learn how to make decisions in a complex environment. DRL is a subset of artificial intelligence that focuses on making decisions based on the environment and the rewards associated with each action.The goal of DRL is to maximize the long-term reward of an agent. In order to do this, the agent must use a combination of deep learning, reinforcement learning and other AI techniques to learn which actions will lead to the highest reward. DRL is used to solve a variety of problems, from playing video games to controlling robots. It is also used in autonomous driving and robotics, as well as for financial trading. DRL is a powerful tool for solving complex problems and has been used in a variety of research projects. DRL has the potential to revolutionize the way we interact with machines and the environment.

The flexible job shop scheduling problem (FJSP) holds significant importance in both theoretical research and practical applications. Given the complexity and diversity of FJSP, improving the generalization and quality of scheduling methods has become a hot topic of interest in both industry and academia. To address this, this paper proposes a Preference-Based Mask-PPO (PBMP) algorithm, which leverages the strengths of preference learning and invalid action masking to optimize FJSP solutions. First, a reward predictor based on preference learning is designed to model reward prediction by comparing random fragments, eliminating the need for complex reward function design. Second, a novel intelligent switching mechanism is introduced, where proximal policy optimization (PPO) is employed to enhance exploration during sampling, and masked proximal policy optimization (Mask-PPO) refines the action space during training, significantly improving efficiency and solution quality. Furthermore, the Pearson correlation coefficient (PCC) is used to evaluate the performance of the preference model. Finally, comparative experiments on FJSP benchmark instances of varying sizes demonstrate that PBMP outperforms traditional scheduling strategies such as dispatching rules, OR-Tools, and other deep reinforcement learning (DRL) algorithms, achieving superior scheduling policies and faster convergence. Even with increasing instance sizes, preference learning proves to be an effective reward mechanism in reinforcement learning for FJSP. The ablation study further highlights the advantages of each key component in the PBMP algorithm across performance metrics.

The high cost of space mission operations has motivated several space agencies to prioritize the development of autonomous spacecraft command and control technologies. Deep reinforcement learning (DRL) techniques present one promising domain for the creation of autonomous agents for complex, multifaceted operations problems. This work examines the feasibility of adapting DRL-driven policy generation algorithms to problems in spacecraft decision-making, including strategies for framing spacecraft decision-making problems such as Markov decision processes, avenues for dimensionality reduction, and simplification using expert domain knowledge, sensitivity to hyperparameters, and robustness in the face of mismodeled environmental dynamics. In addition, consideration is given to ensuring the safety of these approaches by hybridizing them with correct-by-construction control techniques in a novel adaptation of shielded deep reinforcement learning. These strategies are demonstrated against a prototypical low-fidelity stationkeeping scenario and a high-fidelity attitude mode management scenario involving flight heritage attitude control and momentum management algorithms. DRL techniques are found to compare favorably to other black-box optimization tools or heuristic solutions for these problems and to require similar network sizes and training durations as widely used testing datasets in the deep learning community.

Solving the flexible job shop scheduling problem with sequence-dependent setup times

Production scheduling is a critical requirement in today’s production systems. The production scheduling problem focuses on the rational allocation of production resources, determining the processing order of production tasks and operations, satisfying production constraints arising from the production environment such as the number of machines and task duration, and achieving optimal production goals, such as completing all tasks in the fastest way possible. The Flexible Job Shop Scheduling (FJSP) problem is a classical production scheduling issue, an extension of the Job Shop Scheduling (JSP) problem. The JSP problem involves allocating a set of jobs to a set of machines, where each job consists of multiple operations that must adhere to a precedence order. Each machine can process only one operation at a time for each job, and the time required for each operation on the corresponding machine is predetermined. The challenge in the JSP problem is to determine an appropriate order for processing operations or assignments, based on the production objective. FJSP represents a more complex and flexible variant of this problem, where at least one operation of a task can be performed on a set of alternative machines. Given that FJSP problems are NP-hard, solutions tend to be complex, varied, and often have limited applicability. Many FJSP scenarios require targeted modeling. In this paper, I propose a modeling approach for the static scheduling problem of job shops, considering low flexibility and demonstrating its usability by comparing it with other scheduling approaches, such as those based on genetic algorithm modeling. The paper is organized as follows:The first part of this paper offers a brief introduction to the FJSP problem, including background, research motivation, and main contributions, the second part provides a literature review,including a brief review and introduction of recent solutions to the FJSP problem, the third part describes a specific modeling approach, the fourth part details the test experiments and results of the modeling approach and includes comparisons with some other approaches, the fifth part analyzes the results from the fourth part and discusses the advantages and disadvantages of the modeling approach relative to others, and the sixth part concludes with a summary of findings and recommendations.

A linguistic-based meta-heuristic modeling and solution approach for solving the flexible job shop scheduling problem (FJSSP) is presented in this study. FJSSP is an extension of the classical job-shop scheduling problem. The problem definition is to assign each operation to a machine out of a set of capable machines (the routing problem) and to order the operations on the machines (the sequencing problem), such that predefined performance measures are optimized. In this research, the scope of the problem is widened by taking into account the alternative process plans for each part (process plan selection problem). Probabilistic selection of alternative process plans and machines are also considered. The FJSSP is presented as a grammar and the productions in the grammar are defined as controls (Baykasoglu, 2002). Using these controls and Giffler and Thompson's (1960) priority rule-based heuristic along with the multiple objective tabu search algorithm of Baykasoglu et al. (1999) FJSSP is solved. This novel approach simplifies the modeling process of the FJSSP and enables usage of existing job shop scheduling algorithms for its fast solution. Instead of scheduling job shops with inflexible algorithms that cannot take into account the flexibility which is available in the job shop, the present algorithm is developed which can take into account the flexibility during scheduling. Such an approach will considerably increase the responsiveness of the job shops.

Reinforcement and deep reinforcement learning for wireless Internet of Things: A survey

Flexible Job Shop Scheduling (FJSS) problem has many real-world applications such as manufacturing and cloud computing, and thus is an important area of study. In real world, the environment is often dynamic, and unpredicted job orders can arrive in real time. Dynamic FJSS consists of challenges of both dynamic optimisation and the FJSS problem. In Dynamic FJSS, two kinds of decisions (so-called routing and sequencing decisions) are to be made in real time. Dispatching rules have been demonstrated to be effective for dynamic scheduling due to their low computational complexity and ability to make real-time decisions. However, it is time consuming and strenuous to design effective dispatching rules manually due to the complex interactions between job shop attributes. Genetic Programming Hyper-heuristic (GPHH) has shown success in automatically designing dispatching rules which are much better than the manually designed ones. Previous works only focused on standard job shop scheduling with only the sequencing decisions. For FJSS, the routing rule is set arbitrarily by intuition. In this paper, we explore the possibility of evolving both routing and sequencing rules together and propose a new GPHH algorithm with Cooperative Co-evolution. Our results show that co-evolving the two rules together can lead to much more promising results than evolving the sequencing rule only.

Future wireless communication networks tend to be intelligentized to accomplish the missions that cannot be preprogrammed. In the new intelligent communication systems, optimizing the network performance has become a challenge due to the ever-increasing complexity of the network environment. New theories and technologies for intelligent wireless communications have obtained widespread attention, among which deep reinforcement learning (DRL) is an excellent machine learning technology. DRL has great potential in enhancing the intelligence of wireless communication systems while overcoming the above challenge. This paper presents a review on applications of DRL in intelligent wireless communications with focuses on millimeter wave (mmWave), intelligent caching and unmanned aerial vehicle (UAV) scenarios. We first introduce the concepts and basic principles of single/multi-agent DRL techniques. Then, we review the related works where DRL algorithms are used to address emerging issues in wireless communications. These issues include mmWave communication, intelligent caching, UAV aided communication, and handover/access control in HetNets. Finally, critical challenges and future research directions of applying DRL in intelligent wireless communications are outlined.

This paper presents an optimization via simulation approach to solve dynamic flexible job shop scheduling problems. In most real-life problems, certain operation of a part can be processed on more than one machine, which makes the considered system (i.e., job shops) flexible. On one hand, flexibility provides alternative part routings which most of the time relaxes shop floor operations. On the other hand, increased flexibility makes operation machine pairing decisions (i.e., the most suitable part routing) much more complex. This study deals with both determining the best process plan for each part and then finding the best machine for each operation in a dynamic flexible job shop scheduling environment. In this respect, a genetic algorithm approach is adapted to determine best part processing plan for each part and then select appropriate machines for each operation of each part according to the determined part processing plan. Genetic algorithm solves the optimization phase of solution methodology. Then, these machine-operation pairings are utilized by discrete-event system simulation model to estimate their performances. These two phases of the study follow each other iteratively. The goal of methodology is to find the solution that minimizes total of average flowtimes for all parts. The results reveal that optimization via simulation approach is a good way to cope with dynamic flexible job shop scheduling problems, which usually takes NP-Hard form.

Mixed integer goal programming models for the flexible job-shop scheduling problems with separable and non-separable sequence dependent setup times

A shuffled cellular evolutionary grey wolf optimizer for flexible job shop scheduling problem with tree-structure job precedence constraints

An efficient evolutionary grey wolf optimizer for multi-objective flexible job shop scheduling problem with hierarchical job precedence constraints