Dynamic Scheduling of Cybersecurity Analysts for Minimizing Risk Using Reinforcement Learning

In this paper, we propose READYS, a reinforcement learning algorithm for the dynamic scheduling of computations modeled as a Directed Acyclic Graph (DAGs). Our goal is to develop a scheduling algorithm in which allocation and scheduling decisions are made at runtime, based on the state of the system, as performed in runtime systems such as StarPU or ParSEC. Reinforcement Learning is a natural candidate to achieve this task, since its general principle is to build step by step a strategy that, given the state of the system (the state of the resources and a view of the ready tasks and their successors in our case), makes a decision to optimize a global criterion. Moreover, the use of Reinforcement Learning is natural in a context where the duration of tasks (and communications) is stochastic. We propose READYS that combines Graph Convolutional Networks (GCN) with an Actor-Critic Algorithm (A2C): it builds an adaptive representation of the scheduling problem on the fly and learns a scheduling strategy, aiming at minimizing the makespan. A crucial point is that READYS builds a general scheduling strategy which is neither limited to only one specific application or task graph nor one particular problem size, and that can be used to schedule any DAG. We focus on different types of task graphs originating from linear algebra factorization kernels (CHOLESKY, LU, QR) and we consider heterogeneous platforms made of a few CPUs and GPUs. We first propose to analyze the performance of READYS when learning is performed on a given (platform, kernel, problem size) combination. Using simulations, we show that the scheduling agent obtains performances very similar or even superior to algorithms from the literature, and that it is especially powerful when the scheduling environment contains a lot of uncertainty. We additionally demonstrate that our agent exhibits very promising generalization capabilities. To the best of our knowledge, this is the first paper which shows that reinforcement learning can really be used for dynamic DAG scheduling on heterogeneous resources.

Unlike steady-state optimization and linear model predictive control, dynamic optimization and nonlinear model predictive control is underway to industry application. This paper discusses the implementation issues of two-level control strategy integrated dynamic real-time optimization and nonlinear model predictive control of a heat-integrated distillation column system. To realize this scheme, four nonlinear programming problems: economic operation optimization, data reconciliation and state estimation, parameter estimation and nonlinear model predictive control, are considered based on system rigorous dynamic model. The economic operation optimization is to minimize the cost of the plant. The unmeasured states are estimated by measurements of temperatures, pressures and flow rates. Time-variant parameters such as tray efficiencies and feed composition are updated on-line. The sequential quadratic programming and orthogonal collocation are used to solve the dynamic nonlinear optimization problems.

For one intelligent processing system which inclusionRGV, how to effectively use various resources to rationally and dynamically perform dynamic scheduling to improve production efficiency is the key. This paper studies only the material processing operations of a single process. According to the processing process of a given material, we need to focus on analyzing its dynamic scheduling strategy. In a material processing system with an established 8-hour working time, maximizing the amount of material processing is the primary goal. However, the increased amount of material processing is obtained by continuously completing the accumulation of work tasks. Therefore, the core is to convert the material processing quantity maximization model in the system into a task selection planning model based on time loss minimization, and seek each with a greedy algorithm. A task selects the optimal solution locally, and approximates each local optimal combination as a global optimal.

Approximation Algorithms for Dynamic Assortment Optimization Models

<p>Changes in the marketing environment and consumer behavior are the driving force for the development of online marketing. Although traditional marketing communication still exists, it has been unable to adapt to the marketing needs of modern cultural brands. On this basis, this paper combines the cloud service platform to design an interactive cultural brand marketing system. In view of the problems of improper task scheduling and resource waste in cloud platform resource scheduling in actual situations, a dynamic resource scheduling optimization model under the cloud platform environment is established, and fuzzy evaluation rules are designed. Moreover, through problem analysis, based on the reinforcement learning algorithm, this paper proposes a deep reinforcement learning resource scheduling algorithm based on tabu search, and combines the algorithm to design the functional module of the marketing system. On this basis, this paper designs an experiment to verify the performance of this interactive cultural brand marketing system. The research results prove that the marketing system constructed in this paper has certain reliability.</p> <p> </p>

Energy minimization in dynamic train scheduling and control for metro rail operations

With the increasing complexity of enterprise systems and the rise in cyber threats, managing security risks while optimizing resources has become a significant challenge. Traditional models often address security and resource management in isolation, making it difficult to adapt to evolving threats and dynamic workloads. This paper proposes the deep learning-based dynamic security assessment and optimization model, which integrates dynamic security assessment, anomaly detection, multi-modal data fusion, security investment optimization, and cloud resource optimization into a unified framework. By leveraging deep learning techniques such as convolutional neural networks for feature extraction and recurrent neural networks for temporal anomaly detection, alongside reinforcement learning for resource optimization, the deep learning-based dynamic security assessment and optimization model provides real-time risk evaluation and adapts resource allocation based on system needs.

Intelligent manufacturing plays a significant role in Industry 4.0. Dynamic shop scheduling is a key problem and hot research topic in the intelligent manufacturing systems, which is NP-hard. However, traditional shop scheduling mode, dynamic event prediction approach, scheduling model and scheduling algorithm, cannot cope with increasingly complicated problems under kinds of scales production disruptions in the real-world production. To deal with these problems, this paper proposes a new joint data-model driven dynamic scheduling architecture for intelligent workshop. The architecture includes four new and key characteristics in the aspects of scheduling mode, dynamic event prediction, scheduling model and algorithm. More specifically, the new scheduling mode introduces data analytics methods to quickly and accurately deal with the dynamic events encountered in the production process. The new prediction model improves the deep learning method, and further applies it predict the dynamic events accurately to provide reliable input to the dynamic scheduling. The new scheduling model proposes a new hybrid rescheduling and inverse scheduling model, which can cope with almost scales of abnormal production problems. The new scheduling algorithm hybridizes dynamic programming and intelligent optimization algorithm, which can overcome the disadvantages of the two methods based on the analysis of solution space. The dynamic programming is employed to provide high-quality initial solutions for the intelligent optimization algorithm by reducing the computation time greatly. To sum up, the presented architecture is a new attempt to understand the problem domain knowledge and broaden the solving idea, which can also provide new theories and technologies to manufacturing system optimization and promote the applications of the theoretical results.

Attempts to address the production scheduling problem thus far rely on simplifying assumptions, such as static environment and inflexible size of the problem, which compromises the schedule performance in practice due to many unpredictable disruptions to the system. Thus, the study of scheduling in the presence of real-time events, termed dynamic scheduling, continues to attract attention given the agility, flexibility, and timeliness modern production systems must deliver. Additionally, the changing nature of the manufacturing system also raises new challenges to existing scheduling strategies. At the front-end, the development of advanced data creation and exchange frameworks such as the Internet of things and cyber-physical system and their applications to the industrial environment have created an abundance of industrial data, while at the backend, edge and cloud computing technologies greatly enhance the capacity to process that data. Industrial data must be mined and analyzed so that the investment in infrastructure is not wasted, and the production system managed more effectively and in real-time. Many data-driven technologies have been adopted in scheduling research, a promising candidate among them being reinforcement learning (RL) which is able to build a direct mapping from observation of environment to actions that improve its performance. In this thesis, a deep multi-agent reinforcement learning (deep MARL) architecture is proposed to solve the dynamic scheduling problem (DSP). The deep reinforcement learning (DRL) algorithm is used to train the decentralized scheduling agents, to capture the relationship between information on the factory floor and scheduling objectives, with the aim of making real-time decisions for a manufacturing system with frequent unexpected events. Two major aspects of deep MARL application to DSP are addressed in this work, namely the conversion from traditional static scheduling problem (SSP) to dynamic scheduling in a practical context, and the adaptation of existing deep MARL algorithms to solve the scheduling problem in such an environment. Some impractical constraints of traditional studies are removed to create a research context that is closer to actual practice, result in a scheduling problem of variable size and scope. Specialized state and action representations that can handle the ever-changing specification of problem are developed; the criteria of feature selection in dynamic environment are also discussed. Recent progressions in DRL and MARL research are integrated into the proposed approach after selection and adaptation. In addition, various improvements to common deep MARL architecture are proposed, including the lightweight multilayer perceptron (MLP) encoder that is efficient in handling unstructured industrial data, a training scheme under the multi-agent architecture to improve the stability of training and overall performance, and knowledge-based reward-shaping techniques to decompose the joint reward signal into individual utilities to speed up the learning and encourage cooperative behavior between agents. Simulation studies are then conducted for the ablation study and validation. In the first stage, the performance of the proposed approach, either as individual components or as an integrated model, are tested in iterative simulation runs within which a unique instance of production is created. Meanwhile, a set of DRL-based approaches from recent publications are run in parallel. Results suggest that the contribution of each improvement is significant; the integrated architecture also delivers stronger performance than peer DRL-based approaches. For the validation, a set of priority rules that have strong performance in specified context and are widely applied in actual production scheduling are used as the benchmark. Proposed approach also provides performance gain compared to the strongest rule, with a minor increase in computation cost and negligible latency in decision-making.

Aiming at the field of high-demand takeaway delivery, a dynamic scheduling strategy based on multi-rider collaborative delivery status is proposed, and the model implementation is obtained by constructing a collaborative dynamic scheduling optimization model and using the optimization TS algorithm to obtain a multi-rider scheduling scheme in a dynamic environment, so that the total delivery time is minimized. Finally, the experimental results show the effectiveness and feasibility of the model and algorithm implementation.

The objective of this article is to study the order allocation problems in a stochastic supply chain. This supply chain consists of a company that can order raw materials from multiple suppliers. At any time of a supply order, decisions have to be made by the company concerning the respective supplier order quantities and price breaks, so that the total purchasing cost and disruption risk cost are minimized, while maintaining a specified service level and quality. For this purpose, we have developed an integrated multi-objective decision model under random constraint to supply order allocation.

PurposeBasic project control through traditional methods is not sufficient to manage the majority of real-time events in most construction projects. The purpose of this paper is to propose a Dynamic Scheduling (DS) model that utilizes multi-objective optimization of cost, time, resources and cash flow, throughout project construction.Design/methodology/approachUpon reviewing the topic of DS, a worldwide internet survey with 364 respondents was conducted to define end-user requirements. The model was formulated and solution algorithms discussed. Verification was reported using predefined problem sets and a real-life case. Validation was performed via feedback from industry experts.FindingsThe need for multi-objective dynamic software optimization of construction schedules and the ability to choose among a set of optimal alternatives were highlighted. Model verification through well-known test cases and a real-life project case study showed that the model successfully achieved the required dynamic functionality whether under the small solved example or under the complex case study. The model was validated for practicality, optimization of various DS schedule quality gates, ease of use and software integration with contemporary project management practices.Practical implicationsOptimized real-time scheduling can provide better resources management including labor utilization and cost efficiency. Furthermore, DS contributes to optimum materials procurement, thus minimizing waste.Social implicationsOptimized real-time scheduling can provide better resources management including labor utilization and cost efficiency. Furthermore, DS contributes to optimum materials procurement, thus minimizing waste.Originality/valueThe paper illustrates the importance of DS in construction, identifies the user needs and overviews the development, verification and validation of a model that supports the generation of high-quality schedules beneficial to large-scale projects.

The nonlinear model predictive control method is applied to design emergency voltage controller. Based on quasi-steady-state approximation, the receding dynamic optimization model is established which subjected to differential-algebraic equations with continuous-discrete time variables. To enhance computational efficiency and control precision of solving this dynamic optimization model, direct dynamic optimization approach combined with moving finite elements are proposed to solve the problem of dynamic optimization. Firstly, the research interval is divided into finite elements. Radau collocation method is used to convert the optimization model into a nonlinear programming by approximating state variable, algebraic variable and control variable profiles by a family of polynomials on each element. Then moving finite elements is introduced to adjust the length of each interval dynamically, achieve the precise positioning of control variable and improve the accuracy of the algorithm. The interior-point algorithm given by AMPL as a mathematical optimization solver is used to solve the problem. Simulation results on New England 10-machine 39-bus system verify effectiveness of the proposed method.

Comparison of dynamic scheduling policies for hybrid make-to-order and make-to-stock production systems with stochastic demand

Integration of set point optimization techniques into nonlinear MPC for improving the operation of WWTPs