Evolutionary programming with ensemble of explicit memories for dynamic optimization

Many applications can be formalized as dynamic problems and their main goals are to track the moving optimum. Many algorithms that address efficiently static problems, fail when applied to dynamic problems due to their inability to adapt and respond to changes in the environment. A different topology multi-swarm PSO is proposed to deal with problems in dynamic environment. The proposed algorithm integrates two different topology subswarms and the two subswarms exchange their best particle at checkpoints. One rapidly convergence to the optima and the other is responsible for local search and diversity reservation. The experiments demonstrate the proposed algorithm is effective and stable in dynamic environment.

Haven’t there been enough approaches to scheduling problems? In terms of variety the answer might be ‘yes’. However, the answer is not as straightforward in terms of effectiveness. In many scheduling problems, it is highly improbable, if not impossible, to obtain optimal schedules within a reasonable amount of time in spite of adopting a wide range of approaches, including evolutionary computation (EC), artificial neural networks (ANN), fuzzy systems (FS), simulated annealing (SA) and Tabu search (TS). In recent years attention has been drawn to another biologically-inspired computing paradigm called artificial immune systems (AIS). An AIS abstracts and models the principles and processes of the biological immune system in order to effectively tackle challenging problems in dynamic environments. Major AIS models include negative selection, clonal selection, immune networks and more recently danger theory (Garrett in Evol. Comput. 13(2):145–177, 2005). There are some similarities between these principles and processes in the immune system and those found in other nature-inspired computing approaches, EC and ANN in particular. However, there are also substantial differences. In particular, adaptive cloning and mutation processes make AIS distinctive and useful.KeywordsSchedule ProblemAdaptive Immune SystemClonal SelectionArtificial Immune SystemTask GraphThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The increasing volume of traffic has led to severe challenges, including traffic congestion, heightened energy consumption, increased air pollution, and prolonged travel times. Addressing these issues requires innovative approaches for optimizing road network utilization. While Deep Reinforcement Learning (DRL)-based methods have shown remarkable effectiveness in dynamic scenarios like traffic management, their primary focus has been on single-agent setups, limiting their applicability to real-world multi-agent systems. Managing agents and fostering collaboration in a multi-agent reinforcement learning scenario remains a challenging task. This paper introduces a cooperative multi-agent federated reinforcement learning algorithm named FLDQN to address the challenge of agent cooperation by solving travel time minimization challenges in dynamic multi-agent reinforcement learning (MARL) scenarios. FLDQN leverages federated learning to facilitate collaboration and knowledge sharing among intelligent agents, optimizing vehicle routing and reducing congestion in dynamic traffic environments. Using the SUMO simulator, multiple agents equipped with deep Q-learning models interact with their local environments, share model updates via a federated server, and collectively enhance their policies using unique local observations while benefiting from the collective experiences of other agents. Experimental evaluations demonstrate that FLDQN achieves a significant average reduction of over 34.6% in travel time compared to non-cooperative methods while simultaneously lowering the computational overhead through distributed learning. FLDQN underscores the vital impact of agent cooperation and provides an innovative solution for enabling agent cooperation in a multi-agent environment.

Recent models of spiking neuronal networks have been trained to perform behaviors in static environments using a variety of learning rules, with varying degrees of biological realism. Most of these models have not been tested in dynamic visual environments where models must make predictions on future states and adjust their behavior accordingly. The models using these learning rules are often treated as black boxes, with little analysis on circuit architectures and learning mechanisms supporting optimal performance. Here we developed visual/motor spiking neuronal network models and trained them to play a virtual racket-ball game using several reinforcement learning algorithms inspired by the dopaminergic reward system. We systematically investigated how different architectures and circuit-motifs (feed-forward, recurrent, feedback) contributed to learning and performance. We also developed a new biologically-inspired learning rule that significantly enhanced performance, while reducing training time. Our models included visual areas encoding game inputs and relaying the information to motor areas, which used this information to learn to move the racket to hit the ball. Neurons in the early visual area relayed information encoding object location and motion direction across the network. Neuronal association areas encoded spatial relationships between objects in the visual scene. Motor populations received inputs from visual and association areas representing the dorsal pathway. Two populations of motor neurons generated commands to move the racket up or down. Model-generated actions updated the environment and triggered reward or punishment signals that adjusted synaptic weights so that the models could learn which actions led to reward. Here we demonstrate that our biologically-plausible learning rules were effective in training spiking neuronal network models to solve problems in dynamic environments. We used our models to dissect the circuit architectures and learning rules most effective for learning. Our model shows that learning mechanisms involving different neural circuits produce similar performance in sensory-motor tasks. In biological networks, all learning mechanisms may complement one another, accelerating the learning capabilities of animals. Furthermore, this also highlights the resilience and redundancy in biological systems.

This paper presents an architecture that makes it possible to construct dynamic systems capable of growing in dimension and adapting its knowledge to environmental changes. An architecture must define the components of the system (agents in this case), as well as the way in which those components communicate and interact with each other in order to achieve the system’s goals. The work presented here focuses on the development of an agent-based architecture, based on the use of deliberative agents, that incorporate case based reasoning. The proposed architecture requires an analysis and design methodology that facilitates the building of distributed systems using this technology. The proposal combines elements of existing methodologies such as Gaia and AUML in order to take advantage of their characteristics. Moreover the architecture takes into account the possibility of modelling problems in dynamic environments and therefore the use of autonomous models that evolve over time. To solve this problem the architecture incorporates CBR-agents whose aim is to acquire knowledge and adapt themselves to environmental changes. The architecture has been applied to model for evaluating the interaction between the atmosphere and the ocean, as well as for the planification and optimization of sea routes for vessels. The system has been tested successfully, and the results obtained are presented in this paper.KeywordsMultiagent SystemActivity DiagramUser AgentAcquaintance ModelVessel AgentThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Weather-impacted asset routing is a complex problem, involving nonlinear, non-convex costs due to spatiotemporally varying uncertainty of weather parameters over a finite time horizon. To this end, we have augmented a previously published tool for multiobjective planning and asset routing, termed TMPLAR, with a new approximate dynamic programming-based Pareto optimization (NAPO) algorithm. TMPLAR is a mixed-initiative tool to solve the asset routing problem in dynamic and uncertain environments. It is built upon multi-objective dynamic programming algorithms to route assets in a timely fashion, while considering objectives, such as fuel efficiency, voyage time, distance, and adherence to real world constraints (asset vehicle limits, navigator-specified deadlines, etc.). The asset routing problem is exacerbated by the need to address multiple conflicting objectives, spatial and temporal uncertainty associated with the weather and multiple constraints on asset operation. The NAPO algorithm optimizes weather-based objectives in a reasonable amount of time, optimizing arrival and departure times at waypoints, asset speed and bearing. The key algorithmic contribution is a fast approximate method for substantially containing the computational complexity by generating the Pareto-front of the multi-objective shortest path problem for networks with stochastic non-convex edge costs, utilizing approximate dynamic programming and clustering techniques. The proposed algorithm is validated and we compare its performance with the new approach to multi-objective A* (NAMOA*).

PurposeThe purpose of this paper is to improve the traditional bug algorithms for the navigation of mobile robots in unknown environments by considering the limitations in previous works such as generating long path, limited to static environments as well as ignoring implementation issues. With this purpose, a new bug‐type algorithm termed Distance Histogram Bug (DH‐Bug) is proposed for overcoming these limitations.Design/methodology/approachDH‐Bug redefines the traditional motion modes and switching criteria to shorten the path length. In order to extend the framework of the traditional bug algorithms to tackle the navigation problem in dynamic environments, a new mode and the related switching conditions are designed for dealing with moving obstacles. Moreover, a realization method termed the Distance Histogram (DH) method which takes many implementation issues into full account is proposed for implementing DH‐Bug on real robots.FindingsDH‐Bug is convergent in static environments and it also guarantees “approximate” convergence based on several reasonable assumptions when there are moving obstacles. Simulations results show that DH‐Bug generates shorter average path length than some classical Bug algorithms in static environments and it also performs well in most simulations that contain moving obstacles except for some extremely adverse scenarios which have been discussed in the paper. Experiments on real robots further verify its applicability in both static environments and dynamic environments containing moving obstacles.Originality/valueCompared with previous works, DH‐Bug has three main contributions. First, in static environments, it can shorten the average path length than many classical bug algorithms in the premise of guaranteeing convergence. Second, it can be applied in dynamic environments containing moving obstacles. Third, unlike the previous bug algorithms that always ignore the practical implementation issues, DH‐Bug presents not only an abstract concept, but also a realization approach for realizing this concept on real robots.

Visual localization has been well studied in recent decades and applied in many fields as a fundamental capability in robotics. However, the success of the state of the arts usually builds on the assumption that the environment is static. In dynamic scenarios where moving objects are present, the performance of the existing visual localization systems degrades a lot due to the disturbance of the dynamic factors. To address this problem, we propose a novel sparse motion removal (SMR) model that detects the dynamic and static regions for an input frame based on a Bayesian framework. The similarity between the consecutive frames and the difference between the current frame and the reference frame are both considered to reduce the detection uncertainty. After the detection process is finished, the dynamic regions are eliminated while the static ones are fed into a feature-based visual simultaneous localization and mapping (SLAM) system for further visual localization. To verify the proposed method, both qualitative and quantitative experiments are performed and the experimental results have demonstrated that the proposed model can significantly improve the accuracy and robustness for visual localization in dynamic environments.<;/p><;p><;i>Note to Practitioners<;/i>-This article was motivated by the visual localization problem in dynamic environments. Visual localization is well applied in many robotic fields such as path planning and exploration as the basic capability for a mobile robot. In the GPS-denied environments, one robot needs to localize itself through perceiving the unknown environment based on a visual sensor. In real-world scenes, the existence of the moving objects will significantly degrade the localization accuracy, which makes the robot implementation unreliable. In this article, an SMR model is designed to handle this problem. Once receiving a frame, the proposed model divides it into dynamic and static regions through a Bayesian framework. The dynamic regions are eliminated, while the static ones are maintained and fed into a feature-based visual SLAM system for further visual localization. The proposed method greatly improves the localization accuracy in dynamic environments and guarantees the robustness for robotic implementation.

: This research effort focused on three core research challenges: (1) How to explain the gap between formal analysis and practical performance for combinatorial search; (2) How to characterize and capture hidden tractable structure in real-world problems; and, (3) How to further boost combinatorial search methods for real-world problems. A series of advanced formal models for predicting the runtime of combinatorial search methods were developed. Models of runtime distributions of search methods capturing exponential and power law (heavy-tailed) regimes for both complete and incomplete randomized search methods were introduced, together with a generative model that generates search trees with any pre-defined degree of heavy-tailedness. New methods for the efficient computation of the number solution clusters and their marginal distributions were developed. The notion of backdoor sets, a measure that characterizes hidden problem structure was extended to encompass combinatorial optimization problems as well as learning during search, thereby providing novel insights into the connection between the hidden structure of optimization problems and the surprising efficiency of today s optimization engines. A novel Markov Chain Monte Carlo sampling strategy, inspired by a flat histogram method from statistical physics, was developed to compute the density of states of a Boolean formula. Multi-agent inference problems in dynamic environments were formulated into the framework of message passing algorithms and graphical models, generalizing the standard Kalman filter to the distributed case. A new hybrid strategy for the MaxSAT problem was also proposed, combining the complementary strength of local search and systematic search, bringing the best of both worlds in a way that is ideal for current multi-core architectures.

Time-reliability optimization for the stochastic traveling salesman problem

Purpose – The combined heat and power dispatch (CHPD) aims to optimize the outputs of online units in a power plant consisting thermal generators, co-generators and heat-only units. Identifying the operating point of a co-generator within its feasible operating region (FOR) is difficult. This paper aims to solve the CHPD problem in static and dynamic environments. Design/methodology/approach – The CHPD plant operation is formulated as an optimization problem under static and dynamic load conditions with the objectives of minimizations of cost and emissions subject to various system and operational constraints. A novel bio-inspired search technique, grey wolf optimization (GWO) algorithm is used as an optimization tool. Findings – The GWO-based algorithm has been developed to determine the preeminent power and heat dispatch of operating units within the FOR region. The proposed methodology provides fuel cost savings and lesser pollutant emissions than those in earlier reports. Particularly, the GWO always keeps the co-generator’s operating point within the FOR, whereas most of the existing methods fail. Originality/value – The GWO is applied for the first time to solve the CHPD problems. New dispatch schedules are reported for 7-unit system with the objectives of total fuel cost and emission minimizations, 24-unit system for economic operation and 11-unit system in dynamic environment. The simulation experiments reveal that GWO converges quickly, consistent and the statistical performance clears its applicability to CHPD problems.

Composite particle optimization with hyper-reflection scheme in dynamic environments

An optical detection system is built by using model building method based on the modulation transfer function. The effects of space environment and noise model have been considered and added in this system. In this model, we solve the establishment of the complex imaging model problem in dynamic running environment. Besides, this system can meet the demand of different space target detection task and the performance of the different stages of life span parallel analysis and evaluation system. The numerical simulations are performed to check the validity and capability of the proposed optical detection system. KEYWORD: Space target; Dynamic environment; Optical detection system

To address the resource allocation problem in dynamic environments where multiple unmanned aerial vehicle base stations (UAV-BSs) provide efficient downlink services to ground users, this paper proposes a novel hierarchical decision-making mechanism based on the Proximal Policy Optimization (PPO) algorithm. The proposed method optimizes time-frequency resource allocation in the downlink, aiming to maximize the total user throughput over multiple time slots. By constructing channel and interference models, the complex multi-channel resource allocation problem is decomposed into a series of single-channel decision subproblems, significantly reducing the action space complexity. Specifically, the original exponential complexity O(NM) (where N is the number of users and M is the number of channels) is reduced to a linear complexity O(N), effectively alleviating the curse of dimensionality. Simulation results demonstrate that the proposed hierarchical architecture, integrated with the PPO algorithm, achieves superior performance in terms of total throughput, convergence speed, and stability compared to existing methods. This study provides new insights and technical support for efficient resource management in UAV-BS systems operating in complex and dynamic environments.

This study proposes a distributed formation control framework for multi-agent systems by integrating deep reinforcement learning (DRL) with interdisciplinary methodologies from control theory, robotics, and artificial intelligence. Aimed at addressing collaborative formation challenges in dynamic environments, the approach uses graph-theoretic modeling for formation-keeping and obstacle avoidance and dynamic Attention-based Advantage Actor-Critic (DA-A2C) algorithm to solve collaborative formation problems in dynamic environments like autonomous drone swarms and robotic fleets. To balance formation consistency and obstacle repulsion, this strategy optimizes neighbor perception dynamic weight allocation using the dynamic Ω-zone mechanism. Experimental validations in simulated environments with static/moving obstacles show superior performance over SA2C, MATD3, and DDPG baselines, covering formation control and obstacle avoidance in complex situations and outperforming other algorithms in adaptability and robustness. This research can improve cross-domain applications in intelligent transportation, industrial automation, and emergency response systems. The method works in multi-robot formation systems, and future study will focus on 3D environment generalization and efficient communication protocols.