An object transportation system with multiple robots and machine learning

This paper investigates the development of a physical multi-robot system, where a group of intelligent robots work cooperatively to transport an object to a goal location and orientation in an unknown dynamic environment. Multi-agent technology and machine learning are integrated into the same physical platform to provide innovative capabilities for carrying out the task. First, a new multi-agent architecture is developed. Second, two methods that facilitate optimized machine learning, Reinforcement learning (RL) and genetic algorithms (GA), are integrated into the decision-making agent in the architecture. An arbitrator is incorporated into a probabilistic switching scheme for selecting the optimal strategy in a given state of the task. Finally, the robot force/motion control and local modeling are integrated into the architecture to implement a physical multi-robot system. Simulation and experimental studies are carried out to demonstrate the feasibility of the system.

Gradual spatial pattern formation of homogeneous robot group

This paper describes an extended Genetic Algorithm Approach for path planning of multiple mobile robots with obstacle detection and avoidance in static and dynamic scenarios. Through the software Netlogo, used in simulations of multi-agent applications, a model was developed for the given problem. The model, which contains multiple robots and a scenario with several dynamic and static obstacles, is responsible for determining the best path used by the robots to achieve the goal state in a shorter number of steps and avoiding collisions. Additionally, a performance evaluation of this model in comparison with A* algorithm is presented.

Wireless cellular networks have many parameters that are normally tuned upon deployment and re-tuned as the network changes. Many operational parameters affect reference signal received power (RSRP), reference signal received quality (RSRQ), signal-to-interference-plus-noise-ratio (SINR), and, ultimately, throughput. In this paper, we develop and compare two approaches for maximizing coverage and minimizing interference by jointly optimizing the transmit power and downtilt (elevation tilt) settings across sectors. To evaluate different parameter configurations offline, we construct a realistic simulation model that captures geographic correlations. Using this model, we evaluate two optimization methods: deep deterministic policy gradient (DDPG), a reinforcement learning (RL) algorithm, and multi-objective Bayesian optimization (BO). Our simulations show that both approaches significantly outperform random search and converge to comparable Pareto frontiers, but that BO converges with two orders of magnitude fewer evaluations than DDPG. Our results suggest that data-driven techniques can effectively self-optimize coverage and capacity in cellular networks.

Cloud computing provides on-demand access to a shared puddle of computing resources, containing applications, storage, services, and servers above the internet. This allows organizations to scale their IT infrastructure up or down as needed, reduce costs, and improve efficiency and flexibility. Improving professional guidelines for social media interactions is crucial to address the wide range of complex issues that arise in today’s digital age. It is imperative to enhance and update professional guidelines regarding social media interactions in order to effectively tackle the multitude of intricate issues that emerge. In this paper, we propose a reinforcement learning (RL) method for handling dynamic resource allocation (DRA) and load balancing (LB) activity in a cloud environment and achieve good scalability and a significant improvement in performance. To address this matter, we propose a dynamic load balancing technique based on Q-learning, a reinforcement learning algorithm. Our technique leverages Q-learning to acquire an optimal policy for resource allocation in real-time based on existing workload, resource accessibility, and user preferences. We introduce a reward function that takes into account performance metrics such as response time and resource consumption, as well as cost considerations. We evaluate our technique through simulations and show that it outperforms traditional load balancing techniques in expressions of response time and resource utilization while also reducing overall costs. The proposed model has been compared with previous work, and the consequences show the significance of the proposed work. Our model secures a 20% improvement in scalability services. The DCL algorithm offers significant advantages over genetic and min-max algorithms in terms of training time and effectiveness. Through simulations and analysis on various datasets from the machine learning dataset repository, it has been observed that the proposed DCL algorithm outperforms both genetic and min-max algorithms. The training time can be reduced by 10% to 45%, while effectiveness is enhanced by 30% to 55%. These improvements make the DCL algorithm a promising option for enhancing training time and effectiveness in machine learning applications. Further research can be conducted to investigate the potential of combining the DCL algorithm with a supervised training algorithm, which could potentially further improve its performance and apply in real-world application.

Reinforcement learning, as a branch of machine learning, has been gradually applied in the control field. However, in the practical application of the algorithm, the hyperparametric approach to network settings for deep reinforcement learning still follows the empirical attempts of traditional machine learning (supervised learning and unsupervised learning). This method ignores part of the information generated by agents exploring the environment contained in the updating of the reinforcement learning value function, which will affect the performance of the convergence and cumulative return of reinforcement learning. The reinforcement learning algorithm based on dynamic parameter adjustment is a new method for setting learning rate parameters of deep reinforcement learning. Based on the traditional method of setting parameters for reinforcement learning, this method analyzes the advantages of different learning rates at different stages of reinforcement learning and dynamically adjusts the learning rates in combination with the temporal-difference (TD) error values to achieve the advantages of different learning rates in different stages to improve the rationality of the algorithm in practical application. At the same time, by combining the Robbins–Monro approximation algorithm and deep reinforcement learning algorithm, it is proved that the algorithm of dynamic regulation learning rate can theoretically meet the convergence requirements of the intelligent control algorithm. In the experiment, the effect of this method is analyzed through the continuous control scenario in the standard experimental environment of ”Car-on-The-Hill” of reinforcement learning, and it is verified that the new method can achieve better results than the traditional reinforcement learning in practical application. According to the model characteristics of the deep reinforcement learning, a more suitable setting method for the learning rate of the deep reinforcement learning network proposed. At the same time, the feasibility of the method has been proved both in theory and in the application. Therefore, the method of setting the learning rate parameter is worthy of further development and research.

Chemical heuristics have been fundamental to the advancement of chemistry and materials science. These heuristics are typically established by scientists using knowledge and creativity to extract patterns from limited datasets. Machine learning offers opportunities to perfect this approach using computers and larger datasets. Here, we discuss the relationships between traditional heuristics and machine learning approaches. We show how traditional rules can be challenged by large-scale statistical assessment and how traditional concepts commonly used as features are feeding the machine learning techniques. We stress the waste involved in relearning chemical rules and the challenges in terms of data size requirements for purely data-driven approaches. Our view is that heuristic and machine learning approaches are at their best when they work together.

The supervisory control of robots is a very demanding application. In the context of robots control in assistive environments, it is important that the robot user is able to give commands to robots in a way that is easy and intuitive. There are several tasks that can be achieved using robots under assistive environments. It is challenging to efficiently control multiple robots / robots with degrees of freedom with a simple/intuitive interface by a single operator. In this proposal, we propose the use of Reinforcement Learning for intuitive interface mapping. Based on interaction with the environments, we can determine the optimal interface mapping through the process of Reinforcement Learning. The novelty of this paper is the use of changing reward functions based on qualitative performance evaluation for the Reinforcement Learning algorithm. In this paper, we show that the use of proposed reward functions can result in optimal/intuitive interface mapping for multiple robots / robots with degrees of freedom control applications.

Reinforcement learning (RL) algorithms have been successfully applied to control tasks associated with unmanned aerial vehicles and robotics. In recent years, safe RL has been proposed to allow the safe execution of RL algorithms in industrial and mission-critical systems that operate in closed loops. However, if the system operating conditions change, such as when an unknown fault occurs, typical safe RL algorithms cannot adapt while retaining past knowledge. Continual RL algorithms have been proposed to address this issue. However, the impact of continual adaptation on the system’s safety is an understudied problem. In this paper, we study the intersection of safe and continual RL. First, we empirically demonstrate that a popular continual RL algorithm, elastic weight consolidation, does not satisfy safety constraints in nonlinear systems subject to varying operating conditions. Specifically, we study the MuJoCo HalfCheetah and Ant environments with velocity constraints and sudden joint loss non-stationarity. Then, we show that an agent trained using constrained policy optimization, a safe RL algorithm, experiences catastrophic forgetting in continual learning settings. With this in mind, we explore a simple reward-shaping method to ensure that elastic weight consolidation prioritizes remembering both safety and task performance for safety-constrained, nonlinear, and non-stationary systems.

Data quality is a critical aspect of data analytics since it directly influences the accuracy and effectiveness of insights and predictions generated from data. Artificial Intelligence (AI) schemes have grown in the existing era of technological advancement, which provides innovative exposure to healthcare applications. Reinforcement Learning (RL) is a subfield and an influential Machine Learning (ML) model aimed at optimizing decision-making by association with dynamic environments. In healthcare applications, RL can modify conduct strategies, enhance source application, and improve patient investigation history by using various data modalities. The worth of the data quality regulates how effective RL is in healthcare applications. In healthcare, the model predictions have a direct impact on patient's lives, and poor data quality often leads to wrong evaluations that expose patient safety and treatment quality. Biases in data quality have also presented a challenging influence on the RL model's effectiveness and accuracy. RL models have enormous potential in healthcare; however, various strategic limitations prevent their widespread acceptance and deployment. The implementation of RL in healthcare faces serious issues, mostly around data quality, bias, and tactical difficulties. This study delivers a broad survey of these challenges, emphasizing how imbalanced, imperfect, and biased data can affect the generalizability and performance of RL models. We critically assessed the sources of data bias, comprising demographic imbalances and irregularities in electronic health records (EHRs), and their impact on RL algorithms. This survey aims to present a detailed study of the complex circumstances relating to data quality, data biases, and strategic barriers in RL models deploying in healthcare applications. However, the main contribution of the proposed study is that it provides a systematic review of these challenges and delivers a roadmap for future work intended to refine the consistency, fairness, and scalability of RL in healthcare sectors.

The simple and labor-intensive tasks of workers on the job site are rapidly becoming digital. In the work environment of logistics warehouses and manufacturing plants, moving goods to a designated place is a typical labor-intensive task for workers. These tasks are rapidly undergoing digital transformation by leveraging mobile robots in automated warehouses. In this paper, we studied and tested realistically necessary conditions to operate mobile robots in an automated warehouse. In particular, considering conditions for operating multiple mobile robots in an automated warehouse, we added more complex actions and various routes and proposed a method for improving sparse reward problems when learning paths in a warehouse with reinforcement learning. Multi-Agent Reinforcement Learning (MARL) experiments were conducted with multiple mobile robots in an automated warehouse simulation environment, and it was confirmed that the proposed reward model method makes learning start earlier even there is a sparse reward problem and learning progress was maintained stably. We expect this study to help us understand the actual operation of mobile robots in an automated warehouse further.

Multi-robot coalition formation (MRCF) problem deals with the formation of subsets of robotic to handle a particular task. In such a system, every task is executed by multiple robots. Thus, cooperation and coordination among the robots is very important. One of the key issues to be investigated for smooth operation of a multi-robot systems is finding an optimal task allocation among the suitably formed robot groups (sub sets). Considering the complete execution of available tasks, the problem of assigning available resources (robot features) to the tasks is computationally complex, which may further increase as number of tasks increases. Genetic algorithms (GA) have been found quite efficient in solving such complex computational problems. There are several algorithms based on GA to solve MRCF problems but none of them have considered the dynamic variants. Thus we apply immigrants based GAs viz. RIGA (random immigrants genetic algorithm) and EIGA (elitism based immigrants genetic algorithm) to optimal task allocation in MRCF problem. Comparative performance evaluation has been made with respect to SGA (standard genetic algorithm). Finally, we report a novel use of these algorithms making them adaptive with certain modification in their traditional attributes by adaptively choosing the parameters of genetic operators. We name them as aRIGA (adaptive RIGA) and aEIGA (adaptive EIGA). Simulations experiments have demonstrated that RIGA and EIGA produces better solutions then SGA in both the cases (with fixed and adaptive genetic operators). Among them, EIGA and aEIGA outperforms RIGA and aRIGA respectively.

As AI technology advances, interest in performing multi-robot autonomous missions for manned-unmanned teaming (MUM-T) is increasing. In order to develop autonomous mission performance technology for multiple robots, simulation technology that reflects the characteristics of real robots and can flexibly apply various missions is needed. Additionally, in order to solve complex non-linear tasks, an API must be provided to apply multi-robot reinforcement learning technology, which is currently under active research. In this study, we propose the campaign model to flexibly simulate the missions of multiple robots. We then discuss the results of developing a simulation environment that can be edited and run and provides a reinforcement learning API including acceleration performance. The proposed simulated control module and simulated environment were verified using an enemy infiltration scenario, and parallel processing performance for efficient reinforcement learning was confirmed through experiments.

Background: The central nervous system (CNS) is optimizing arm movements to reduce some kind of cost function. Simulating parts of the nervous system is one way of obtaining accurate information about the neurological and treatment of neuromuscular diseases. The primary purpose of this paper is to model and control the human arm in a reaching movement based on reinforcement learning (RL) theory. Methods: First, Zajac’s muscle model has improved by a fuzzy system. Second, the proposed muscle model applied to the 6 muscles, which are responsible for a two-link arm that moves in the horizontal plane. Third, the model parameters are approximated based on the genetic algorithm (GA). Experimental data recorded from healthy subjects for assessing the approach. At last, the RL algorithm has utilized to guide the arm for reaching tasks. Results: The results show that: (1) The proposed system is temporally similar to a real arm movement. (2) The RL algorithm can generate the motor commands obtained from electromyographies (EMGs). (3) The similarity of obtained activation function from the system has compared with the real data activation function, which may prove the possibility of RL in the CNS (basal ganglia). Finally, in order to have a graphical and effective representation of the arm model, the virtual reality environment of MATLAB has been used. Conclusion: Since the RL method is a representative of the brain’s control function, it has some features, such as better settling time, not having any peek overshoot, and robustness.

P2P power trading based on reinforcement learning for nanogrid clusters