Multi-robot Cooperative System Modeling and Control Software Development

The present study describes the development of control hardware and software for a mobile welding robot. This robot is able to move and perform welding tasks in a double hull structure. The control hardware consists of a main controller and a welding machine controller. Control software consists of four layers. Each layer consists of modules. Suitable combinations of modules enable the control software to perform the required tasks. Control software is developed using C programming under QNX operating system. For the modularizing architecture of control software, we designed control software with four layers: Task Manager, Task Planner, Actions for Task, and Task Executer. The embedded controller and control software was applied to the mobile welding robot for successful execution of the required tasks. For evaluate this imbedded controller and control software, the field tests are conducted, it is confirmed that the developed imbedded controller of mobile welding robot for shipyard is well designed and implemented.

With the advancement of intelligent perception in multirobot systems, cooperative localization in dynamic environments has become a critical component. However, existing multirobot cooperative localization systems still fall short in meeting high-precision localization requirements in Global Navigation Satellite System (GNSS)-denied environments. In this paper, we propose a factor-graph-based resilient cooperative localization (FG-RCL) algorithm for multirobot systems. This algorithm integrates measurements from visual sensors and Ultra-WideBand (UWB) to achieve accurate cooperative state estimation—overcoming the visibility issues of visual sensors within limited fields of view. We utilize the Joint Probabilistic Data Association (JPDA) algorithm to calculate the corresponding probabilities of multiple visual detection measurements between robots and assign them to their respective edges in the factor graph, thereby addressing the data association challenges in visual detection measurements. Finally, simulation results demonstrate that the proposed algorithm significantly reduces the influence of visual detection measurement interference on the performance of cooperative localization. Experimental results indicate that the proposed algorithm outperforms UWB-based and vision-based methods in terms of localization accuracy. The system is implemented using a factor-graph-based optimization approach, and it exhibits scalability and enables plug-and-play for sensors. Furthermore, it demonstrates resilience in abnormal situations.

With the growing adoption of Industry 4.0 concepts in production systems, new challenges arise in engineering control software. Highly distributed control with tight real-time constraints and safety regulations results in increasingly complex software. Current research focuses on increasing the abstraction with new architectures and modularization of software. The presented PhD research addresses modeling of the interactions between control software components, and of the emergent behavior of these compositions. Such behavior models can support the initial implementation, and facilitate (semi-)automated testing and monitoring of control software. Finally, visualizing behavior in a model can enhance understandability of existing control software, when software developers need not access abstracted hierarchy levels to deduct their functionality. This work aims at optimizing the benefit of behavior models in developing control software: Modeling the expected behavior directly for new software will allow using them throughout the software life-cycle. For legacy software, the initial development effort of behavior models will be minimized by automatically capturing behavior models from the implementation. The approach is evaluated in case studies and user studies to integrate experiences from the industrial domain into this software engineering research.

Summary Mobile robots are sophisticated mechatronic systems. Sensing, communication, and control technologies of mechatronic systems are applicable in multi-robot cooperative tasks. In this chapter, a fast computer vision algorithm is presented for a multi-robot cooperative control system. This algorithm identifies the current poses (positions and orientations) of a robot and the manipulated object (a rectangle box) from a color image, in real time. Two main challenges are faced in the multi-robot task considered here. The first one concerns the response speed of the vision subsystem. Ordinary computer vision algorithms available in the literature usually are rather slow and unacceptable in the present real-time system. The second challenge comes from uneven lighting, which makes it very difficult for the vision subsystem to trace the same color blob in different positions. A fast computer vision algorithm is presented to cope with these challenges. First, an image in the RGB (red—green—blue) color space is converted into the HSI (hue—saturation—intensity) color space. Then the saturation and the intensity components of the image are removed, and only the hue component is retained. This operation effectively removes the disturbances caused by uneven lighting conditions in the environment when the robot and the object move from one position to another. Second, filtering and template matching technologies are employed to remove the disturbances from the background and other objects in the image. Finally, coordinate transformations are used to reconstruct the poses of the robot and the object when they are moving. The experiment results are presented to show the feasibility and the effectiveness of the algorithm.

A metamodeling approach for safe control software

Mobile edge computing (MEC) deployment in a multi-robot cooperation (MRC) system is an effective way to accomplish the tasks in terms of energy consumption and implementation latency. However, the computation and communication resources need to be considered jointly to fully exploit the advantages brought by the MEC technology. In this paper, the scenario where multi-robots cooperate to accomplish the time-critical tasks is studied, where an intelligent master robot (MR) acts as an edge server to provide services to multiple slave robots (SRs) and the SRs are responsible for the environment sensing and data collection. To save energy and prolong the function time of the system, a task assignment and resource management scheme is proposed to optimize the computation and communication resources. In the scheme, not only the energy consumption, but also the remaining energies of the SRs are considered to enhance the robustness of the system. Through the analysis and simulation results, we demonstrate that the proposed scheme effectively allocates tasks and resources, while the scheme has low computational complexity and robust to the unreliable wireless communication channels.

Multirobot systems are a promising step forward in supporting human operators, since they can provide a wider set of capabilities and an improved system performance in terms of adaptability, resilience and scalability, without increasing the complexity of the single system element. Team heterogeneity is a multirobot system strength, but it raises interoperability problems which need to be solved to avoid the overall system performance and costs to be impacted by issues such as vendor dependency, high integration costs, high system maintenance costs. Moreover, interoperability is mandatory for a sound integration of multirobot systems as components of System of System architecture where both legacy and innovative segments need to coordinate each others. This paper describes the outcome of the Simulated Interactive Robotics Initiative (SIRI), a project aimed at providing a first assessment of the interoperability between Multirobot and legacy Command & Control Systems. The project developed an interoperability solution which is based upon the following key standards: the Data Distribution Services for Real Time Systems (DDS), which addresses the data exchange service interoperability, and the National Information Exchange Model (NIEM), which addresses the data models interoperability. SIRI provides a preliminary demonstration of the interoperability advantages stemming from the synergic adoption of the mentioned standards in multirobot system and paves the way for a more consolidated and robust framework which follows the system architecture drivers of Interoperable Open Architecture (IOA) a set of guidelines which has been already resulted in cost saving in the procurement of many large defence systems.KeywordsRobotic SystemMilitary OperationUnmanned Ground VehicleInteroperability IssueMultirobot SystemThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

A three-step approach is proposed to substantially reduce the high cost of development of control software for an FMS on the shop floor. First, by introducing the concept of multiple computing processes into Automata Theory, a finite capacity machine (FCM) is presented as an extended finite machine. An FCM is used to model the control of a flexible manufacturing system, which avoids the state explosion problem. Secondly, a structured adaptive supervisory control (SASC) model meeting the desired control objectives for an FMS is derived from a composed FCM. The SASC model can be constructed systematically. Finally, generic software components for an SASC model are described, which allow the SASC model to be cost-effectively transferred into shop-floor applicable control software.

Automated PLC Software Testing using adapted UML Sequence Diagrams

In this paper, a multi-robot cooperative transport system based on NAO robot is designed. Based on the kinematics modeling, object recognition and localization and target grasping of humanoid robots in complex scenes, the binocular vision module is extended by using multiple humanoid robots NAO. Finally, based on multi-sensor fusion and driven by cooperative handling task, the dynamic identification of object position and pose, trajectory planning of robot motion and cooperative handling control are analyzed and designed. Experimental tests in real environment show that the system has the advantages of intelligence and good synergistic effect, which has reference value for the application and in-depth development of service robot.

The control and coordination of multiple mobile robots is a challenging task; particularly in environments with multiple, rapidly moving obstacles and agents. This paper describes a robust approach to multi-robot control, where robustness is gained from competency at every layer of robot control. The layers are: (i) a central coordination system (MAPS), (ii) an action system (AES), (iii) a navigation module, and (iv) a low level dynamic motion control system. The multi-robot coordination system assigns each robot a role and a sub-goal. Each robot’s action execution system then assumes the assigned role and attempts to achieve the specified sub-goal. The robot’s navigation system directs the robot to specific goal locations while ensuring that the robot avoids any obstacles. The motion system maps the heading and speed information from the navigation system to force-constrained motion. This multi-robot system has been extensively tested and applied in the robot soccer domain using both centralized and distributed coordination.

Nowadays due to the technology development and use of digital computers in various systems, need for development of high performance and robust software is attracting great attentions. Because of increasing complexity in algorithms and implementation hardware for embedded systems, proper simulation tools are required. In sophisticated systems design, hardware in the loop (HIL) simulation is known as a prominent simulation tool before realistic tests of the system and a step after software simulation. Simultaneously it can be used for verification and validation of automation and control software. HIL has had an historical background in aerospace industries. Recently, this tool has spread in different steps of system life cycle such as design, development, implementation and test of various applications including automobile industry, shipbuilding, power lines, robotic systems and etc. Utilizing a suitable hardware in the loop laboratory, in system design stages is a practical way to increase the system reliability and efficiency as well as value of product. Also, by proper investigation in this modelling and simulation method, many errors can be avoided in design procedure of software and hardware as well as their interconnections. In this study, structure and components of an hardware in the loop laboratory for different systems are explored, also it is tried to more evaluate the applications of HIL simulations in dynamics and control engineering. At last, general structure of an hardware in the loop lab for diverse industries is proposed and discussed.

In a smart energy system, power converters should be controlled by complex control software. However the efficient development of reliable control software is difficult. We have proposed to introduce a “model-based design” process using MATLAB/Simulink, established in the automotive and aerospace fields, to the development of power electronics systems. However, it was difficult in the past because calculating switching circuit as the plant model is difficult in MATLAB/Simulink due to the socalled stiff equation problem. Thus, in this study, we have developed a simulation environment in which the standard model-based design tool, MATLAB/Simulink, and the simulation engine for power switching circuits, SCALE, work cooperatively. SCALE was originally developed to solve stiff equation and efficiently simulate power switching circuits. This software environment enables us to design control algorithms with intuitive block diagram expression and test them in actual hardware, without writing a C program. Furthermore to improve this environment, a mechanism that handles the multiple sample rates was introduced. Smart energy systems that use several controllers with different sampling rates can be simulated in the proposed environment. Thus, we can develop control software for smart energy systems in a timely and cost effective manner.

Introduction. The “Smart Agroˮ committee of Research and Education Center “Engineering of the Future” has identified a number of tasks relevant for improving the efficiency of precision, soil-protecting and conservation agriculture. One of these tasks is the development of a digital multi-agent system, which provides a number of services for agricultural enterprises, developers and manufacturers of agricultural machinery. The purpose of the present study is to model an autonomous mobile robotic platform, including the development of software and hardware for trajectory control. Materials and Methods. To solve the problem, there are used modern CAx systems and their applications, the methods of 3D and full-body modeling, and the method of numerical solution of problems in solid mechanics. To expand and improve the standard functionality of CAx-systems (SolidWorks) in the software implementation of trajectory control algorithms, the methods and technologies of programming using API SolidWorks, VisualStudio C++ (MFC, ATL, COM) are used, and to build physical full-scale models ‒ Arduino and fischertechnik platforms. Results. The result of the study is a software and hardware module of trajectory control for an integrated (physical and virtual) model of a mobile robotic platform, which can be provided to the consumer as a service for technology autonomation. For the developed integrated model, control algorithms for various types of trajectories were tested. Discussion and Conclusion. The developed integrated software and hardware model of trajectory control can be used by developers and manufacturers of agricultural machinery, and directly by agro-enterprises for implementing typical technological processes. A feature of the implementation is an open hardware and software interface that provides the integration of mobile robotic platforms based on a digital multi-agent system.

In this paper, multirobot system is introduced in map-building task. The data fusion and multirobot cooperation are focused on. A suitable range sensor model is addressed, and several situations that occurred in multiple robots building map are analyzed. Evidential reasoning based on Dempster-Shafer theory is used in data fusion, and cooperation strategies are presented to avoid invalid sensing information.