Blockly goes to work: Block-based programming for industrial robots

An industrial robot is an automatic multi-purpose manipulator, programmable in three or more axes. A program written in a high-level programming language controls these robots, many of these programs involve multiple tasks controlling different robots. Data races are a common problem in concurrent and multi-threaded programming and they are of big concern for the multi-task industrial robotics programmers too. We present a static analysis method for detecting data races in multi-task programs for industrial robots. We propose a technique based on a relation that models when two or more statements from a task occur in between two or more statements in another task. Our static analysis is preceded by a manual, dynamic analysis step for ensuring consistency among tasks for one of the constructs which involves a task waiting for a particular duration. We define a set of not-occurs in-between rules to detect whether two statements in different tasks may race with each other. We have developed a prototype implementation of our tool for the Rapid programming language that is used to program industrial robots of ABB. Rapid has all the features of a typical programming language for industrial robots and hence our race detection framework will generalize to any programming language for industrial robots.

End-user programmers need programming tools that are easy to learn and use. Development environments for end-users often support one of two visual modalities: block-based programming or data-flow programming. In this work, we discuss differences in how these modalities represent programs, and why existing block-based programming tools are better suited for imperative tasks while data-flow programming better supports nested expressions. We focus on robot programming as an end-user scenario that requires both imperative and expressions-based code in the same program. To study how end-user tools can better support this scenario, we propose two programming system designs: one that changes how blocks represent nested expressions, and one that combines block-based and data-flow programming in the same hybrid environment. We compared these designs in a controlled experiment with 113 end-user participants who solved programming and program comprehension tasks using one of the two environments. Both groups indicated a small preference for the hybrid system in direct comparison, but participants who used blocks to solve tasks performed better on average than hybrid system users and gave higher usability ratings. These findings suggest that despite the appeal of data-flow programming, a well-adapted block-based programming interface can lead end-users to more programming success.

Surface manufacturing is a process of adding material to or removing material from the surfaces of a part, where industrial robots are usually used. Robot programming for these applications is still time consuming and costly. Typical teaching methods are not cost effective and efficient. There are many offline programming methods developed to reduce the robot programming effort. However, these methods suffer many practical issues, such as cable/hose tangling, robot configuration, collision and reachability. To solve these problems, this paper discusses a new method to generate robot programs based on existing ones in the database, which contains not only the robot paths, but also the programmers' knowledge and process parameters. Since industrial robots have been used in production for decades, there are many robot programs for different parts generated by the robot programmers. Some of these parts are quite similar, such as car bodies. Therefore, these robot programs can be transformed to generate new ones with some adjustments based on the geometry of new parts. In this paper, a robot path generation method is developed based on the existing robot paths in the database. Experiments were performed to validate the developed methodology. The results are very promising in reducing the programming efforts in surface manufacturing.

In this paper, a range sensing setup for performing detection and monitoring in an industrial robot workspace is presented. The setup uses a ring of Time-of-Flight laser range sensors mounted on a robot. A 3D simulation setup with the properties of the sensor mounted on a UR10 robot and a simple pick and place task with a human avatar is used to analyze its behavior and viability with industrial arm robots. Collision detection strategies based on human-robot separation distance and relative speeds are also implemented. These strategies are evaluated based on the human safety, robot performance and productivity of the task. The parameters and results of the experiments are tabulated. The results show the benefits of achieving dynamic awareness of the robot in comparison with the conventional methods used in industry. The development of the prototype sensor ring is also shown and the future work discussed.

Intuitive robot programming through environment perception, augmented reality simulation and automated program verification

A Graphical User Interface for Industrial Robot Programming in Non-Repetitive Tasks

Modern programming of industrial robots is often based on the teach-repeat paradigm: a human operator places the robot in many key positions, for teaching its task. Then the robot can repeat a path defined by these key positions. This paper proposes a vision-based approach for the automation of the teach stage. The approach relies on a constant auto-calibration of the system. Therefore, the only requirement is a precise geometrical description of the part to process. The realism of the approach is demonstrated through the emulation of a glue application process with an industrial robot. Results in terms of precision are very promising.

Industrial robots and cobots are widely deployed in most industrial sectors. However, robotic programming still needs a lot of time and effort in small batch sizes, and it demands specific expertise and special training, especially when various robotic platforms are required. Actual low-code or no-code robotic programming solutions are exorbitant and meager. This work proposes a novel approach for no-code robotic programming for end-users with adequate or no expertise in industrial robotic. The proposed method ensures intuitive and fast robotic programming by utilizing a finite state machine with three layers of natural interactions based on hand gesture, finger gesture, and voice recognition. The implemented system combines intelligent computer vision and voice control capabilities. Using a vision system, the human could transfer spatial information of a 3D point, lines, and trajectories using hand and finger gestures. The voice recognition system will assist the user in parametrizing robot parameters and interacting with the robot’s state machine. Furthermore, the proposed method will be validated and compared with state-of-the-art “Hand-Guiding” cobot devices within real-world experiments. The results obtained are auspicious, and indicate the capability of this novel approach for real-world deployment in an industrial context.

Gesture-based control systems for industrial robotics benefit from a natural form of human machine interaction. Among the high costs of such systems, there are high requirements for safety, robustness of the gesture recognition and general applicability in the industrial environment. Novel consumer electronic sensors provide a robust recognition, reliability and adequate accuracies at low costs. We present our work on gesture-based control and programming with an emphasis on a natural definition of complex trajectories. Besides standard online programming methods, we introduce a programming technique, which enables the worker to define a movement by his body parts, e.g. the hand. Imitating the human movement, the robot can move its end-effector relatively from a starting point.

Programming of industrial robots is normally carried out via so-called Teach or Offline programming methodologies. Both methodologies have associated weaknesses like a high time consumption for programming on sculptured surfaces (especially Teach), and both methodologies cannot, at the same time, carry information of the real work piece and the ideal work piece (CAD-model). To be able to see both the real and ideal world is especially important when the industrial robot is used in manufacturing processes like machining, grinding, deburring etc. where the difference between the real and ideal world represents the material removal. This paper presents a new vision based programming methodology which combines information of the real and ideal world, especially adapted for robot grinding and deburring operations. Further, the presented methodology is especially developed with simplicity in mind when it comes to man-machine communication. Thus, a standard marker pen can be used by the operator to draw the robot path directly onto the work piece. This line is captured by a vision system and finally leads to the generation of the robot path. The presented methodology is a further development of the previous work of the authors as presented (Solvang et al., 2007 and Sziebig, 2007), especially concerning optimization of the robot tool positioning with respect to a work piece surface.

Additive manufacturing or 3D printing slowly and steadily replaces traditional subtractive manufacturing, where parts are manufactured by sequentially removing pieces of the workpiece material. In 3D printing, the material is added in layers, which enables producing parts of complex geometric shapes made from expensive metals with higher productivity and economic efficiency. In order to perform 3D printing on an industrial scale, it is necessary to combine robotics with 3D printing software. This article presents the results of the process design for 3D printing of a compressor impeller prototype. A detailed methodology of customization of the KUKA industrial robot for the 3D printing process is given. Once the impeller model was created in the CAD environment, it was cut into layers in the slicer program. Special software for offline programming of industrial robots was used to translate the program into the robot language and perform 3D printing using an experimental robotic complex. As a result, a prototype of a low-carbon steel compressor impeller was printed using a welding machine and steel wire. The developed technological process was successful and proved that an industrial robot combined with a welding machine can be used as an industrial 3D printer for printing real machine parts.

In recent years only large scale enterprises have applied industrial robots, but with decreasing prime cost their application is getting more and more interesting for medium sized companies. But the cost of programming is still comparatively high: The huge variety of different manufacturer-dependent and complex offline-programming tools require high-qualified personnel for the whole process of software development. Furthermore most of the text-oriented programming systems lack user-friendly interfaces and the graphic-supported systems with simulation tools and CAD-data interfaces are still too expensive. Due to these reasons, the Institute for Production Automation at the University of Kaiserslautern developed a new offline programming tool for industrial robots, based on the technique of visual programming. The advantages of this technique is already used in a lot of software applications and is offering a good support especially for non-experts. With the help of this tool, called ROBICON, the user is able to design, build and document any robot program in any code, by using graphical symbols. The knowledge of a textual programming language is not necessary anymore.

This paper presents a fast and accurate method for programming simple strategies for small autonomous robots based on embedded processing platforms. There are several approaches in programming of mobile and industrial robots, but the most common one is online programming. This method begins with so called teaching phase during which human operator guides the robot by issuing a set of simple commands. All that commands are automatically logged and saved into a command log, which in the operation phase eventually serves as final program. In case of industrial robots, the whole process including teaching and operation phase runs on the same platform, usually PC, which by the rule has convenient user interface. On the other side, small embedded robot platforms do not have appropriate user interfaces and their programs must be written in the form of hard-coded precompiled firmware. This paper describes the architecture of the robot and supporting system which are designed in the way that support convenient usage of PC based software during development phase, and a fast and accurate method for translation of tested command sequence into the embedded platform firmware running the robot during the deployment phase.

the deals with the implementation of the method of the trajectories generating for end effectors and control programs for industrial robots in an uncertain working environment. To generate correct trajectories, it is proposed to sequentially process data from vision systems, correct them in the human-machine interface, generate a control program in a generalized form using universal commands and then translate it into the form that a specific industrial robot requires. Experimental studies of the proposed method were carried out, which confirmed the operability and effectiveness of the proposed method.

A new wave of collaborative robots designed to work alongside humans is bringing the automation historically seen in large-scale industrial settings to new, diverse contexts. However, the ability to program these machines often requires years of training, making them inaccessible or impractical for many. This project rethinks what robot programming interfaces could be in order to make them accessible and intuitive for adult novice programmers. We created a block-based interface for programming a one-armed industrial robot and conducted a study with 67 adult novices comparing it to two programming approaches in widespread use in industry. The results show participants using the block-based interface successfully implemented robot programs faster with no loss in accuracy while reporting higher scores for usability, learnability, and overall satisfaction. The contribution of this work is showing the potential for using block-based programming to make powerful technologies accessible to a wider audience.