Vision Based Robot Programming

Surface manufacturing is a process of adding material to or removing material from the surfaces of a part, where industrial robots are usually used. Robot programming for these applications is still time consuming and costly. Typical teaching methods are not cost effective and efficient. There are many offline programming methods developed to reduce the robot programming effort. However, these methods suffer many practical issues, such as cable/hose tangling, robot configuration, collision and reachability. To solve these problems, this paper discusses a new method to generate robot programs based on existing ones in the database, which contains not only the robot paths, but also the programmers' knowledge and process parameters. Since industrial robots have been used in production for decades, there are many robot programs for different parts generated by the robot programmers. Some of these parts are quite similar, such as car bodies. Therefore, these robot programs can be transformed to generate new ones with some adjustments based on the geometry of new parts. In this paper, a robot path generation method is developed based on the existing robot paths in the database. Experiments were performed to validate the developed methodology. The results are very promising in reducing the programming efforts in surface manufacturing.

A virtual-physical collision detection interface for AR-based interactive teaching of robot

Intuitive robot programming through environment perception, augmented reality simulation and automated program verification

Modern programming of industrial robots is often based on the teach-repeat paradigm: a human operator places the robot in many key positions, for teaching its task. Then the robot can repeat a path defined by these key positions. This paper proposes a vision-based approach for the automation of the teach stage. The approach relies on a constant auto-calibration of the system. Therefore, the only requirement is a precise geometrical description of the part to process. The realism of the approach is demonstrated through the emulation of a glue application process with an industrial robot. Results in terms of precision are very promising.

This paper presents the use of a laser tracker for programming paths of industrial robots. The idea of operation, application, and the offered accuracy of the laser trackers are discussed. Then, the developed method of using a tracker for determining the path points of an industrial robot is presented. The requirements related to the developed method are indicated. The parameters of the industrial robot related to the repeatability and accuracy of the robots and the Absolute Accuracy option—increasing the accuracy—are characterized. The developed method is based on the accuracy of robots and the conversion of the position and orientation of individual coordinate systems. In the developed method, the robot is not used during programming and the programmer indicates the path points with the help of a retroreflector. The algorithm, which is the most important part of the developed method, has been implemented in the Rapid programming language to automate the programming process. This article provides an example of the use of the developed method to program the robot path during the grinding process. The advantages of the developed method are characterized and compared to existing solutions. In addition, the limitations and achieved accuracies are indicated.

Gesture-based control systems for industrial robotics benefit from a natural form of human machine interaction. Among the high costs of such systems, there are high requirements for safety, robustness of the gesture recognition and general applicability in the industrial environment. Novel consumer electronic sensors provide a robust recognition, reliability and adequate accuracies at low costs. We present our work on gesture-based control and programming with an emphasis on a natural definition of complex trajectories. Besides standard online programming methods, we introduce a programming technique, which enables the worker to define a movement by his body parts, e.g. the hand. Imitating the human movement, the robot can move its end-effector relatively from a starting point.

The integration of equipment and other devices built into industrial robot cells with modern Ethernet interface technologies and low-cost mass produced devices (such as vision systems, laser scanners, force torque-sensors, PLCs and PDAs etc.) enables integrators to offer more powerful and smarter solutions. Nevertheless, the programming of all these devices efficiently requires very specific knowledge about them, such as their hardware architectures and specific programming languages as well as details about the system's low level communication protocols. To address these issues, this paper describes and analyses the Plug-and-Play architecture. This is one of the most interesting service-oriented architectures (SOAs) available, which exhibits characteristics that are well adapted to industrial robotics cells. To validate their programming features and applicability, a test bed was specially designed. This provides a new graphical service orchestration which was implemented using Workflow Foundation 4 of .NET. The obtained results allowed us to verify that the use of integration schemes based on SOAs reduces the system integration time and is better adapted to industrial robotic cell system integrators.

This paper presents a fast and accurate method for programming simple strategies for small autonomous robots based on embedded processing platforms. There are several approaches in programming of mobile and industrial robots, but the most common one is online programming. This method begins with so called teaching phase during which human operator guides the robot by issuing a set of simple commands. All that commands are automatically logged and saved into a command log, which in the operation phase eventually serves as final program. In case of industrial robots, the whole process including teaching and operation phase runs on the same platform, usually PC, which by the rule has convenient user interface. On the other side, small embedded robot platforms do not have appropriate user interfaces and their programs must be written in the form of hard-coded precompiled firmware. This paper describes the architecture of the robot and supporting system which are designed in the way that support convenient usage of PC based software during development phase, and a fast and accurate method for translation of tested command sequence into the embedded platform firmware running the robot during the deployment phase.

An adaptive tool path generation algorithm for precision surface machining

This paper presents a novel tablet based end-user interface for industrial robot programming (called Hammer). This application makes easier to program tasks for industrial robots like polishing, milling or grinding. It is based on the Scratch programming language, but specifically design and created for Android OS. It is a visual programming concept that allows non-skilled programmer operators to create programs. The application also allows to monitor the tasks while it is being executed by overlapping real time information through augmented reality. The application includes a teach pendant screen that can be customized according to the operator needs at every moment.

AimsDespite the 1988 ‘Dundee Initiative’, which maximised the use of view and grant examinations to reduce the invasive forensic autopsy rate in Tayside, the view and grant itself remains controversial....

An industrial robot is an automatic multi-purpose manipulator, programmable in three or more axes. A program written in a high-level programming language controls these robots, many of these programs involve multiple tasks controlling different robots. Data races are a common problem in concurrent and multi-threaded programming and they are of big concern for the multi-task industrial robotics programmers too. We present a static analysis method for detecting data races in multi-task programs for industrial robots. We propose a technique based on a relation that models when two or more statements from a task occur in between two or more statements in another task. Our static analysis is preceded by a manual, dynamic analysis step for ensuring consistency among tasks for one of the constructs which involves a task waiting for a particular duration. We define a set of not-occurs in-between rules to detect whether two statements in different tasks may race with each other. We have developed a prototype implementation of our tool for the Rapid programming language that is used to program industrial robots of ABB. Rapid has all the features of a typical programming language for industrial robots and hence our race detection framework will generalize to any programming language for industrial robots.

Industrial robots and cobots are widely deployed in most industrial sectors. However, robotic programming still needs a lot of time and effort in small batch sizes, and it demands specific expertise and special training, especially when various robotic platforms are required. Actual low-code or no-code robotic programming solutions are exorbitant and meager. This work proposes a novel approach for no-code robotic programming for end-users with adequate or no expertise in industrial robotic. The proposed method ensures intuitive and fast robotic programming by utilizing a finite state machine with three layers of natural interactions based on hand gesture, finger gesture, and voice recognition. The implemented system combines intelligent computer vision and voice control capabilities. Using a vision system, the human could transfer spatial information of a 3D point, lines, and trajectories using hand and finger gestures. The voice recognition system will assist the user in parametrizing robot parameters and interacting with the robot’s state machine. Furthermore, the proposed method will be validated and compared with state-of-the-art “Hand-Guiding” cobot devices within real-world experiments. The results obtained are auspicious, and indicate the capability of this novel approach for real-world deployment in an industrial context.

In order to have an efficient use of digital image processing and pattern recognition techniques, it is necessary to understand the human visual system behaviour. The human visual system comprises the human eye and some areas of the human brain which performs neurological processing information. Human eye and brain together convert the optical information in a visual scene perception, the eye functions as the human visual system camera, and his function is to transform light electromagnetic waves to small electrical signals carrying the information to the brain to achieve data analysis and build a structured high resolution image. For a machine vision, this process represents a very complex task to implement; adequate elements have to exist like: environment perception, information processing and primitive base knowledge generation to achieve specific actions for decision making based on interpretation of such information. Artificial vision then might be defined as: machine implementation with capabilities of visual perception of the surrounding environment, extraction of region of interest, analysis, scene interpretation and decision making. In this regard some authors like Haralick & Shapiro have defined machine vision as the science which study and develop the theoretical bases and algorithms to obtain information of the real world from one or some images (Haralick&Shapiro, 1992). Another notable definition is proposed by Pajares et al. that define it as a machine capability to see its rounded world, to understand the structure and properties of a 3D world based in the analysis of one or more 2D images. The system described in this article resembles the above capability in automated industrial applications using manufacturing machinery or robots, which in some moment of the industrial process need to obtain its position and location within their working space, so to accomplish their labour tasks efficiently. A real example is locating and positioning working pieces within a manufacture area during assembly, painting, sorting or storage operations. The goal is to identify the working piece and make reference to a visual scene, then with a geometrical model, the working piece position is calculated and so the necessary path to reach the pieces using a robot-arm as described by de Lope (de Lope, et al., 1997) typically in an eye-in-hand configuration. A frequent requirement that we have found during manipulative tasks using mobile robots or multiple collaborative robot arms is to accurately locate objects within the work space or

The block-based approach to programming is an effective way to engage young learners in programming and the powerful ideas of computing. In this paper, we explore the potential of using this same approach in a very different programming context: industrial robotics. Using a customized language built with the Blockly library, we created a block-based interface for programming a one-armed industrial robot. This paper presents a block-based robot programming language called Robot Blockly, focusing on how the various affordances of block-based programming were utilized to make the challenge of robot programming more accessible. We also present results from a small-scale study showing adults with no prior programming experience successfully programming a virtual robot to accomplish a pick and place task. The contribution of this work is in showing the potential for block-based programming beyond young learners and classrooms.