Intuitive Robot Programming Using Augmented Reality

In this article, we present a client/server augmented reality (AR) system for viewing complex assembly models on mobile phones. Complex model information is located on a PC which takes care of all of the heavy AR tracking and rendering computation. A camera phone is used as a client to show this information, augmented on still images as animated view. The mobile phone interface also supports correct masking of the assembly pieces as they come together, as well as a graphics only viewing mode intended for better understanding of the assembly process. In addition to describing the mobile augmented assembly system we present results from two pilot user studies evaluating elements of the user interface. INTRODUCTION In order to remain competitive, today’s industry is employing a growing number of product variations with shorter and shorter life cycles, requiring efficient ways to handle the production planning and processing. Assembly line workers face more and more complex tasks and an increasing memory load is posed by the new production needs. Experienced workers may be able to handle several product variants at the same time, but new workers require training and guidance to handle the work tasks. Related activities such as maintenance and repair also require a lot of know-how of complex products. Guidance in assembly tasks is commonly provided by using printed blueprints or other documents. However, these are mainly two dimensional images or text while the real assembly task is three dimensional, making the relationship between parts hard to understand. Computer based technologies have been developed such as electronic guides, multimedia training material and interactive documents. However, these typically require the user to turn away from the assembly task to study a computer screen separate from the workspace. In our research we explore how augmented reality (AR) technology can be used to provide more intuitive guides to assist with assembly, repair and training tasks. With AR technology, synthetic objects can be merged with the user’s view of the real world so that the user can perform the assembly task without needing to look at a separate screen. The virtual objects appear in the correct position so that AR enables the creation of realistic looking 3D animated assembly “manuals”. Other researchers have developed prototype AR systems for supporting assembly tasks. These are typically based on head mounted displays (HMDs) connected to desktop or wearable computers. However, wearable computers are relatively expensive, they have a short battery life, and taking them into a harsh industrial environment is not always possible. Head mounted displays can also be bulky, they involve safety issues, and many workers are reluctant to wear them. To overcome these disadvantages, we are interested in developing a reliable mobile phone based AR assembly system. Today’s mobile phones have fast processors, significant memory, 3D graphics support, and high resolution displays. They contain various accessories that can be used for developing mobile AR applications such as good quality cameras, connectivity by Bluetooth, WLAN and GPRS, and various SAM.700.003.1155

Driven by digitalization in supply chains, the use of Collaborative Robots (COBOTs) has become increasingly widespread in recent years. They significantly contribute to process efficiency by working in place of, or in collaboration with, humans in a variety of operations, including welding, painting, assembly and disassembly, transportation, packaging, and palletizing. However, when uncertainty and different criteria are taken into account, decision support systems that compare practical robots based on their suitability for specific needs are inadequate. This study presents a comprehensive multi-criteria decision-making (MCDM) framework for prioritizing COBOTs with different features used in digital supply chain processes. Based on in-depth research in the literature and the opinions of experts working in companies that use relevant robots in the industry, the criteria to be evaluated when selecting COBOT types are identified. The importance of these criteria was determined using the Picture Fuzzy Step-wise Weight Assessment Ratio Analysis (PiF-SWARA) method, which effectively captures the uncertainty in experts’ decisionmaking processes. Subsequently, alternative COBOT types were ranked using the Picture Fuzzy Combinative Distance-Based Assessment (PiFCODAS) approach. This case study, which evaluates the PiF-SWARACODAS concept, reveals that according to expert assessments, cost is the most important criterion in COBOT selection, followed by process quality and space utilization. The findings about the selection of types emphasize that high-efficiency articulated robots operating at high speeds under mass production conditions are the primary priority. These robots are followed by humanoid robots. The third most important are power and force-limiting robots. The fourth and fifth types of COBOTs are hand-guided and safety-monitored stop robots. Validation and sensitivity analyses confirmed the robustness of the results. Overall, the proposed framework not only clarifies the key priorities for manufacturing facilities but also provides a validated decision support tool to align digitalization strategies with the most appropriate COBOT investments.OPEN ACCESS Received: 25/08/2025 Accepted: 15/10/2025 Published: 15/12/2025 An extensive literature review was conducted to examine previous research on COBOT applications in the supply chain, to identify evaluation criteria and alternatives for this study, and to highlight the originality of this study. The concept of COBOT was included in books, journals and conference proceedings accessible through the SCOPUS database. However, the literature search with the related keyword found too many studies. Therefore, the scope was further customized to include different types of COBOT models that can be used throughout the supply chain. To make the literature review process more systematic, the PRISMA methodology, which stands for “Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)”, was used. The PRISMA approach, designed by Moher et al. [10], has become widely used among academics in recent years. Sources are analyzed and evaluated according to the determined eligibility criteria. Then, appropriate studies are selected. Table 1lists the keywords used to perform a PRISMA method literature review, along with the number of studies found using these Table 1: Literature search with A total of 1179 studies were found in SCOPUS after searching with the were found. The results were duplicated because some

The use of collaborative robots in the industrial domain has significantly grown in the last years, allowing humans to operate in the same workspace occupied by robots without any physical barriers. Understandably, the safety of the human operator has been a major concern both for researchers and regulatory bodies. The power and force limited modality of robots is of particular interest in that sense, being used in order to bound the energy of eventual collisions when a close physical interaction with humans is necessary. Such an interaction modality allows the robotic system to operate without the use of barriers, but a measurement of the force and pressure occurring due to a contact must be provided as part of the risk assessment. However, the precise procedure to follow in order to reliably provide such measures is still unclear for users and system integrators willing to self-assess the safety of their own collaborative robotic system. In this work, the repeatability and reliability of such testing procedures and measures are analyzed with an interlaboratory comparison approach, with the aim to establish the degree of variability possibly encountered when performing the same test under slightly different conditions.

Current paper deals with a new tendency in robotics, consisting in broader and broader exploiting of socalled collaborative robots, sharing their workspace with humans. An overview of research in this area has been performed. Some aspects of robot programming method, called programming by demonstration, have been discussed. In particular, the authors try to answer the question, whether the idea of direct human-robot cooperation can be applied also in welding. Examples of research in this area have been presented. The paper discusses some as- pects of human-robot communication. It presents a conception of a novel method of calibration of human-robot voice communication system, now under development at the Warsaw University of Technology. This method is focused on specific requirements resulting from the use of collaborative robots.

A proposal of an assembly workstation for car panel fitting aided by an augmented reality device

The use of collaborative robots (cobots) and human robot collaboration has started to increase in diverse industrial areas in the past years. However, cobots are seldom applied in construction work even if there is a growing need for robotics solutions in this area. This study aims to present the possibilities of applying lightweight cobots for single tasks in the construction sector. This paper presents the development of a proof-of-concept dedicated to screwing gypsum board panels to the ceiling of a room. After identifying the challenges related to this task, we present the cobot solution that we implemented for this purpose. Our results show that repetitive screwing work in upright position achieved by cobot can benefit workers by removing ergonomical strain. At the same time, cobot ensures that this task is achieved with constant performance. Future research directions lead to the search for better accuracy in screw positioning and more dynamic ways of collaboration with human workers.

Robotic assembly tasks involve complex and low-clearance insertion trajectories with varying contact forces at different stages. While the nominal motion trajectory can be easily obtained from human demonstrations through kinesthetic teaching, teleoperation, simulation, among other methods, the force profile is harder to obtain especially when a real robot is unavailable. It is difficult to obtain a realistic force profile in simulation even with physics engines. Such simulated force profiles tend to be unsuitable for the actual robotic assembly due to the reality gap and uncertainty in the assembly process. To address this problem, we present a combined learning-based framework to imitate human assembly skills through hybrid trajectory learning and force learning. The main contribution of this work is the development of a framework that combines hierarchical imitation learning, to learn the nominal motion trajectory, with a reinforcement learning-based force control scheme to learn an optimal force control policy. To further improve the imitation learning part, we develop a hierarchical architecture, following the idea of goal-conditioned imitation learning, to generate the trajectory learning policy on the skill level offline. Through experimental validations, we corroborate that the proposed learning-based framework is robust to uncertainty in the assembly task, can generate high-quality trajectories, and can find suitable force control policies, which adapt to the task's force requirements more efficiently.

Scaling up and automating business processes makes one of the greatest contributions to the development of the business. The growth of sales of the goods or services may lead to the sophistication of the enterprise's business processes, increased number of staff, and the necessity to create new departments and divisions. Unfortunately, increased number of employees does not always correlate with the increase in the revenue of the enterprise. Instead, forced business process automation is regarded to be more effective. The integration of robotics solutions is one of the main ways to automate the production. This kind of modernization is one of the key stages to ensure further development within the Fourth Industrial revolution. The main result of the business processes automation and the whole Fourth Industrial Revolution (Industry 4.0) will be a fully “unmanned” production facility, where robotic solutions have entirely replaced the employees at their workspaces. In this way, robots will be able to perform those kinds of work that are dangerous for the humans. The robots-based production processes can be continuous bringing a positive impact on the efficiency and the productivity of the enterprise. This paper describes the automation of the business process on production facilities with the use of collaborative robotics.

This scientific paper examines the use of collaborative robots in the context of Industry 5.0 and its impact on practical engineering education. The aim of this paper is to highlight the importance as well as the challenges of using robots in Industry 5.0 and to analyze how this can be integrated into engineering education. The paper discusses various aspects such as the technological advances, the need for interaction between humans and robots, and the competencies that engineers need in an increasingly digitalized working world. Furthermore, practical methods and approaches in engineering education at the Zittau/Görlitz University of Applied Sciences are presented to prepare students for the requirements of Industry 5.0. Additionally, a draft of a future interdisciplinary learning factory 5.0 will be presented, in which networked technologies with the focus on robotics run through various scenarios in the context of Industry 5.0 at different locations.

In recent years, collaborative robots have been introduced into production systems as a new generation of industrial robots, sharing work with workers. Unlike industrial robots and workers in the past to work in isolation, collaborative robots and workers can often share work tasks and space. The introduction of these collaborative robots makes human-robot collaboration (HRC) more and more important, which is an emerging technology in the field of production systems. According to the characteristics of workers and collaborative robots, a measurement model of assembly complexity is first proposed. The assembly tasks are analyzed from the three aspects of process depth, operation process, and cognitive process, and the complexity of each assembly task is evaluated. The indicator of the machine-machine collaboration attribute measures each task to determine whether the task can be completed by a collaborative robot; finally, an assembly task allocation evaluation model is established, to ensure the assembly complexity of each worker’s task is balanced, so that the cognitive load assigned to the robot task is lower than the worker’s task, to maximize production efficiency and reduce the assembly system cost.

Skill-based instruction of collaborative robots in industrial settings

Learning from Demonstration (LfD) is a family of methods used to teach robots specific tasks. It is used to assist them with the increasing difficulty of performing manipulation tasks in a scalable manner. The state-of-the-art in collaborative robots allows for simple LfD approaches that can handle limited parameter changes of a task. These methods however typically approach the problem from a control perspective and therefore are tied to specific robot platforms. In contrast, this paper proposes a novel motion planning approach that combines the benefits of LfD approaches with generic motion planning that can provide robustness to the planning process as well as scaling task learning both in number of tasks and number of robot platforms. Specifically, it introduces Dynamical Movement Primitives (DMPs) based LfD as initial trajectories for the Stochastic Optimization for Motion Planning (STOMP) framework. This allows for successful task execution even when the task parameters and the environment change. Moreover, the proposed approach allows for skill transfer between robots. In this case a task is demonstrated to one robot via kinesthetic teaching and can be successfully executed by a different robot. The proposed approach, coined Guided Stochastic Optimization for Motion Planning (GSTOMP) is evaluated extensively using two different manipulator systems in simulation and in real conditions. Results show that GSTOMP improves task success compared to simple LfD approaches employed by the state-of-the-art collaborative robots. Moreover, it is shown that transferring skills is feasible and with good performance. Finally, the proposed approach is compared against a plethora of state-of-the-art motion planners. The results show that the motion planning performance is comparable or better than the state-of-the-art.

One of the major challenges faced by Augmented Reality (AR) systems is linking virtual content accurately on physical objects and locations. This problem is amplified for applications like mobile payment, device control or secure pairing that requires authentication. In this paper, we present an active LED tag system called GLITTER that uses a combination of Bluetooth Low-Energy (BLE) and modulated LEDs to anchor AR content with no a priori training or labeling of an environment. Unlike traditional optical markers that encode data spatially, each active optical marker encodes a tag's identifier by blinking over time, improving both the tag density and range compared to AR tags and QR codes.We show that with a low-power BLE-enabled micro-controller and a single 5 mm LED, we are able to accurately link AR content from potentially hundreds of tags simultaneously on a standard mobile phone from as far as 30 meters. Expanding upon this, using active optical markers as a primitive, we show how a constellation of active optical markers can be used for full 3D pose estimation, which is required for many AR applications, using either a single LED on a planar surface or two or more arbitrarily positioned LEDs. Our design supports 108 unique codes in a single field of view with a detection latency of less than 400 ms even when held by hand.

Assembly task is an activity of collecting parts/components and bringing them together through assembly operations to perform one or more of several primary functions. As an emerging and powerful technology, Augmented Reality (AR) integrates images of virtual objects into a real world. Due to its self-characteristic features, AR is envisaged to provide great potentials in guiding assembly task. In this paper, it reviews some AR applications in the area of assembly and elaborates the great potentials of integrating animated agent with current AR technique in guiding product assembly. Besides, in view that different assembly operations share certain common features and functions in essence, the authors formulate the experimental framework for evaluating cognitive workload of using animated AR system in general assembly task and make the framework general to be applied in evaluating diverse classes of AR systems for different assembly operations.

Optimal layout planning for human robot collaborative assembly systems and visualization through immersive technologies