Unmanned vehicle control using simulation and virtual reality techniques

Teleoperation effectiveness was almost stagnant during several decades since the discovery of the master-slave structure by Goertz in the 50s; in spite of some technical improvements and computer developments [1]. Recent advances on virtual reality (VR) techniques can be considered as the additional ingredient to the total telerobotic renewal, and novel architectures could be developed. Its efficiency is reaching now a considerable level of achievements. It allows a clear improvement in traditional applications such as nuclear or space activities and it also opens many new fields of application, where teleoperation represents a major percentage of robotic development. Build on previous state-of-the-art reports [2, 3, 4], this chapter reviews up-to-date achievements applying virtual reality techniques to teleoperation: we recall some relevant achievements of VR in solving difficult teleoperation problems such as time delay, operator assistance and sharing robot autonomy by combining different supervision strategies or allowing new human-centred teleoperation schemes. The chapter also discusses new robotic applications that have currently appeared which require additional research efforts and call additional investigations on virtual reality techniques. Among them we can find micro- and nano-teleoperation eventually of livings such us cells and DNA molecules, teleoperation of humanoids and animaloids, teleoperation of unmanned air or terrestrial vehicles, hi-fidelity telepresence, multi-operator multirobots teleoperation, etc. These are exemplified along with their specific challenges.

This article starts form the virtual reality technique and simply describes its characteristic. It researches the technology principle and theory model of virtual lab that based on the virtual reality. Moreover, this article provides the frame design and application of practice education platform in the electromechanical major, which is combined with the subsystem of virtual reality technique, model subsystem and virtual instrument subsystem and based on the virtual reality technique. The result shows through this system can effectively increase the practice teaching effect of electromechanical major.

Virtual museum refers to display and simulate the functions of real museum on the Internet in the form of 3 Dimensions virtual reality by applying interactive programs. Based on Virtual Reality Modeling Language, virtual museum building and its effective interaction with the offline museum lie in making full use of 3 Dimensions panorama technique, virtual reality technique and augmented reality technique, and innovatively taking advantages of dynamic environment modeling technique, real-time 3 Dimensions graphics generating technique, system integration technique and other key virtual reality techniques to make sure the overall design of virtual museum.3 Dimensions panorama technique, also known as panoramic photography or virtual reality, is a technique based on static images of the reality. Virtual reality technique is a kind of computer simulation system which can create and experience the interactive 3 Dimensions dynamic visual world. Augmented reality, also known as mixed reality, is a technique which simulates and mixes the information (visual, sound, taste, touch, etc.) that is difficult for human to experience in reality. These technologies make virtual museum come true. It will not only bring better experience and convenience to the public, but also be conducive to improve the influence and cultural functions of the real museum.

The topic of this paper is a spin-off from a study in a much larger ‘VIRPI—Research and Development Project on Virtual Prototyping Services for the Electronics and Telecommunications Industries’ project. A number of Finnish universities and companies participated in VIRPI, which was funded by the companies and the National Technology Agency TEKES. The goal of the research was to develop virtual prototyping techniques for use in product development. The aim of one of the subprojects of VIRPI was to study the use of Internet-based virtual prototypes for usability testing. The result was that virtual prototypes can be used in usability evaluation, that task-based tests seem more beneficial than the free exploration approach, and that emphasis and further research must be directed to the design of the data logging (Kuutti, Sade and Battarbee 1999; Battarbee et al. 1999). In the study, a usability test with a virtual reality modelling language (VRML) model was simulated. An interactive VRML model of a rather simple conceptual mobile phone was placed in a web browser window, where it could be manipulated, and its buttons could be pressed to navigate in its user interface with the according feedback on its display. The fictive product concept was a very small mobile phone with no numeric keypad, to be used as an extension for a communicator. The VRML model was not a very high fidelity prototype aesthetically, which is typical for virtual reality models, because they must be light enough, in terms of computer file size to be manipulated through the Internet. A VRML model is not as precise and detailed as a Computer Aided Design (CAD) model, but it can represent the interactivity of the product. ‘Virtual reality prototyping is a process in which a product or a product concept, its behaviour and usage are simulated as realistically as possible using computer models and virtual reality techniques’ (Kerttula 1999). In the course of comparing the task-based approach to free exploration as the usability testing strategy, users were shown a physical model of the concept at the end of the

In this paper, we extend the traditional `Query-By-Example' (QBE) approach where example shots are provided to represent a query, and then shots similar to them are retrieved. One crucial problem of QBE is that when example shots for the query are unavailable, the retrieval cannot be performed. To overcome this, we propose an innovative approach, named `Query-By-Virtual-Example' (QBVE), where example shots for any arbitrary query can be created by using virtual reality techniques. We call such example shots `virtual example shots'. In our system, virtual example shots are created by synthesizing user's gesture in front of a video camera, 3D object models (3DCGs) and background images. Experimental results on TRECVID 2009 video data show the validity of substituting virtual example shots with real example shots used in QBE.

Investigations into comparisons of real and virtual environments serve multiple purposes. Results may improve development, implementation and application of virtual environments, inform about potentials for positive transfer of effects and facilitate decision making about appropriate virtual reality techniques in specific situations on human factors and ergonomics in human-system interaction. Absolute and relative estimations of object sizes, colours and greyscales were differentially investigated in virtual and real robot cells. Analyses revealed differences for environments (e.g. size, colour) and in some cases even for specific characteristics under investigation (e.g. robot cell size, 10% greyscales). Differential results on human task performance in virtual environments with potential technical constraints are crucial for safety and health reasons and because they provide a basis for human behaviour consequences and work system design decisions and requirements for transfer research. Results support informed decision making about virtual reality techniques and media selection in OSH applications on training and work systems design.

We reviewed the literature on virtual reality techniques in older adults. Although treatment of specific phobias in older adults by virtual reality exposure therapy (VRET) is feasible and might be effective, this has yet not been evaluated in randomized controlled trials. The few controlled studies that involved older adults with post-traumatic stress disorder (PTSD) did not provide robust evidence that VRET is an effective treatment for older people with PTSD. However, research shows that virtual reality techniques can be beneficial to prevent cognitive decline via either cybercycling or memory trainings. A few studies have investigated whether older people may profit from virtual reality training of motor balance, and results are largely positive. It is concluded that it is therapeutically prudent to give older adults more time to adapt to virtual environments in the treatment of anxiety disorders and to slow down the exposure speed for this group.

The purpose of this study is to present a number of aspects in the modern concept of computer-aided dynamic modal control of unmanned quarry vehicles in open pit mining. In particular, the software and hardware module that is a part of the “Smart Quarry” global structure deals with the conditions of matching a form of specific current trajectories (their deviation to the left or right of the nominal axial trajectory) to information “trajectory” chirp signals. The study employs the methods of wavelet transforms to convert one-dimensional signals that generate unmanned vehicle current trajectories into the time-frequency distributions of Cohen’s class. The formation of unmanned vehicle current trajectories under their deviation to the left / right from the nominal axial trajectory on straight and curved routes is considered schematically. It is noted that the tracking of unmanned current trajectories on quarry routes is carried out taking into account the nature of trajectory signals. The difference between the introduced dynamic modal control of the unmanned vehicle and the static one is formulated. Some fragments displaying 1D-signals in a wavelet medium are introduced into the autonomous and external control subsystems. The computer-aided control system uses such elements of the wavelet transforms technique as Gabor wavelet functions, the wavelet matching pursuit algorithm, and Cohen’s class time-frequency distributions. The research results in formulating the criteria for forming the unmanned vehicle current trajectories by the control system in the form of its reactions to sporadic disturbances caused by the occurrence of static or dynamic obstacles on a route. The algorithm of dynamic modal control of current trajectories has been developed. The concept of forward and reverse transient processes of signals of unmanned vehicle trajectory deviation has been introduced. The estimation procedure of modal controller parameters has been described. The algorithm has been developed for modal controller matrix recalculation, which has the form of the chain of sequentially implemented matrix procedures. It should be noted in conclusion that a computer-aided system for modal control of current trajectory deviation has been developed on the basis of the performed research. It enables to implement the functions of controlling the dynamics of technological and safe movement of unmanned vehicles along the quarry routes in a conflict environment of open pit mining.

This paper proposes a novel solution to the problem of robust ripple‐free hybrid robust regulator ensuring the elimination of intersample ripples. The paper considers a structurally stable regulation approach in the hybrid framework. A regulator is designed using this technique under periodic sampling and is applied to the nonlinear model of a quadrotor. The control of unmanned autonomous vehicles has become a research topic that is important by itself, particularly in the case of multi‐rotor systems due to their versatility in various applications. Tracking a reference signal for such vehicles can be challenging, particularly when the frequency of communication is limited, or the control station is located far away. The proposed controller is hence tested for this challenging application.

Integrated architecture for industrial robot programming and control

Unmanned Ground Vehicles (UGVs) have advantages over people in a number of different applications, ranging from sentry duty, scouting hazardous areas, convoying goods and supplies over long distances, and exploring caves and tunnels. Despite recent advances in electronics, vision, artificial intelligence, and control technologies, fully autonomous UGVs are still far from being a reality. Currently, most UGVs are fielded using tele-operation with a human in the control loop. Using tele-operations, a user controls the UGV from the relative safety and comfort of a control station and sends commands to the UGV remotely. It is difficult for the user to issue higher level commands such as patrol this corridor or move to this position while avoiding obstacles. As computer vision algorithms are implemented in hardware, the UGV can easily become partially autonomous. As Field Programmable Gate Arrays (FPGAs) become larger and more powerful, vision algorithms can run at frame rate. With the rapid development of CMOS imagers for consumer electronics, frame rate can reach as high as 200 frames per second with a small size of the region of interest. This increase in the speed of vision algorithm processing allows the UGVs to become more autonomous, as they are able to recognize and avoid obstacles in their path, track targets, or move to a recognized area. The user is able to focus on giving broad supervisory commands and goals to the UGVs, allowing the user to control multiple UGVs at once while still maintaining the convenience of working from a central base station. In this paper, we will describe a novel control system for the control of semi-autonomous UGVs. This control system combines a user interface similar to a simple tele-operation station along with a control package, including the FPGA and multiple cameras. The control package interfaces with the UGV and provides the necessary control to guide the UGV.

Virtual reality: A design tool for enhanced consideration of usability “validation elements”

In this work we introduce a learning system based on Virtual Reality and Augmented Reality for studying analytical-geometric structures that are part of the curriculum in mathematics and physics in high school classes. We believe that an immersive study environment has several advantages with respect to traditional two-dimensional environments (such as a book or the simple screen of a PC or even a tablet), such as the spatial understanding of the concepts exposed, more peripheral awareness and moreover an evident decreasing in the information dispersion phenomenon. This does not mean that our teaching proposal has to substitute the traditional approaches, but it can be seen as a robust tool to support learning. In the first phase of our research we have sought to understand which mathematical objects and which tools could have been used to enhance the teaching of mathematics, in order to demonstrate that the use of Virtual Reality and Augmented Reality techniques significantly improves the level of understanding of the mathematical subject being studied by the students. The system which provides for the integration of two machine levels, hardware and software, was subsequently tested by a representative sample of students who then provided feedback through a questionnaire.

The COVID-19 pandemic has caused a change in educational practices, with most students now attending courses remotely via video conferences. Students are less likely to actively participate in class as a result of this shift. Notably, practical lessons need both empirical and theoretical understanding, which video education falls short of providing. The current distant methods of practical education have their limitations. This research proposes a system that uses metaverse and virtual reality techniques in the classroom to address these issues. The proposed system is used to build an aviation repair simulator and test it against video instruction. Knowledge retention and acquisition tests were evaluated, and presence was explored via survey replies, to assess the efficacy of the educational process. The experimental findings indicate that the team using the above-proposed approach achieved superior scores compared to the video instruction control on both knowledge assessments.

VIRTOOL – Virtual Reality for Machine – Tool Training