VREM: An advanced virtual environment for micro assembly

The multi-modal information presentation, integrated into the virtual environment (VE), has potential for stimulating different senses, improving the user’s impression of immersion, and increasing the amount of information that is accepted and processed by the user’s perception system. The increase of the useful feedback information may reduce the user’s cognitive load, thus enhancing the user’s efficiency and performance while interacting with VEs. This paper presents our creation of a multi-sensory virtual assembly environment (VAE) and the evaluation of the effects of multi-sensory feedback on the usability. The VAE brings together complex technologies such as constraint-based assembly simulation, optical motion tracking technology, and real-time 3D sound generation technology around a virtual reality workbench and a common software platform. The usability evaluation is in terms of its three attributes: efficiency of use, user satisfaction, and reliability. These are addressed by using task completion times (TCTs), questionnaires, and human performance error rates (HPERs), respectively. Two assembly tasks have been used to perform the experiments, using sixteen participants. The outcomes showed that the multi-sensory feedback could improve the usability. They also indicated that the integrated feedback offered better usability than either feedback used in isolation. Most participants preferred the integrated feedback to either feedback (visual or auditory) or no feedback. The participants’ comments demonstrated that nonrealistic or inappropriate feedback had negative effects on the usability, and easily made them feel frustrated. The possible reasons behind the outcomes are also analysed by using a unifying human computer interaction framework. The implications, concluded from the outcomes of this work, can serve as useful guidelines for improving VE system design and implementation.

In order to realize assembly process planning in virtual reality environment, an assembly process planning generation method based on real-virtual mapping of basic motion sequence is proposed. Based on the analysis of current assembly process content from enterprise, assembly process information is modeled; standard assembly operations and basic assembly motion are defined. The mapping matrix among standard assembly operations and basic assembly motions are set up. A method to get basic motion sequence in virtual environment during virtual assembling process by real user is put forward. A prototype system is developed based on these research results, and the system function is demonstrated through assembly process of automobile engine components. It shows that this assembly process generation method based on real-virtual mapping of motion sequence is feasible, and it also provides a new idea for the application of virtual assembly technique for product manufacturing process.KeywordsVirtual realityVirtual assemblyAssembly process planningStandard assembly operationAssembly basic motionReal-virtual mapping

This paper presents the creation of an assembly simulation environment with multisensory feedback (auditory and visual), and the evaluation of the effects of auditory and visual feedback on the task performance in the context of assembly simulation in a virtual environment (VE). This VE experimental system platform brings together complex technologies such as constraint-based assembly simulation, optical motion tracking technology, and real time 3D sound generation technology around a virtual reality workbench and a common software platform. A peg-in-a-hole and a Sener electronic box assembly task have been used as the task cases to conduct the human factor experiment, using sixteen participants. Both objective performance data (i.e., task completion time, TCT; and human performance error rate, HPER) and subjective opinions (i.e., questionnaires) on the utilization of auditory and visual feedback in a virtual assembly environment (VAE) have been gathered from the experiment. Results showed that the introduction of auditory and/or visual feedback into VAE did improve the assembly task performance. They also indicated that integrated feedback (auditory plus visual) offered better assembly task performance than either feedback used in isolation. Most participants preferred integrated feedback to either individual feedback (auditory or visual) or no feedback. The participants' comments demonstrated that nonrealistic or inappropriate feedback had a negative effect on the task performance, and easily made them frustrated.

This paper outlines an IoT based collaborative framework which provides a foundation for cyber physical interactions and collaborations for advanced manufacturing domains; the domain of interest is the assembly of micro devices. The design of this collaborative framework is discussed in the context of IoT networks, cloud computing as well as the emerging Next Internet which is the focus of recent initiatives in the US, EU and other countries. A discussion of some of the key cyber physical resources and modules is outlined followed by a discussion of the implementation and validation of this framework.

Navigation is playing an increasingly important role in virtual environments (VE). Today virtual worlds are very large and present challenging navigation tasks. Usually a VE contains few visual details and hence few landmarks to support navigation. In this paper, we propose a technique to automatic identify landmarks in large virtual environment using spatial data mining techniques. Identified landmarks are used to enrich landmark knowledge which is essential for navigation in VE. Initially three datasets with different number of objects have been generated based on a realistic distribution of buildings on one of newest and well designed Egyptian cities. A new model for extracting landmarks in VE was developed based on the models used in real environment. The proposed technique was applied on the generated cities and successfully identified the landmarks.

The current study concerns the identification of possible differences in perception between the virtual and the real world in terms of the effect on brain activity. For this reason, an EEG device was used to capture participants' brain activity in different brain areas during their exposure to different virtual and real environments. The environments considered in this study portray a classroom environment with a scenario suitable for teacher training and professional development. The first aim of the experiment is to investigate if exposure to a virtual environment can affect motor, cognitive or other function of the users, and the second aim is to test if the graphics content and nature of such an environment can influence the user experience. During the study, the optimum duration of exposure in a virtual environment was also assessed by measuring the time that the brain needs to perceive and adapt to the new state. Our results, consisting of EEG data analyzed in 10 Regions of Interest (ROIs) and responses from an Igroup Presence questionnaire, indicated a significant difference in each brain area, especially in the frontal and occipital region, when a participant was exposed to a non-realistic virtual environment, compared to a realistic one, highlighting the impact of the selected virtual environment design. The results of the experiment can play an important role in defining the characteristics of optimal virtual environments for virtual reality-based training applications.

Virtual reality (VR) collaborative manipulation requires a proper understanding of the user’s self, other collaborators, and shared objects in the virtual environment. Users must be aware of the movements of themselves and others to coordinate their actions when they hand over an object or move it together. The awareness of collaboration plays an essential role in virtual environments. However, previous studies have focused on providing awareness only for cooperative manipulation. This study proposes a novel framework for dual awareness that comprehensively manages the state of interactions between multiple users and shared virtual objects and provides distinct sensory cues to distinguish themselves from collaborators. The experimental results showed that participants could easily and quickly collaborate on manipulation using our feedback, and they also answered positively in the subjective evaluation.

Virtual fire drill system supporting co-located collaboration

Virtual reality environments are increasingly being used for upper limb rehabilitation in poststroke patients. Our goal was to determine if arm reaching movements made in a 2-D video-capture virtual reality environment are similar to those made in a comparable physical environment. We compared arm and trunk kinematics for reaches made with the right, dominant arm to three targets (14 trials per target) in both environments by 16 adults with right poststroke hemiparesis and by eight healthy age-matched controls. Movement kinematics were recorded with a three-camera optoelectronic system at 100 samples/s. Reaching movements made by both control and stroke subjects were affected by viewing the targets in the video-capture 2-D virtual environment. Movements were slower, shorter, less straight, less accurate and involved smaller ranges of shoulder and elbow joint excursions for target reaches in the virtual environment compared to the physical environment in all subjects. Thus, there was a decrease in the overall movement quality for movements made in the 2-D virtual environment. This suggests that 2-D video-capture virtual reality environments should be used with caution when the goal of the rehabilitation program is to improve the quality of movement patterns of the upper limb.

Kinematics of Pointing Movements Made in a Virtual Versus a Physical 3-Dimensional Environment in Healthy and Stroke Subjects

The metaverse seamlessly integrates physical and digital spaces, enabling AI-driven innovations in virtual interactions, autonomous avatars, and real-time experiences. However, increased reliance on AI brings sweeping cybersecurity challenges, such as adversarial attacks, deep fake impersonation, and AI-driven phishing campaigns. The security of the metaverse is vital for the sustainability of user trust and system integrity. As AI assumes a larger role in virtual environments, proactive cybersecurity measures must be taken to counter emerging threats. This paper introduces DAI-TIRS, a holistic security framework designed to proactively secure the metaverse. DAI-TIRS is the integration of machine learning-based anomaly detection, dynamic honeypots, and predictive threat modeling that detect, classify, and mitigate AI-driven threats in real-time. By utilizing MITRE ATT&CK and the PyTM framework, it constantly learns new emerging threats through advanced behavioral analytics and keeps pace with the adversarial AI model’s evolution. The experimental results from a simulated metaverse environment demonstrate that DAI-TIRS achieves 93% accuracy in threat detection, 90% precision in classifying the severity, and a 36.9% faster threat mitigation response time than the average performance of baseline models, as detailed in the paper. These findings underscore the critical need for adaptive AI-based cybersecurity solutions that will enhance the resilience, trust, and integrity of metaverse ecosystems. This research establishes DAI-TIRS as an advanced cybersecurity framework that has demonstrated its adaptability and effectiveness in countering AI-driven threats across multiple sectors. The source code for DAI-TIRS is available on GitHub: [https://github.com/sharmamohini762/DAI-TIRS-Code]

Virtual hand interactions play key roles in virtual environments. The recent addition of force feedback to virtual reality simulations has enhanced their realism, especially when dexterous manipulation of virtual objects is concerned. In the past decades, much effort has been made on virtual hand modeling from the perspectives of computer animation and human computer interaction. However, much less attention is paid on haptic modeling of flexible virtual hand. In this paper, we propose a 4-layer flexible virtual hand model for virtual hand haptic interaction. The skin layer, kinematics layer, collision detection layer and haptic layer are integrated into a sophisticated virtual hand to simulate the human hands natural anatomy in its appearance and motion, and to reflect the area contact feature of force feedback datagloves. The infrastructure and details of the flexible virtual hand model are discussed. Experimental results show that the proposed flexible virtual hand demonstrates good performance in virtual hand haptic applications.

This paper deals with a method for presenting reaction forces in virtual assembly environments, which is used for instructing or training techniques of assembly works. There is an important demand for the reaction force on stress of parts in the virtual assembly environment. In this paper, the C-Space (Configuration Space) representation and the hierarchical processing are proposed for satisfying the demand. The C-Space can represent all stress relations of parts clearly, and can indicate the current location immediately by making the space divided into some zones of geometrically important states for works. The hierarchical processing is realized with two-levels layer and reduces the calculation delay for controlling the reaction force. The qualitative level processing recognizes a task by kinematic reasoning with the C-Space, and the quantitative level processing calculates the reaction force by task based reaction models and physical data. A case study with two experiments shows that the method is effective for presenting reaction forces on a screw tightening work.

Virtual reality (VR) is a technology which is often regarded as a natural extension to 3D computer graphics with advanced input and output devices. One of the benefits that education reaps from VR is the ability to visualize, simulate and interact with virtual objects. This ability allows the student to have the trial and error session before practicing it on real things. The integration of this new technology with software systems for education, design and manufacturing will provide a new boost to the field of computer-aided design. One aspect of design and manufacturing which may be significantly affected by VR is design for assembly. The development of a realistic virtual assembly environment is challenging because of the complexity of the physical processes and the limitation of available VR technology. Many research activities in this domain primarily focused on particular aspects of the assembly task such as the feasibility of assembly operations in terms of interference between the manipulated parts. The virtual assembly environment proposed in this research is on mechanical part assembly of Arrick Robot.

This paper presents our creation and evaluation of multi-modal interface for a virtual assembly environment. It involves implementing an assembly simulation environment with multi-sensory feedback (visual and auditory), and evaluating the effects of multi-modal feedback on assembly task performance. This virtual environment experimental platform brought together complex technologies such as constraint-based assembly simulation, optical motion tracking technology, and real-time 3D sound generation technology around a virtual reality workbench and a common software platform. A peg-in-a-hole and a Sener electronic box assembly tasks have been used as the task cases to perform human factor experiments, using sixteen subjects. Both objective performance data (task completion time, and human performance error rates) and subjective opinions (questionnaires) have been gathered from this experiment.