Evaluation of interaction methods in an Extended Reality training environment

Evaluating (audio)visual quality and Quality of Experience (QoE) from the user’s perspective, has become a key element in optimizing users’ experiences and their quality. Traditionally, the focus lies on how multi-level quality features are perceived by a human user. The interest has however gradually expanded towards human cognitive, affective and behavioral processes that may impact on, be an element of, or be influenced by QoE, and which have been underinvestigated so far. In addition, there is a major discrepancy between the new, broadly supported and more holistic conceptualization of QoE proposed by Le Callet et al. (2012) and traditional, standardized QoE assessment. This paper explores ways to tackle this discrepancy by means of a multi-instrumental approach. More concretely, it presents results from a lab study on video quality (N=27), aimed at going beyond the dominant QoE assessment paradigm and at exploring affective aspects in relation to QoE and in relation to perceived overall quality. Four types of data were collected: ‘traditional’ QoE self-report measures were complemented with ‘alternative’, emotional state- and user engagement-related self-report measures to evaluate QoE. In addition, we collected EEG (physiological) data, gazetracking data and facial expressions (behavioral) data. The video samples used in test were longer in duration than is common in standard tests allowing us to study e.g. more realistic experience and deeper user engagement. Our findings support the claim that the traditional QoE measures need to be reconsidered and extended with additional, affective staterelated measures.

Lately, user quality of experience (QoE) during their interaction with a system is a significant factor in the assessment of most systems. However, user QoE is dependent not only on the content served to the users, but also on the performance of the service provided. This chapter describes a novel QoE layer that extends the features of classic adaptive e-learning systems in order to consider delivery performance in the adaptation process and help in providing good user perceived QoE during the learning process. An experimental study compared a classic adaptive e-learning system with one enhanced with the proposed QoE layer. The result analysis compares learner outcome, learning performance, visual quality and usability of the two systems and shows how the QoE layer brings significant benefits to user satisfaction improving the overall learning process.

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

PurposeThe United Kingdom (UK) construction industry is experiencing a significant skills shortage due to the advancements of the Fourth Industrial Revolution (4IR), leading to a high demand for reskilling. However, there is a lack of effective training frameworks for this purpose. This study aims to develop a framework to serve as a training strategy for reskilling construction professionals in the 4IR era, addressing the existing skills gap in the industry.Design/methodology/approachThis study adopts random sampling in collecting data from stakeholders in the UK construction industry through the aid of a close-ended questionnaire. The data were subjected to principal component analysis (PCA) using SPVSS V24 and confirmatory factor analysis using AMOS Graphics V28 software. The PCA was used to group the training strategies extracted from literature into meaningful constructs, which helped inform the development of the conceptual framework. Covariance-based structural equation modelling was then used to validate the framework, with model validation carried out using fit indices such as discriminant validity, standardized root mean square residual, comparative fit index, normed fit index and others.FindingsThe model revealed that the key training strategies for reskilling construction workers in the 4IR era are immersive training, experiential learning, and collaborative workforce development. The findings indicate that immersive training, particularly through technologies like virtual reality and augmented reality, plays a critical role in equipping workers with both the technical and soft skills required for the future of construction. The study recommends that 4IR-driven technologies should also be used as reskilling tools for construction workers, as this approach would allow them to learn while working.Originality/valueMany studies have advocated for the reskilling of construction workers. However, there is a notable gap in research on the training strategies for effective reskilling of construction workers in the 4IR era. This paper provides a framework for stakeholders to implement effective reskilling strategies, thereby enhancing the industry’s capability to thrive in the 4IR era.

In recent years, quality of experience (QoE) has been investigated and proved to have both influential factors on user's visual quality and perceptual quality, while the perceptual quality means user's requirement on personalized content should be acquired in optimized quality. That's to say, those segments holding user interested content such as highlights need to be allocated more network resource in a resource-limited streaming scenario. However, all the existing HTTP-based adaptive methods only focus the content-agnostic bitrate adaptation according to limited network resources or energy resource, since they ignored user perceived semantics on some important segments, which suffered less quality on the important segments than on those ordinary ones, so as to hurt the overall QoE. In this paper, we have proposed a new semantic-aware adaptation scheme for MPEG-DASH services, which decides how to preserve bandwidth and buffering time depending on content descriptors for the perceived important content to users. Further, a semantic-aware probe and adaptation (SMA-PANDA) algorithm has been implemented in a DASH client to compare with conventional bitrate adaptions. Preliminary results show that SMA-PANDA achieves better QoE and flexibility on streaming user's interested content on MPEG-DASH platform, and it also aggressively helps user interested content compete more resource to deliver high quality presentation.

IntroductionExtended reality (XR) technologies, particularly gaze‐based interaction methods, have evolved significantly in recent years to improve accessibility and reach broader user communities. While previous research has improved the simplicity and inclusivity of gaze-based choice, the adaptability of such systems ‐ particularly in terms of user comfort and fault tolerance ‐ has not yet been fully explored.MethodsIn this study, four gaze‐based interaction techniques were examined in a visual search game in virtual reality (VR). A total of 52 participants were involved. The techniques tested included selection by dwell time, confirmation by head orientation, nodding and smooth pursuit eye movements. Both subjective and objective performance measures were assessed, using the NASA‐TLX for perceived task load and time to complete the task and score for objective evaluation.ResultsSignificant differences were found between the interaction techniques in terms of NASA‐TLX dimensions, target search time and overall performance. The results indicate different levels of efficiency and intuitiveness of each method. Gender differences in interaction preferences and cognitive load were also found.DiscussionThese findings highlight the importance of personalizing gaze‐based VR interfaces to the individual user to improve accessibility, reduce cognitive load and enhance the user experience. Personalizing gaze interaction strategies can support more inclusive and effective VR systems that benefit both general and accessibility‐focused populations.

At the time of this paper, the advances in light field technology already offer 3D displays that immerse the users without the need for additional viewing devices. Despite the numerous advantages and attractive capabilities of such glasses-free 3D displays, their user interface methods are quite complicated and they are currently underwhelming when compared to conventional 2D displays, due to the fact that visual feedback can only be rendered sharply on the emission surface of light field displays. The sharp rendering of user interfaces is a necessity, as blur may hinder their fundamental functions. When it comes to 2D displays, many user interaction techniques and interfaces have been devised. Rendering a user interface on a 2D display could be done in various ways, such as rendering overlays on top of the rendered scene, or by using billboards. These are extensively used in modern video games. User interaction methods have proven their importance and added efficiency to virtual environments throughout the years. Due to their overall value and usefulness, interaction techniques develop immediately as new types of displays arise. With the recent advancements in visualization technologies, user interfaces have been redesigned for use in AR, VR and MR visualization. This includes on-screen augmentation, which enables interaction with visual content on the screen. Although light field displays contain immense potentials, only basic user interfaces have been devised thus far, including FOX (Focus Sliding surface), which grants users the option to scale and to rotate 3D objects. In this paper, we visualize the theater model on real light field displays and we test the different interactions by means of a monitor room. The theatre model is analogous to real-life theatres, where viewers may observe the theatrical presentation on the stage from various angles. The motivation to choose the theater model was the fact that light field visualization similarly allows multiple simultaneous viewers within its field of view, in which the content can be observed in an angle- dependent manner. Moreover, from the users’ perspective, the theater model is thus familiar and it provides high-quality visual feedback. Furthermore, theater stages encompass a lot of interactions, including rigging and flying systems, pulleys, rotating stages, lights, curtains etc. In order to test the different interaction methods on light field displays, a theater model depicting the virtual environment was implemented. Methods for rendering the monitor room and the results of the interactions are discussed in the paper, illustrated by images of the actual visualization on light field displays. It is shown that producing plausible results with no noticeable visual artifacts is challenging, yet possible. The scientific contributions of the paper also highlight the various novel user interfaces for future light field systems and services.

Quality of Experience (QoE) is a human centric quality evaluation method which provides the blue print of human needs, perceptions, feelings and experiences with respect to a multimedia service. In a communications ecosystem, human interaction takes place alongside technological, contextual, and business domains, thus producing a holistic view on QoE formation. In this paper, we investigate the relationship between human perceived QoE and context for burgeoning 3-dimensional (3D) audio teleconferencing services. 3D audio teleconferencing applications are customizable by generating different virtual acoustic environments (VAE), where parameters such as virtual room size and competing talker conditions can be adjusted for a particular application. The impact of different VAE characteristics on perceived QoE, however, is still unknown. In this study, four QoE factors were investigated across different VAE scenarios. It was found that a) medium-size virtual rooms produce optimal perceived QoE, b) competing talkers of mixed gender could be easily located in the virtual space, and c) competing speaker gender had no significant effect on perceived audio quality

Training for assembly simulations can be provided using a wide range of technologies from a simple computer-based training (CBT) approach to a complex virtual reality (VR)-based immersive training (IMT) approach. The CBT approach allows user interactions through traditional keyboard and mouse applications, while the IMT approach immerses the user in a virtual environment for a more realistic experience. Typically, for a particular scenario, tools and applications for each of these approaches are developed independently. Consequently, there is much duplication of data and effort, and a lack of synchronization between them. This paper focuses on an integrated approach with support from ontologies to address this problem. Ontologies provide an opportunity to capture and manage common data and map concepts from one application to another in a logical and measured manner. Methods are developed to enable knowledge in these ontologies to be used and shared in a comprehensive and effective manner between CBT and IMT tools. The key contribution of this work is that the ontologies instantiating concepts and properties for the training domain are used effectively among different training tools to deal with common and disparate characteristics between them.

Although on-device video super-resolution enables high-quality live 360-degree streaming on mobile devices, existing methods often waste energy by overlooking perceived visual quality. In this paper, we present EOS, an energy-efficient on-device super-resolution system for mobile omnidirectional video (ODV) live streaming. EOS reduces energy waste by dynamically adjusting super-resolution complexity based on the predicted visual quality of super-resolved frames. This approach raises two challenges: (1) designing an adaptive inference policy that maximizes energy savings while minimizing degradation in Quality-of-Experience (QoE), and (2) developing a method to predict visual quality under the constraints of mobile ODV live streaming. To tackle these challenges, EOS introduces EOS SR and a No-Reference Up-scaling Quality Prediction scheme. EOS SR employs a device-agnostic, scalable deep neural network optimized for mobile devices, with an energy-aware scheduler that jointly selects the optimal super-resolution model and GPU frequency. The No-Reference Upscaling Quality Prediction scheme estimates visual quality across arbitrary viewpoints in real time without requiring high-resolution reference videos. Experiments on commodity smartphones show that EOS reduces average power consumption by 34.6%–49.9% compared to baseline methods, while preserving high visual quality and frame rates.

Virtual Reality Scenario Design for Distance Learning With Google Poly

Encountered-Type Haptic Displays (ETHDs) represent a field of haptic displays with the premise of not using any type of actuator directly in contact with the user skin, thus providing an alternative integration of haptic displays in virtual environments. In this paper, we present novel interaction techniques (ITs) dedicated to ETHDs. The techniques aim at addressing the issues commonly presented for these devices such as limited contact areas, lags and unexpected collisions with the user. First, our paper proposes a design framework based on several parameters defining the interactive process between user and ETHD (input, movement control, displacement and contact). Five techniques based on different ramifications of the design space framework were conceived, respectively named: Swipe, Drag, Clutch, Bubble and Follow. Then, a use-case scenario was designed to depict the usage of these techniques on the task of touching and coloring a wide, flat surface. Finally, a user study based on the coloring task was conducted to assess the performance and user experience for each IT. Results were in favor of Drag and Clutch techniques which are based on manual surface displacement, absolute position selection and intermittent contact interaction. Taken together our results and design methodology pave the way to the design of future ITs for ETHDs in virtual environments.

Over the years, various immersive virtual training environments (iVTEs) have been developed, allowing companies to start transitioning to Virtual Reality (VR) technologies to train their personnel. This transition forces companies to start using game engines as a foundation to develop such iVTEs, which also requires a multidisciplinary team. When developing such training environments, challenges on how to present tasks to users arise. The way these tasks are presented can dictate the efficacy of the VR training application. This paper presents three different task presentation methodologies (avatar animation, videos, and instruction manual) and assesses them using 36 participants, divided into those three groups. Usability, sense of presence, satisfaction, cybersickness, and technology acceptance variables were studied and results indicated that only the total number of actions performed had differences between groups where the instruction manual reported the higher number of actions (usability) when compared to the other conditions. Therefore it was concluded that the instruction manual proved to be where users kept losing focus and making more actions. It was also concluded that all conditions had a similar sense of presence, satisfaction, cybersickness, and acceptance scores.

Virtual Reality (VR) is an artificial environment created for presented for users to interact to virtual world and to be experienced in the way that users believe and accept it such a real environment. The environment is created to be three dimensional world using computer technology which is called Virtual Environment (VE). Virtual Environment makes user experiences immersion and realize to be inside and the part of the virtual world. Therefore, Virtual Environment is very important factor for making user immersive effectively, user must be able to explore what appears in the virtual environment and be able to interact by changing perspectives seamlessly. Generally, Virtual Environment (VE) should be made to be reality as much as possible by using high resolution of 3D model to increase more information and detail to the model in the environment. However, high resolution model, high polygon, really needs high performance computing to calculate rendering process to support user's interaction in real-time. For this paper, we focus on implementing Relief Mapping technique to reduce the polygonal bas-relief built in 3D world which is applied to VR of Sino Portuguese Architecture due to the facade of buildings consist of many bas-relief pattern via game engine, Unity3D, for real-time rendering and VR walkthrough. The result of this uses a single polygon/1 bas-relief and the quality of visualization after rendering process is compatible to the conventional process.

In the immersive transformer training simulator, with inertia motion capture system, the operator's motion data was captured and analyzed. The motions capture data of hand movements and gesture recognition algorithm were studied. A method of continuous space-time tracking of the hand joints was realized using skeleton graph. The hand movement features of natural human-machine interaction were defined. A method of hand segmentation from the depth graph background using hand joints of skeleton graph was proposed. Three kinds of gesture recognition method were realized with SVM and studied about stability and robustness. Semantics of gesture, mapped to virtual environments input events. The immersive transformer training simulator was realized based on the research.