Investigating Resolution Strategies for Workspace-Occlusion in Augmented Virtuality

The increasing demand for intelligent and sustainable energy management within higher education institutions has encouraged the adoption of IoT-based solutions; however, traditional IoT dashboards typically rely on text-based device lists and non-intuitive identifiers that lack spatial context. As a result, users often struggle to understand which physical devices they are controlling, leading to confusion, poor user experience, and a higher risk of operational errors when managing smart campus facilities. This study aims to develop and validate a Digital Twin–based Smart Campus system capable of synchronizing physical electronic devices with an interactive 3D virtual environment in real time, providing a spatially accurate digital representation of the lecturer room that mirrors the real-world layout. The research employs a systematic workflow that includes problem identification, literature analysis, installation of IoT devices such as Zigbee smart switches and ESP32 IR blasters, creation of a web-based Digital Twin interface, and development of optimized 3D room models using Blender. System testing was conducted to evaluate physical-to-digital and digital-to-physical synchronization performance, and FPS benchmarking was performed to assess usability across high-end, mid-range, and entry-level devices. The results show that the Digital Twin maintains 100% synchronization accuracy with millisecond-level response times and runs smoothly on diverse hardware. By enabling users to interact with devices directly through a virtual environment that visually matches the real room, the system reduces operational mistakes, improves user experience, and enhances awareness of energy usage. The study concludes that the proposed Digital Twin approach effectively overcomes key limitations of traditional IoT dashboards and offers a scalable, practical framework for Smart Campus implementations.

We propose Imaginary Devices, a set of freehand gestures that mimic the use of physical input devices. Imaginary Devices allow users to choose the input modality best suited for the task at hand, such as a steering wheel for a driving game or a joystick for a flight simulator. Exploiting the skills that users have acquired using physical input devices, they can instantly begin interacting with an Imaginary Device. Since no physical device is involved, users can switch quickly and effortlessly among a number of devices.We demonstrate the potential of Imaginary Devices with Grand Theft Auto, a game that requires players to change between roles often and quickly, and we examine the viability of the concept in two user studies. In the first study, we found that participants produced a wide range of postures to represent each device but all were able to reproduce the correct posture after a short demonstration. In the second study, we found that Imaginary Devices afford precise input control and approach the baseline performance set by physical devices.

The time of the Covid-19 pandemic stopped the theatre process and opened the way for new forms and techniques in the virtual environment, updating the concept of e-theatre. The research of these processes from an aesthetic point of view has also stimulated the development of a new discourse in Latvian theatre science, analyzing the technical solution of virtual or e-productions, the impact on viewers, as well as other aspects related to theatrical manifestations in the virtual environment. The aim of the article is to provide an overview of the aesthetic strategies and means of expression in Latvian e-theatre, researching this process from a semiotic approach (performance signs and sign systems in the virtual environment), as well as a phenomenological take (the impact of e-performances on the spectator). In the spring of 2020, soon after the Covid-19 restrictions, virtual experiments quickly developed in Latvian theatre. In the context of the global pandemic caused by Covid-19, we can talk about a complete transition of the theatre to a virtual space, forcibly changing the forms of performances to digital formats that do not require viewers and actors to be in the same room. At the same time, this forced transition of the theatre to the digital environment is connected with other risks, such as the natural competition between e-theatre and other virtual content on the internet. The author of the article offers an in-depth analysis of three different Latvian e-theatre performances, offering an overlook of the trends of digital theatre.

Despite the fact that dematerialization of all our multimedia content brings advantages, the absence of physical medium brings also a lack of tangibility. This paper presents a gesture-based interface to bring back tangibility in our dematerialized multimedia content. Our concept is to apply the drag-and-drop paradigm in the real world, using a remote control as a portal between virtual content and physical devices. In order to validate our concept, we designed a prototype focused on the transfer of digital pictures between two different devices: from a TV screen to a digital photo frame. User experiment has proven that our concept brings efficiency, easiness and a natural way to interact with digital multimedia content.

Background/Objectives: Virtual reality content is changing into a form of pursuing experience and convenience as it gradually emphasizes user interaction, and user-centric designs and interfaces like UI/UX are performing important roles in creating a new communication and interaction. Thus, when designing content for a virtual environment, a design environment that allows users to focus within the virtual space, toward the cognitive factor, must be created by means of user comprehension and experience. Methods/Statistical Analysis: Current virtual reality content requires adaptive learning in terms of new device control, with various devices and interaction functions. Hence, studies, in accordance with the cognitive process from virtual reality, are necessary in order to achieve an effective experience based on virtual reality content, because user needs in regards to comprehension and convenience toward new environments are being generated. The present study analyzes virtual reality contents based on cognitive affordance theory through information and behavior detected by visual perception in order to achieve efficiency for convenient environments, utility, and user participation expansion, which is achieved by restricting contents to those only of the CAVE system type. Findings: The sub-elements of cognitive affordance theory are divided into perception, function, and behavior, and the data analyzing the correlation in accordance with each type of virtual reality content establishes a theory for an efficient virtual reality content usage plan, presenting a utilization model. Research on contents based on cognitive affordance has been positively affected by content understanding and focus, affected from the perception factor out of each cognitive affordance element. Application/Improvements: Such results show that more convenient environments for users will be built and content satisfaction will be increased, and may be presented as an efficient utilization plan for virtual reality content development.

The need for training in dangerous situations in all levels of an educational system through secure environments leads to the implementation of suitable Educational Virtual Environments (EVEs). Participants' sense of presence is a factor of major importance. This work is an evaluation of the factors that contribute to the sense of presence in an EVE in earthquake situations. We have studied the factors of mental transfer, the virtual environment's realism and the contribution of sensory stimuli. The results showed that the sense of presence depends on the attribution of virtual environment's content and on its enrichment with external sensory stimuli.

In many virtual reality systems, user physical workspace is superposed with a particular area in a virtual environment. The spatial consistency between the real and virtual interactive space allows users to take advantage of physical workspace to walk and to interact intuitively with the real and virtual contents. To maintain such spatial consistency, application designers usually deactivate user virtual navigation capability. This limits user reachable virtual area, and segments the spatial consistency required sub-task from a continuous scenario mixing large scale navigation. In order to provide users with a continuous virtual experience, we introduce two switch techniques to help users to recover the spatial consistency in some predefined virtual areas with teleportation navigation: simple switch and improved switch. We conducted a user study with a box-opening task in a CAVE-like system to evaluate the performance and usability of these techniques under different conditions. The results highlight that assisting the user on switching back to a spatially consistent situation ensures entire workspace accessibility and decreases time and cognitive effort used to complete the sub-task. The simple switch results in less task completion time, less cognitive load, and is globally preferred by users. With additional visual feedback of user switch destination, the improved switch seems to provide the user with a better understanding of the resulting spatial configuration of the switch.

Traditional textile technologies are part of cyber-culture in all its complexity but the solutions only record the technological past and do not influence the personal development of underprivileged groups. The digital information development should mark changes not only in communication but also in education and human communities' dynamic. E-learning programs in cyber-space are undoubtedly agents of cultural change but they should be created for an immediate impact on communities and for stressing issues like alternatives for unemployed people and lifelong learning for a greater mobility. This great power of the Web to reunite people and shape their future is conditioned by the involvement and decision making of individuals. For an interactive e-learning system we have to use real time simulation social games that allow the creation of ideal spaces for the development of a learning-by-playing virtual environment, focused on skills and technologies. Also, augmented reality applications are important in developing intuitive interaction between the user and the virtual content. Using video games tactile devices or TUNE technology could enhance the real time involvement in the exploration of the "memories" of physical artifacts and technologies. "The Maps of Time" is a project developed by a team of artists and researchers from the National University of Art, Bucharest. We are building a structure for the immersive experimentation of ancient technologies in the virtual world with the purpose of preserving and revitalizing ancient traditional technologies bring them into contemporary context and present them as viable and tested solutions in lifelong education. The e-learning aspect of the project focuses on local community empowerment using visual support, expert online lessons and advice for learning traditional technologies (textile, ceramic or metal technologies). Although the e-learning sessions involve a small community in Oltenia region (south of Romania), especially school children, we are aiming for a larger audience, from experts to individuals that want to develop artistic skills. Interactive media will be used to grow awareness on sustainable traditional technologies. Digitalizing ancient technologies operational phases not only helps children discover talents but could also offer alternatives for unemployed people and be a method of active recreation. Learning technologies, creating objects to populate habitats, involving other people develops an interactive virtual community in which everyone can learn skills via information-knowledge gaining, specialist training, artist and designer involvement.

Thinning trainer based on forest-growth model, virtual reality and computer-aided virtual environment

Most advanced immersive devices provide collaborative environment within several users have their distinct head-tracked stereoscopic point of view. Combining with common used interactive features such as voice and gesture recognition, 3D mouse, haptic feedback, and spatialized audio rendering, these environments should faithfully reproduce a real context. However, even if many studies have been carried out on multimodal systems, we are far to definitively solve the issue of multimodal fusion, which consists in merging multimodal events coming from users and devices, into interpretable commands performed by the application. Multimodality and collaboration was often studied separately, despite of the fact that these two aspects share interesting similarities. We discuss how we address this problem, thought the design and implementation of a supervisor that is able to deal with both multimodal fusion and collaborative aspects. The aim of this supervisor is to ensure the merge of user’s input from virtual reality devices in order to control immersive multi-user applications. We deal with this problem according to a practical point of view, because the main requirements of this supervisor was defined according to a industrial task proposed by our automotive partner, that as to be performed with multimodal and collaborative interactions in a co-located multi-user environment. In this task, two co-located workers of a virtual assembly chain has to cooperate to insert a seat into the bodywork of a car, using haptic devices to feel collision and to manipulate objects, combining speech recognition and two hands gesture recognition as multimodal instructions. Besides the architectural aspect of this supervisor, we described how we ensure the modularity of our solution that could apply on different virtual reality platforms, interactive contexts and virtual contents. A virtual context observer included in this supervisor in was especially designed to be independent to the content of the virtual scene of targeted application, and is use to report high-level interactive and collaborative events. This context observer allows the supervisor to merge these interactive and collaborative events, but is also used to deal with new issues coming from our observation of two co-located users in an immersive device performing this assembly task. We highlight the fact that when speech recognition features are provided to the two users, it is required to automatically detect according to the interactive context, whether the vocal instructions must be translated into commands that have to be performed by the machine, or whether they take a part of the natural communication necessary for collaboration. Information coming from this context observer that indicates a user is looking at its collaborator, is important to detect if the user is talking to its partner. Moreover, as the users are physically co-localised and head-tracking is used to provide high fidelity stereoscopic rendering, and natural walking navigation in the virtual scene, we have to deals with collision and screen occlusion between the co-located users in the physical work space. Working area and focus of each user, computed and reported by the context observer is necessary to prevent or avoid these situations.

In most applications of virtual environments (VEs), like training and design evaluation, a good sense of orientation is needed in the VE. Orientation performance when moving around in the real world relies on visual as well as proprioceptive feedback. However, the navigation metaphors which are used to move around the VE often lack proprioceptive feedback. Furthermore, the visual feedback in a VE is often relatively poor compared to the visual feedback available in the real world. Therefore, we have quantified the influence of visual and proprioceptive feedback on orientation performance in VEs. Subjects were immersed in a virtual forest and were asked to turn specific angles using three navigation metaphors, differing in the kind of proprioceptive feedback which is provided (no proprioceptive feedback, vestibular feedback, and vestibular and kinesthetic feedback). The results indicate that the most accurate turn performance is found when kinesthetic feedback is present, in a condition where subjects use their legs to turn around. This indicates that incorporating this kind of feedback in navigation metaphors is quite beneficial. Orientation on only the visual component is most inaccurate, leading to progressively larger undershoots for larger angles.

AI's next big challenge is to master the cognitive abilities needed by intelligent agents that perform actions. Such agents may be physical devices such as robots, or they may act in simulated or virtual environments through graphic animation or electronic web transactions. This book is about integrating and automating these essential cognitive abilities: planning what actions to undertake and under what conditions, acting (choosing what steps to execute, deciding how and when to execute them, monitoring their execution, and reacting to events), and learning about ways to act and plan. This comprehensive, coherent synthesis covers a range of state-of-the-art approaches and models –deterministic, probabilistic (including MDP and reinforcement learning), hierarchical, nondeterministic, temporal, spatial, and LLMs –and applications in robotics. The insights it provides into important techniques and research challenges will make it invaluable to researchers and practitioners in AI, robotics, cognitive science, and autonomous and interactive systems.

Brain-computer interface (BCI) devices are designed to bypass neuromuscular pathways enabling an individual to operate an external device using neural activity instead of motor activity. The use of BCIs in robotics control has important applications of assisting patients with severe motor disabilities, such as exploration, home assistance, and control of an electric wheelchair. This paper presents a development of a practical implementation of a BCI by utilizing low-cost technologies and asynchronous signal processing techniques for EEG signal acquisition and processing to navigate a quadcopter. A wireless EEG headset was used to acquire the occipital alpha and the frontal muscular artifacts from a user along with the gyroscope signals from the headset. These signals were transmitted wirelessly to a PC for further processing. Signal-processing algorithms were developed and implemented in the PC to extract the patterns from the acquired signals. These patterns, representing the intentions of the user, were then used to control a quadcopter. The design was successfully tested in a virtual environment and a physical device. The results from this preliminary pilot study proved that the design could be used with high accuracy to control a robotic device.

The design of virtual environments (VEs) can strongly influence users’ emotions. These VEs are also an important aspect of immersive Virtual Reality (VR) exergames – training system that can inspire athletes to train in a highly motivated way and achieve a higher training intensity. VR-based training and rehabilitation systems can increase a user’s motivation to train and to repeat physical exercises. The surrounding VE can potentially predominantly influence users’ motivation and hence potentially even physical performance. Besides providing potentially motivating environments, physical training can be enhanced by gamification. However, it is unclear whether the surrounding VE of a VR-based physical training system influences the effectiveness of gamification. We investigate whether an emotional positive or emotional negative design influences the sport performance and interacts with the positive effects of gamification. In a user study, we immerse participants in VEs following either an emotional positive, neutral, or negative design and measure the duration the participants can hold a static strength-endurance exercise. The study targeted the investigation of the effects of (1) emotional VE design as well as the (2) presence and absence of gamification. We did not observe significant differences in the performance of the participants independent of the conditions of VE design or gamification. Gamification caused a dominating effect on emotion and motivation over the emotional design of the VEs, thus indicating an overall positive impact. The emotional design influenced the participants’ intrinsic motivation but caused mixed results with respect to emotion. Overall, our results indicate the importance of using gamification and support the commonly used emotional positive VEs for physical training. They further indicate that the design space could also include other emotional directions of VE design.

People often rely on workspace awareness to collaborate with one another, especially in learning and problem-solving tasks. Workspace awareness is defined as an understanding of a current state of the work environment and collaborators’ activities. However, workspace awareness is difficult to maintain in virtual environments (VE), since some information from the real world is missing. Fortunately, VEs are flexible and can be designed to compensate for a lack of real world cues. This thesis aims to improve collaborative workspace awareness through enhancing the design of VEs. Our proposed approaches improve workspace awareness in three ways. The first approach examines space configurations that could improve awareness in a collaboration. The second explores different strategies for updating the visual environment so that users are aware of their collaborators’ actions. The third focuses on finding visual cues that raise the user’s awareness when collaborating with multiple users. Prior research studies suggested that users operate in workspace to manage their attention and divide their tasks when working in the real world. The space within the users’ reach is often assigned as a private space and the space out of users’ reach is assigned as a group space. Following this approach, we propose to set up a virtual space in a desktop virtual reality environment by assigning the space in front of the desktop screen as a private space and the space behind the screen as a group space. According to our user study, this setup improves collaborations between users. To gain further insight into information sharing strategies in VEs, a user study was conducted to compare different strategies for updating the VEs in complex remote problem-solving tasks. From the study, real-time updating is found to be not suitable for workspace awareness, especially when users are performing different tasks. On the other hand, an updating strategy that updates only information that is relevant to the users’ actions performs the best. Moreover, users change collaboration strategies depending on the update timing. Design guidelines are proposed from the user study as a direction for different collaborative situations. While there are many research works that allow users to work together in one-to-one collaborations, less attention has been paid to one-to-many collaborations. This is due to the lack of visual awareness cues that allow users to spectate multiple users. To alleviate this limitation, a prototype system named ObserVAR is proposed. ObserVAR utilizes Augmented Reality (AR) technology to help an instructor in spectating multiple Virtual Reality (VR) users by augmenting the instructor’s environment with virtual awareness cues generated from the VR users. In the development of ObserVAR, different awareness cues are compared in a user study to determine its suitability under various circumstances. The virtual awareness cues are then optimized further using techniques derived from graph visualization to reduce visual clutter. Finally, the knowledge gained from improving workspace awareness using different approaches is used to formulate a collaborative framework for learning and problem-solving tasks. The contributions of this thesis include an enhanced design of desktop virtual reality for collaborations; multiple improvements in workspace awareness across different aspects; the design and development of the ObserVAR system for one-to-many collaborations; and a framework for collaborative problem-solving that utilizes the proposed designs. This thesis explores various enhancements of VEs, which lead to a better performance in collaborative learning and problem-solving tasks. The VE designs in this thesis can be used as guidelines for future collaborative problem-solving systems.