Informatics in Radiology: Intuitive User Interface for 3D Image Manipulation Using Augmented Reality and a Smartphone as a Remote Control

Three-dimensional sound's effectiveness in virtual reality (VR) environments has been widely studied. However, due to the big differences between VR and augmented reality (AR) systems in registration, calibration, perceptual difference of immersiveness, navigation, and localization, it is important to develop new approaches to seamlessly register virtual 3-D sound in AR environments and conduct studies on 3-D sound's effectiveness in AR context. In this paper, we design two experimental AR environments to study the effectiveness of 3-D sound both quantitatively and qualitatively. Two different tracking methods are applied to retrieve the 3-D position of virtual sound sources in each experiment. We examine the impacts of 3-D sound on improving depth perception and shortening task completion time. We also investigate its impacts on immersive and realistic perception, different spatial objects identification, and subjective feeling of human presence and collaboration. Our studies show that applying 3-D sound is an effective way to complement visual AR environments. It helps depth perception and task performance, and facilitates collaborations between users. Moreover, it enables a more realistic environment and more immersive feeling of being inside the AR environment by both visual and auditory means. In order to make full use of the intensity cues provided by 3-D sound, a process to scale the intensity difference of 3-D sound at different depths is designed to cater small AR environments. The user study results show that the scaled 3-D sound significantly increases the accuracy of depth judgments and shortens the searching task completion time. This method provides a necessary foundation for implementing 3-D sound in small AR environments. Our user study results also show that this process does not degrade the intuitiveness and realism of an augmented audio reality environment

Recupera algunos de los elementos menos estudiados y reconocidos en el ensayo latinoamericano: su profunda vocación de reflexión, diálogo y mediación intelectual; su plasticidad y apertura a la búsqueda del sentido; su vínculo con la sociabilidad, la conversación y la generación de un espacio simbólico de encuentro; su capa cidad de establecer una nueva retroalimentación entre lectura y escritura.

Augmented reality sickness questionnaire (ARSQ): A refined questionnaire for augmented reality environment

Augmented reality environments face significant challenges in achieving context and semantic awareness, which are relevant for enhancing user experiences. Although machine learning has been extensively applied to areas such as object and action recognition, there is a knowledge gap in the integration of these techniques to improve contextual understanding in AR environments. This study explores how machine learning enhances object and action recognition, improving context awareness in augmented reality environments. A systematic review of fifteen studies revealed advancements in deep learning, semantic web, knowledge graphs, and multimodal interfaces, which collectively improved the capabilities of augmented reality systems. The findings include innovations in aspects such as 3D hand pose estimation, semantic gaze analysis through eye tracking, and hybrid approaches that optimize real-time processing. These developments improve augmented and mixed reality environments by enabling more realistic, interactive, and responsive applications. However, persistent challenges such as latency, scalability, and environmental adaptation underscore the need for more research. These results suggest a guide for the development of highly immersive and efficient AR/MR systems, with the potential to improve user interaction and operational capabilities.

The purpose of the research is to compare the impact of students’ learning performance between an augmented reality (AR) environment and in-class environment. To create an efficient AR environment, we used Microsoft HoloLens, which is the next generation of a see-through holographic computer. We developed the AR learning module: ergonomic guidelines for manual material handling (MMH). We hypothesize that AR changes the way students understand the concepts of MMH. Our analysis includes a careful evaluation of student experimental skills during the learning activities. This new AR environment could allow students to engage hands-on training of MMH and strengthen their understanding. Student test score was used as metrics for performance assessment. We found a significant improvement on student understanding of MMH lecture after they used the AR module. The findings of this study indicated the potential benefits of using AR environment interfaces in engineering education and training.

3D imaging for on-farm estimation of live cattle traits and carcass weight prediction.

The visualization of medical images with advanced techniques, such as augmented reality and virtual reality, represent a breakthrough for medical professionals. In contrast to more traditional visualization tools lacking 3D capabilities, these systems use the three available dimensions. To visualize medical images in 3D, the anatomical areas of interest must be segmented. Currently, manual segmentation, which is the most commonly used technique, and semi-automatic approaches can be time consuming because a doctor is required, making segmentation for each individual case unfeasible. Using new technologies, such as computer vision and artificial intelligence for segmentation algorithms and augmented and virtual reality for visualization techniques implementation, we designed a complete platform to solve this problem and allow medical professionals to work more frequently with anatomical 3D models obtained from medical imaging. As a result, the Nextmed project, due to the different implemented software applications, permits the importation of digital imaging and communication on medicine (dicom) images on a secure cloud platform and the automatic segmentation of certain anatomical structures with new algorithms that improve upon the current research results. A 3D mesh of the segmented structure is then automatically generated that can be printed in 3D or visualized using both augmented and virtual reality, with the designed software systems. The Nextmed project is unique, as it covers the whole process from uploading dicom images to automatic segmentation, 3D reconstruction, 3D visualization, and manipulation using augmented and virtual reality. There are many researches about application of augmented and virtual reality for medical image 3D visualization; however, they are not automated platforms. Although some other anatomical structures can be studied, we focused on one case: a lung study. Analyzing the application of the platform to more than 1000 dicom images and studying the results with medical specialists, we concluded that the installation of this system in hospitals would provide a considerable improvement as a tool for medical image visualization.

Augmented reality (AR) can support neurosurgical procedures by integrating 3-dimensional (3D) images into real-world views to help anatomical understanding. However, current systems are cumbersome and not attractive for trainees. Studies on wearable AR devices show promise but need further refinement. This preliminary report assesses a novel augmented reality navigation system (ARnavS) using a simultaneous localization and mapping algorithm provided with the Vuforia SDK and a quick response code. The system was applied to burr-hole surgeries in 10 cases of chronic subdural hematoma to evaluate its convenience and accuracy. Ten patients with 11 hematomas underwent burr-hole surgeries using ARnavS. The system was effective without equipment issues, with an average registration time of 2.6 minutes. Discrepancy between planned and actual burr-hole locations was 3.5 mm. Skin incisions avoided 3D image of the superficial temporal artery, with no superficial temporal artery damage in any case. The mean surgical time was 37.1 minutes, and no severe postoperative bleeding occurred. Our ARnavS for burr-hole surgery in chronic subdural hematoma offers good 3D imaging with quick registration. It provided surgeons with better anatomical comprehension, helping to avoid injury to critical structures. Unlike conventional systems, ARnavS allows direct visualization of 3D graphics in the surgical field, beneficial for less-experienced neurosurgeons and medical students. While further studies are needed, the AR navigation system with simultaneous localization and mapping algorithm and quick response code showed precise intraoperative 3D imaging with sufficient accuracy after quick registration. While further studies are needed, it has demonstrated practical usefulness in clinical neurosurgical procedures.

The chemical reaction between different molecules is an important learning subject in a chemistry course. Especially for elementary school students, this can be an abstract concept and therefore difficult to understand. One way to facilitate this learning process is to use Augmented Reality (AR) technology, which is considered as an added value compared to classical learning materials such as textbooks, 2D images, video, etc. Among the different advantages of using AR technology in the context of educational domain is the fact that 3D technology is offering a safe environment especially if the students have to perform critical tasks such as simulating chemical reactions. This work investigates the students’ attitude towards the use of a mobile AR application for learning atoms and molecules reactions. In particular, we focused on female students as in general female students show less interest in science and technology than male students. We were keen to investigate whether the use of AR technology could change this attitude. The students are able to interact with the different AR components to explore the possible reactions in a 3D interface, to see the structure and shape of atoms and molecules, and to view related descriptions in their native language, i.e. the Arabic language. The mobile AR application was evaluated by a class of 12–13 years old (7th grade) students in a Palestinian primary school. The number of participants was 50, all female students. After analyzing the results, we can conclude that the female students had a positive attitude toward the use of this AR application in their learning process.

Virtual characters are now widely used in games, computer-generated (CG) movies, virtual reality (VR), and communication media. The continued technological innovations in motion capture mean that a more natural representation of a three-dimensional character’s motion should be achievable. Many researchers have investigated how virtual characters interact with their surrounding environment through spatial relationships, which were introduced for adapting and preserving character motion. However, technical problems should be resolved to enable the control of characters in augmented reality (AR) environments that combine with the real world, and this can be achieved by adapting motion to environmental differences using original motion datasets. In this paper, we investigate a novel method for preserving automatic motion adaptation for a virtual character in AR environments. We used specific object (e.g., puddle) recognition and the spatial properties of the user’s surrounding space, e.g., object types and positions, and ran validation experiments to provide accurate motion to improve the AR experience. Our experimental study showed positive results in terms of smooth motion in AR configurations. We also found that the participants using AR felt a greater sense of co-presence with the character through adapted motion.

The evolution of 3D technologies shows a cyclical learning curve with a series of hypes and dead ends, with mistakes and consequences. 3D images contain significantly more information than the corresponding 2D ones. 3D display systems should be built on more pixels, or higher speed components. For true 3D, this factor is in the order of 100x, which is a real technological challenge. If not fulfilled, the capabilities of 3D systems will be compromised: headgears will be needed, or the viewers should be positioned or tracked, single-user devices, lack of parallax, missing cues, etc. The temptation is always there: why to provide all the information, just what the person absorbs that moment (subjective or objective visualization). Virtual Reality (VR) glasses have been around for more than two decades. With the latest technical improvements, VR became the next hype. 3D immersion was added as a new phenomenon; however, VR represents an isolated experience, and still requires headgears and a controlled environment. Augmented Reality (AR) in this sense is different. Will the VR/AR hype with the headgears be a dead end? While VR headsets may sell better than smart glasses or 3D TV glasses, also consider that using the technology may require a set of behavioral changes that the majority of people do not want to make. Displays and technologies that restrict viewers, or cause any discomfort will not be accepted on the long term. The newer wave of 3D is forecasted to 2018-2020, answering the need for unaided, limitation-free 3D experience. Light Field (LF) systems represent the next-generation in 3D. The HoloVizio system, having a capacity in the order of 100x, offers natural, restrictions-free 3D experience on a full field of view, enabling collaborative use for an unlimited number of viewers, even in a wider, immersive space. As a scalable technology, the display range goes from monitor-style units, through automotive 3D HUDs, screen-less solutions, up to cinema systems, and Holografika is working on interactive large-scale immersive systems and glasses-free 3D LED walls.

The interpretation of medical images is a primary task for radiologists. Besides two-dimensional (2D) images, current imaging technologies allow for volumetric display of medical images. Whereas current radiology practice increasingly uses volumetric images, the majority of studies on medical image interpretation is conducted on 2D images. The current study aimed to gain deeper insight into the volumetric image interpretation process by examining this process in twenty radiology trainees who all completed four volumetric image cases. Two types of data were obtained concerning scroll behaviors and think-aloud data. Types of scroll behavior concerned oscillations, half runs, full runs, image manipulations, and interruptions. Think-aloud data were coded by a framework of knowledge and skills in radiology including three cognitive processes: perception, analysis, and synthesis. Relating scroll behavior to cognitive processes showed that oscillations and half runs coincided more often with analysis and synthesis than full runs, whereas full runs coincided more often with perception than oscillations and half runs. Interruptions were characterized by synthesis and image manipulations by perception. In addition, we investigated relations between cognitive processes and found an overall bottom-up way of reasoning with dynamic interactions between cognitive processes, especially between perception and analysis. In sum, our results highlight the dynamic interactions between these processes and the grounding of cognitive processes in scroll behavior. It suggests, that the types of scroll behavior are relevant to describe how radiologists interact with and manipulate volumetric images.

Real-time 3D interaction with augmented reality environments is one of the main features of any AR system. However, selecting and manipulating distal 3D virtual objects in AR still suffer from lack of accuracy and precision. In this paper, we propose an alternate 3D interaction technique called for selection and manipulation distant objects in immersive video see-through augmented reality. Zoom-in interaction technique is based on the idea of zooming the captured images. This allows bringing closer both of real and virtual distant objects, while keeping the spatial registration between the virtual and the real scenes thanks to a robust real-time computer vision algorithm for pose estimation. An evaluation and comparison with other well-known technique are given at the end of this paper, in order to validate our proposed approach.

The paper describes a technique that allows measuring and annotating real objects in an Augmented Reality (AR) environment. The technique is based on the marker tracking, and aims at enabling the user to define the three-dimensional position of points, within the AR scene, by selecting them directly on the video stream. The technique consists in projecting the points, which are directly selected on the monitor, on a virtual plane defined according to the bi-dimensional marker, which is used for the tracking. This plane can be seen as a virtual depth cue that helps the user to place these points in the desired position. The user can also move this virtual plane to place points within the whole 3D scene. By using this technique, the user can place virtual points around a real object with the aim of taking some measurements of the object, by calculating the minimum distance between the points, or in order to put some annotations on the object. Up to date, these kinds of activities can be carried out by using more complex systems or it is needed to know the shape of the real object a priori. The paper describes the functioning principles of the proposed technique and discusses the results of a testing session carried out with users to evaluate the overall precision and accuracy.

This paper presents an intuitive and natural gesture-based methodology for solid modelling in the Augmented Reality (AR) environment. The framework of Client/Server (C/S) is adopted to design the AR-based computer aided design (CAD) system. The method of creating random or constraints- based points using gesture recognition is developed to support modelling. In addition, a prototype system of product 3D solid modelling has been successfully developed, we have compared it with traditional CAD systems through several basic design modeling tasks. Finally, analysis of questionnaire feedback survey shows the intuitiveness and effectiveness of the system, and user studies demonstrate the advantage of helping accomplish the product early design and creating and manipulating the 3D model in the AR environment.