Mobile Devices for Virtual Reality Interaction. A Survey of Techniques and Metaphors

Nowadays a wide range of input devices are available to users of technical systems. Especially modern alternative interaction devices, which are known from game consoles etc., provide a more natural way of interaction. But the support in computer programs is currently a big challenge, because a high effort is to invest for developing an application that supports such alternative input devices. For this fact we made a concept for an interaction system, which supports the use of alternative interaction devices. The interaction-system consists as central element a server, which provides a simple access interface for application to support such devices. It is also possible to address an abstract device by its properties and the interaction-system overtakes the converting from a concrete device. For realizing this idea, we also defined a taxonomy for classifying interaction devices by its interaction method and in dependence to the required interaction results, like recognized gestures. Later, by using this system, it is generally possible to develop a user-centered system by integrating this interaction-system, because an adequate integration of alternative interaction devices provides a more natural and easy to understand form of interaction.

This paper introduces iFringe, a mobile application that attempts to incorporate the resource heavy fringe analysis algorithms into the smart mobile devices platform. This first step taken towards mobility in the optical processing field aims to become a catalyst for modernization of various aspects of the field as well as to diversify developments to other applications. Predominantly, the motivation of this work stems from the vastly indifferent human interactive method of mobile devices, which enable images displayed on its touch screen to be manipulated in ways that could enhance the fringe analysis experience. Furthermore, given its hardware compatibility to the conventional fringe projection system, these mobile devices could potentially serve as a much more compact replacement. However, one imperative weakness that mobile devices pose is its limited computing ability. Therefore, to examine the feasibility of incorporating the fringe analysis algorithms into a mobile platform, we have implemented two fundamental fringe analysis techniques, namely the Fourier transform fringe analysis method and the phase-shifting technique. Formulas and processing procedures such as discrete Fourier transform (DFT) and quality-guided phase unwrapping, were included in accordance to their original algorithms to test their performance and usability on a smart mobile device. Details of the implementation and the performance results will also be presented in this paper to demonstrate the practicality of these algorithms on the smart mobile device platform.

The objective of this research project is to develop a user-friendly and cost-effective interactive input device that allows intuitive and efficient manipulation of 3D objects (6 DoF) in virtual reality (VR) visualization environments with flat projections walls. During this project, it was planned to develop an extended version of a laser pointer with multiple laser beams arranged in specific patterns. Using stationary cameras observing projections of these patterns from behind the screens, it is planned to develop an algorithm for reconstruction of the emitter’s absolute position and orientation in space. Laser pointer concept is an intuitive way of interaction that would provide user with a familiar, mobile and efficient navigation though a 3D environment. In order to navigate in a 3D world, it is required to know the absolute position (x, y and z position) and orientation (roll, pitch and yaw angles) of the device, a total of 6 degrees of freedom (DoF). Ordinary laser-based pointers when captured on a flat surface with a video camera system and then processed, will only provide x and y coordinates effectively reducing available input to 2 DoF only. In order to overcome this problem, an additional set of multiple (invisible) laser pointers should be used in the pointing device. These laser pointers should be arranged in a way that the projection of their rays will form one fixed dot pattern when intersected with the flat surface of projection screens. Images of such a pattern will be captured via a real-time camera-based system and then processed using mathematical re-projection algorithms. This would allow the reconstruction of the full absolute 3D pose (6 DoF) of the input device. Additionally, multi-user or collaborative work should be supported by the system, would allow several users to interact with a virtual environment at the same time. Possibilities to port processing algorithms into embedded processors or FPGAs will be investigated during this project as well.

This research study is an attempt to investigate and recommend the applications and usefulness of social media as a knowledge sharing platform using smart mobile devices and students’ academic performances. Research literature reveals that further examination of this idea is a must as it is increasingly becoming an integral part and a current mandate for digital-era learners. This study is exploratory based on learners’ data which includes students from foundation, junior and senior courses at a university. The objective was to cognize learners’ perceptions on collaborative learning while using applications of social media knowledge platform and smart mobile devices. Structural Model approach is followed to validate the data and undertake the required analysis. The resulting analysis shows that a significantly positive relation exists between the available platforms of knowledge applications using smart devices and its substantial impact on students’ overall performances. In addition, it could be observed that generally students have positive and valuable perception on the use of social media knowledge platform and smart mobile devices in collaborative learning settings. Moreover, such settings encourage students-students’ interactions, studentsteachers’ interactions, and the growing trends of students to online experts’ interactions. Furthermore, it was noticed that these interactions improve students’ engagement with such platforms using smart devices, which consequently has significant positive impact on students’ academic performances. The findings propose the inculcation in students the positive role of the use of social media knowledge platforms using smart devices so to encourage collaborative learning, to enhance active learning, to be more dynamic & engaging, to broaden their domain-knowledge, and to promote academic researchoriented learning, which are all crucial for graduate employability.

There has been an explosion of smart mobile devices over the last few years. These smart devices, such as smartphones and tablets, have changed many aspects of modern life. They have also enabled whole new industries to grow up that develop and manufacture of the smart devices. These companion products, however, are mostly built around the smart devices, instead of being tightly integrated into them, and fail to utilize the full capabilities of the smart devices. The main cause for their failure is that the software platforms these smart devices are built on are not open. This hinders efforts by device manufacturers or software developers to create innovative new products or product categories based on those software platforms.In this talk, we present Tizen (www.tizen.org), which is a fully open software platform for embedded systems. Tizen allows for everyone involved in building and using devices built on it to freely define, invent, add new features or business models, or create new device categories. Tizen offers an industry leading HTML5-based application APIs, the preferred development environment for apps and services for the future. The HTML5-based APIs make it easy for developers to create applications that run across various categories of devices such as mobile, in-vehicle infotainment (IVI), Digital TV, netbooks, health and medical devices, etc. In this talk we also present Tizen's optimization technologies that enable HTML5-based applications to enjoy performance comparable to that of native applications. We also describe how Tizen balances the trade-offs between performance and power consumption, which is of extreme importance for mobile devices.

Workstations require use of the hands both for text entry and for cursor-positioning or menu-selection. The physical arrangement does not allow these two tasks to be done concurrently. To remove this restriction, various alternative input devices have been investigated. This work focuses on the class of foot-operated computer input devices, called moles here. Appropriate topologies for foot movement are identified, and several designs for realising them are discussed.

Recent advances in smart mobile devices (e.g., smartphones and tablets) encourage researchers to utilize them in collaborative environments as a medium of interaction with large shared interactive wall displays. Such collaborative setups have additional advantages over the previous ones, e.g., users' freedom to move around the environment, full view of the whole screen, etc. In this work, we focus on finding different possible collaborative position patterns for pairs of users working in a collaborative setup, equipped with a shared interactive tiled-wall display and multiple mobile devices. For this, we observed users in a controlled study with 36 participants, who performed the collaborative test in 18 pairs.

Augmented and virtual reality proved to be valuable solutions to convey contents in a more appealing and interactive way. Given the improvement of mobile and smart devices in terms of both usability and computational power, contents can be easily conveyed with a realism level never reached in the past. Despite the tremendous number of researches related with the presentation of new fascinating applications of ancient goods and artifacts augmentation, few papers are focusing on the real effect these tools have on learning. Within the framework of SmartMarca project, this chapter focuses on assessing the potential of AR/VR applications specifically designed for cultural heritage. Tests have been conducted on classrooms of teenagers to whom different learning approaches served as an evaluation method about the effectiveness of using these technologies for the education process. The chapter argues on the necessity of developing new tools to enable users to become producers of contents of AR/VR experiences.

Falls are a major health concern leading cause of fatal and non-fatal injuries for neurological disorders. Balance dysfunction is one of the common factors to determine fall risk in neurological patients. Preventative measures may help to reduce the incidence and severity of falls for detecting balance function and fall risk factors. However, the objective measures for balance require expensive equipment with the assessment of clinical expertise. A main gap remains in the evaluation method to objectively characterize the balance functions in individuals with high risk of falling. With the development of wearable and mobile devices, recent advances in smart mobile devices may provide a potential opportunity to manage the gap in the detailed quantification of balance impairments. The purpose of this study is to identify whether the biomechanical data measured by the mobile device is reliable to characterize the posture stability in various balance test conditions. A total of 39 children with Down syndrome completed four balance-testing tasks under altered base of support and vision. Simultaneous biomechanical measurements were gathered from the iPod and force plate analysis system during functional balance testing. The force plate and mobile system provided similar patterns of stability across groups. Correlation ([Formula: see text] between two systems for path length, 95% ellipse area, peak-to-peak, standard deviation and mean ranged from 0.60 to 0.99. We expect that the smart mobile device can provide reliable and accurate information to quantify the postural stability in individuals with elderly people or neurological disorders. The objectivity, portability and easy use of such mobile device make it ideal to apply in clinical environments for detecting balance functions and reducing the risk of falls in Down syndrome or other neurological patients.

Autistic people often face challenges in face-to-face social interactions, especially in unfamiliar or high-stakes situations. Virtual reality has been proposed as a tool to support social connection by offering a more controlled and accessible environment. While previous research has focused on virtual reality-based social skills training, little is known about how autistic people experience social interactions in virtual reality. We interviewed 22 autistic young people (aged 9-22 years, MAge = 14.5) after they engaged in a social interaction with a stranger in immersive virtual reality. Using reflexive thematic analysis, we found that participants generally found virtual reality interactions tolerable, engaging and helpful as a low-pressure first step towards more challenging social situations. Virtual reality was valued for reducing sensory overload, increasing confidence and offering novel social opportunities. However, participants did not view virtual reality as a replacement for in-person interaction, which they saw as essential for building meaningful relationships. These findings highlight the potential of virtual reality to support autistic people by modifying the social environment rather than demanding behavioural change. They also challenge deficit-based assumptions about autistic social motivation, emphasising the importance of aligning technological supports with autistic people's lived experiences and preferences.Lay abstractAutistic young people can find social situations stressful and overwhelming, particularly if meeting strangers in high-stakes settings (e.g. first date, job interview). Virtual reality has been suggested as a tool that might help make these interactions easier. In this study, we asked 22 autistic young people, aged 9-22, to try out a social interaction in virtual reality and then tell us about their experience. We found that most participants enjoyed using virtual reality and felt it made social interactions feel safer and less intimidating. They said virtual reality helped reduce distractions and gave them more control over how they presented themselves. Some even said it made them feel more confident when talking to others. However, young people were clear that virtual reality should not replace real-life interactions. They said that in-person contact is still important for building real connections with others. While virtual reality can be a helpful first step when interacting with new people, especially in situations that feel scary or unfamiliar, it is not a substitute for face-to-face relationships. This research shows that autistic young people value social connection and want tools that support, rather than replace, real-life interaction.

The study examined smart learning environments as a pathway to quality university education in Nigeria. The population of the study comprised all lecturers in the four government-owned universities in Bayelsa State. However, the researchers were unable to determine the precise number of lecturers at the time of this investigation due to data collection constraints. Based on the advice of Fox et al. (2007) and Meyer (1979), the researchers chose to adapt 384 samples from a population with an unconstrained range. The data used were collected through an online questionnaire; however, 296 lecturers responded by filling out the questionnaire. The findings reveal that smart learning tools such as projectors, internet facilities, computer-based test, mobile learning devices are readily available, while smart boards, video conferencing tools, digital cameras, and LMS are moderately available. On the contrary, virtual and augmented reality, smart classrooms, and gamification are not widely available, and the majority of the respondents are not familiar with them. There is an increased utilization of internet facilities, computer-based text, projectors, and mobile devices in the teaching processes, while other technologies such as LMS, smart boards, video conferencing tools, as well as virtual and augmented reality, gamification, and smart classes, are still emerging technologies. Lecturers have a positive perception of the utilization of smart learning technologies, noting that lack of adequate funds, lack of adequate electricity supply, and poor internet facilities were some of the major challenges in the utilization of SLE. Based on the findings, the researchers recommended, among others, that the universities should focus on improving the infrastructure, particularly reliable electricity supply and internet connectivity, and increase funding for the development of new smart learning systems, such as virtual reality, gamification, smart classrooms, etc.

Ultrasound mid-air haptic (UMH) devices are promising for tactile feedback in virtual reality (VR), as they do not require users to be tethered to, hold, or wear any device. This approach is less cumbersome, easy to set up, can simplify tracking, and leaves the hands free for concurrent interactions. This chapter explores work conducted at CNRS-IRISA dealing with the challenges arising from the integration of UMH interfaces in immersive VR through three main axes. These are discussed in the wider context of the state of the art on UMH for augmented and virtual reality, and illustrated through several VR use-cases. A first axis deals with device integration into the VR ecosystem. Interaction in immersive VR is based on the synergy between complex input devices allowing real-time tracking of the user and multimodal feedback devices delivering a coherent visual, auditory and haptic picture of a simulated virtual environment (VE). Using UMH in immersive VR therefore hinges on integrating UMH devices such that their operation does not interfere with other input and feedback devices. It is also critical to ensure that UMH feedback is adequately synchronized and co-located with respect to other stimuli, and delivered within a workspace that is compatible with that of VR interaction. Regarding this final point, we propose PUMAH, a robotic solution for increasing the usable workspace of UMH devices. The second and third axes, respectively, focus on stimulus perception and rendering of VE properties. Virtual object properties can be rendered in a variety of ways, through, e.g., amplitude modulation (AM) or spatiotemporal modulation (STM), with many parameters (modulation frequency, spatial sampling, etc.) coming into play, raising questions about the limits of the design space. To tackle this challenge, we begin by conducting psychophysical experimentation to understand the usable ranges for stimulus parameters and understand the perceptual implications of stimulus design choices. We propose an open-source software framework intended to facilitate UMH stimulus design and perceptual evaluation. These results in turn serve as the basis for the design and evaluation of rendering schemes for VR. Using amplitude variations along a focal point path in STM, we investigate the possibility of rendering geometric details and in a second step, sensations of stiffness in VR.

Recently, application scenario of crowdsourcing IoT has covered to e-healthcare service, smart home, smart city, internet of vehicles due to the proliferation of smart devices such as smart mobile devices, smart wearable device, smart medical devices and smart furniture, etc. Patient's data collected by the smart devices send to the various remote medical servers. A group of medical professionals remote access patient data stored at the medical server database. Smart home users want to remote real-time access information of smart devices at home. All these operations need via wireless remote communication, which is suffering from various kinds of threat and attacks. Hence, there are a large number of multi-factor remote authentication and key agreement schemes designed for the application of crowdsourcing IoT. However, in most existing related multi-factor schemes, all factors for identity authentication only act as a parameter for encrypting the local secret key. In this paper, we propose a new secure remote multi-factor authentication scheme that includes three factors: 1) user identity; 2) password; and 3) user biometrics, which are authenticated by the remote server, act as a part of the secret key and participate in the key agreement process. We choose the chaotic map since it has a smaller key size and lower computational overhead, and then achieve remote multi-factor authentication and key agreement by artfully employ it to zero-knowledge technology and the fuzzy extractor technology. Our scheme is more secure and robust since the user revealing nothing sensitive information, and the adversary cannot impersonate any user even if he gets the server's master key. We have done security proof for our proposed scheme using the Random-Or-Real(ROR) model, Burrows-Abadi-Needham (BAN) logic, and ProVerif 2.00 to show that the presented scheme is secure. Also, we give an additional security analysis for other various attacks. Finally, according to the test and simulation result, the proposed scheme is very suitable for the power-constrained smart devices, and in the next generation 5G communication environment, its applicability and usability will be greatly enhanced.

As smart mobile devices are widely used, mobile threats are more serious, so security in mobile becomes more and more important. However, the performance of these devices are not powerful enough to use the same security algorithms as PCs. Public key cryptosystems such as RSA need big primes to enhance the security, however, generating a big prime takes substantial time even on a PC. In this paper, we study two prime generation algorithms for smart mobile devices. First, we analyze a previous prime generation algorithm using a GCD test, named PGCD-MR, and show it sometimes performs inferior to the traditional TD-MR test. Second, we propose a new GCD test, named m-bit GCD-MR, for fast prime generation in both PCs and smart mobile devices. We compare the running times of PGCD-MR, m-bit GCD-MR, and TD-MR combinations on PCs and Samsung Galaxy Tab 10.1. The experimental results show our running time analysis is accurate (only 2% error) and m-bit GCD-MR test is the fastest among three prime generation algorithms. More exactly, m-bit GCD-MR test is about 20% faster than the TD-MR combination.

The use of gesture based interaction with devices has been a significant area of research in the field of computer science since many years. The main idea of these kind of interactions is to ease the user experience by providing high degree of freedom and provide more interactive way of communication with the technology in a natural way. The significant areas of applications of gesture recognition are in video gaming, human computer interaction, virtual reality, smart home appliances, medical systems, robotics and several others. With the availability of the devices such as Kinect, Leap Motion and Intel RealSense cameras accessing the depth as well as color information has become available to the public with affordable costs. The Intel RealSense camera is a USB powered controller that can be supported with few hardware requirements such as Windows 8 and above. This is one such camera that can be used to track the human body information similar to the Kinect and Leap Motion. It was designed specifically to provide more minute information about the different parts of the human body such as face, hand etc. This camera was designed to give users more natural and intuitive interactions with the smart devices by providing some features such as creating 3D avatars, high quality 3D prints, high-quality graphic gaming visuals, virtual reality and others. The main aim of this study is to try to analyze hand tracking information and build a training model in order to decide if this camera is suitable for sign language. In this study, we have extracted the joint information of 22 joint labels per single hand .We trained the model to identify the Indian Sign Language(ISL) numbers from 0-9. Through this study we analyzed that multi-class SVM model showed higher accuracy of 93.5% when compared to the decision tree and KNN models.