A hybrid 2D/3D user interface for immersive object modeling

The thesis presents the research of inventing a novel and natural spatial human-system interface – MagicPad that is to be used in a fully immersive virtual environment. With the advance in Virtual Reality (VR) technology, modern computer systems are capable of immersing users into a synthetic environment. Instead of sitting in front of the monitor, virtual objects are surrounding the users in an immersive VR system and therefore the traditional human-computer interfaces such as mouse and keyboard are no longer appropriate / fit for such purpose. Since then, researchers have been actively seeking new means of human-computer interaction in the three-dimensional space. The objective of this research is to innovate, design and build a novel spatial user interface for immersive VR systems by investigating and identifying the core elements of an intuitive and effective spatial user interface. The research methodology adopted is based on the quantitative research methods where the perception and interaction of human subjects with a synthetic virtual environment such user interfaces are collected to investigate the performance and effectiveness of the spatial user interface. The key elements of an intuitive and effective spatial user interface are first identified and the design concept of the MagicPad user interface prototype is then established. In order to realize the overall design and study the concept of the spatial human-system interface, an immersive Virtual Reality system imseCAVE, which serves as the test bed of the MagicPad user interface is designed and built from scratch. In this research, the MagicPad interface is implemented and quantitative user evaluations with the imseCAVE are carried out to draw conclusion and future research directions in the field of Virtual Reality and 3D User Interfaces.

In this paper we propose vrLib, a toolkit which aims to make 3D graphical user interface and interaction very easy to design in virtual environments. This project provides well known and new interaction techniques and ready to use widgets, which are totally hardware independent. The main contribution allows to elaborate complex 3D user interfaces in same way it is done in 2D. This toolkit provides adaptive interaction, which means interaction techniques can automatically adapt their behavior to 2D and 3D widgets. This short paper describes the vrLib architecture for construction of 3D interactive applications.

Recupera algunos de los elementos menos estudiados y reconocidos en el ensayo latinoamericano: su profunda vocación de reflexión, diálogo y mediación intelectual; su plasticidad y apertura a la búsqueda del sentido; su vínculo con la sociabilidad, la conversación y la generación de un espacio simbólico de encuentro; su capa cidad de establecer una nueva retroalimentación entre lectura y escritura.

3D multi-user virtual environments (MUVE) bring new opportunities to foreign language teaching and learning as a platform for teaching practices. Due to immersive and interactive nature of 3D MUVE, these tools have gained popularity in the field of foreign language learning and teaching. The power of virtual learning environments lies in creating immersive 3-D spaces that give users a sense of learning by doing. Moreover, the environment is interactive and is much the same as real world interaction. MUVE provides strong support for synchronous collaborations, interactions, and immersive environments for experiential and constructivist learning settings. Some key features of the virtual worlds comprise the ability to construct 3D environments that simulate real world situations, as well as the ability to have a virtual identity to create a cross-cultural self and the ability to have participants from around the world acting virtually in designated roles that convey new types of learning within this environment.

Fourth Planet, Inc. - Santa Clara, CA 95054Formerly:NASA Ames Research Center - Intelligent Mechanisms GroupMoffet Field, CA 94301flueckiger@dmt.epfl.chABSTRACTThe expansion of robotic systems’ performance, as well as the need for such machines to work in complex environments(hazardous, small, distant, etc.), involves the need for user interfaces which permit efficient teleoperation. Virtual Reality b asedinterfaces provide the user with a new method for robot task planning and control: he or she can define tasks in a very intuitiv eway by interacting with a 3D computer generated representation of the world, which is continuously updated thanks tomultiple sensors fusion and analysis.The Swiss Federal Institute of Technology has successfully tested different kinds of teleoperations. In the early 90’s, atransatlantic teleoperation of a conventional robot manipulator with a vision feedback system to update the virtual world wasachieved. This approach was then extended to perform teleoperation of several mobile robots (Khepera, Koala) as well as tocontrol microrobots used for microsystems’ assembly in theµm range.One of the problems encountered with such an approach is the necessity to program a specific kinematic algorithm for eachkind of manipulator. To provide a more general solution, we started a project aiming at the design of a “kinematic generator”(CINEGEN) for the simulation of generic serial and parallel mechanical chains.With CINEGEN, each manipulator is defined with an ascii file description and its attached graphics files; inserting a newmanipulator simply requires a new description file, and none of the existing tools require modification. To have a real timebehavior, we have chosen a numerical method based on the pseudo-jacobian method to generate the inverse kinematics of therobot. The results obtained with an object-oriented implementation on a graphic workstation are presented in this paper.Keywords: robot manipulators, general kinematic chains, inverse kinematics, user interface, virtual environment.1. INTRODUCTIONHumans design and build robots to assist or substitute them in a large range of tasks. This leads to machines with incrediblepower and features, which can perform more and more sophisticated tasks. But too many times, the difficulty for the user tocommand the robot, increases with the complexity of the system. Virtual Reality (VR) based interfaces provide a powerful toolto overcome this problem.This paper focuses on a “general kinematic generator” which aims at simulating robots in Virtual Environment. Such a toolhas appeared as a real need for several research labs as well as industrial companies involved in robotics. Two research labs,both working in the robotic field and using Virtual Reality tools for efficient teleoperation have joined their efforts to achievesuch a kinematic generator.1.1. BackgroundThe Micro-Engineering Department (Institut de Microtechnique: IMT) of the Swiss Federal Institute of Technology(EPFL) is involved in robotic design and development, with a special focus on industrial applications. The experience gainedwith industrial partners showed that classical methods for robotic systems programming (off-line as well as on-line) usuallylack on user-friendliness and performance. This is why since 1990 the IMT is developing VR interfaces to simplify robot taskdefinition.Rather than writing down complicated code, the user can interact with a 3D model of a robot in a virtual world. The user isable to define high level tasks (like move to this point or grab this object) in just one hand movement or a button press. The v ir-tual world is continuously updated, thanks to different sensors (camera, scanner, etc.), with all detected objects around therobot. After a simulation stage to check all the potential problems, a concrete task can be achieved by sending the automati-

We conducted two experiments that compared distance perception in real and virtual environments in six visual presentation methods using either timed imagined walking or direct blindfolded walking, while controlling for several other factors that could potentially impact distance perception. Our presentation conditions included unencumbered real world, real world seen through an HMD, virtual world seen through an HMD, augmented reality seen through an HMD, virtual world seen on multiple, large immersive screens, and photo-based presentation of the real world seen on multiple, large immersive screens. We found that there was a similar degree of underestimation of distance in the HMD and large-screen presentations of virtual environments. We also found that while wearing the HMD can cause some degree of distance underestimation, this effect depends on the measurement protocol used. Finally, we found that photo-based presentation did not help to improve distance perception in a large-screen immersive display system. The discussion focuses on points of similarity and difference with previous work on distance estimation in real and virtual environments.

An appropriate user interface to collect human demonstration data for deformable object manipulation has been mostly overlooked in the literature. We present an inter-action design for demonstrating cloth folding to robots. Users choose pick and place points on the cloth and can preview a visualization of a simulated cloth before real-robot execution. Two interfaces are proposed: A 2D display-and-mouse interface where points are placed by clicking on an image of the cloth, and a 3D Augmented Reality interface where the chosen points are placed by hand gestures. We conduct a user study with 18 participants, in which each user completed two sequential folds to achieve a cloth goal shape. Results show that while both interfaces were acceptable, the 3D interface was more suitable for understanding the task, and the 2D interface was suitable for repetition. Results also found that fold previews improve three key metrics: task efficiency, the ability to predict the final shape of the cloth, and overall user satisfaction.

Desktop environments represent a powerful user interface and have been used as the de facto standard human-computer interaction paradigm for over 20 years. But the rising demand of 3D applications dealing with complex datasets exceeds the capabilities of traditional interaction devices and two-dimensional displays. Such applications need more immersive and intuitive interfaces. In order to be accepted by the users, technology-driven solutions that require inconvenient instrumentation, e.g., stereo glasses or tracked gloves, should be avoided. Autostereoscopic display environments equipped with tracking systems enable humans to experience virtual 3D environments more naturally, for instance via gestures, without having to use annoying devices. However, currently these approaches are used only for specially designed or adapted applications. In this paper we introduce new 3D user interface concepts for such setups which require minimal instrumentation of the user and can be integrated easily in everyday working environments. We propose an interaction framework which supports simultaneous display of and simultaneous interaction with both monoscopic as well as stereoscopic contents. We identify the challenges for combined mouse-, keyboard- and gesture-based input paradigms in such an environment and introduce novel interaction strategies.KeywordsHCIautostereoscopic display environments3D user interfaces

This work describes a novel hardware/software platform dedicated to design review process simplification, using immersive reality technologies. The proposed platform allows designers to interface with CAD engines and visualize different data types simultaneously into an immersive and stereoscopic multi-view visualization system. This research focuses on the development of a novel immersive user interface, a smart 3D disk with a set of widgets. During the immersive sessions, the user can activate functionalities using cost-effective pointing devices and the conceived 3dUIs projected into the virtual environment. In this way, it is possible to easily manipulate virtual objects, perform basic operations such as rotations and translations but also more complex CAD functionalities such as surfaces shape modification. Each feature can be selected inside the virtual world using the smart 3D disk. Users evaluations show that the use of a virtual environment may enhance the perception of designers ideas during the design process and the use of smart 3D interfaces simplifies the interaction among user and virtual objects.

PurposeThe purpose of this paper is to present a prototype pluggable service‐oriented virtual learning environment, enabling teachers to create an integrated teaching environment using tools that have been chosen to best meet their academic requirements.Design/methodology/approachThis is an implementation of a WAFFLE Bus. A microkernel software design pattern is used to enable tools to be added and removed from the system. An enterprise service bus is used to provide workflow and message transformation functionality. Tools are managed through web service interfaces and Shibboleth is used to effect interoperability at the web application user interface. The initial services for the prototype were chosen to implement a simple web service teaching workflow.FindingsFirst, Shibboleth is shown to provide a solution to the virtual learning environment tools' interoperability problem. Second, the service‐oriented virtual learning environment naturally leads to the ability to operate with many different types of information channels in and out of the system. This leads to a multiplicity of possible types of context‐dependent user interface. Third, immersive 3D, possibly the most interesting interface, will provide a context amenable to even the smallest development teams for the introduction of artificial intelligence into teaching. Finally, web service workflow is shown to provide a viable option for the implementation of learning designs with advantages and disadvantages compared to existing approaches.Research limitations/implicationsDifferent types of information channels are associated with different security problems. It will be important to determine what the best ways are of establishing secure channels to student personal learning environments. The present web service workflow design tools are of the highest quality and usability, but the design process is still a job for a specialist. It might be possible, however, to modify these open source tools to bring the design process within the grasp of non‐specialists.Originality/valueThe software system presented herein represents one possible path leading away from VLE monolithy using a service‐oriented approach. A new solution to the tools' interoperability problem is presented along with a multi‐faceted approach to the user interface. The enterprise service bus creates a flexible platform for the delivery of web service teaching and learning workflows. It is posited that the use of an immersive 3D user interface will create a context that facilitates the introduction of an artificial intelligence layer into the virtual learning environment that can serve robot teaching avatars.

Nowadays, touchable displays are widely used in most embedded systems, including mobile phones and other hand-held devices. Although interactive user interfaces are intuitively required for these systems, most of interactive graphical user interface implementations usually need lots of system resources. For low-tier embedded systems, we have presented that the text-based user interfaces are more suitable, as shown in our previous works. In this paper, we present an object-oriented approach to provide a full-scale text-based user interface scheme, especially on the low-tier embedded systems. Our application user interface implementation is very simple and intuitive, with few amount of computing resource. It naturally supports multiple overlapping windows, pop-up menus, pull-down menus, dialog boxes, and other user interface components, with graceful object-oriented application programming interfaces. We also provide large size text output for emphasis purposes. All the systems are now available as a software library, and can be used for wide range of applications.

Some members of the so-called virtual world deny that technology such as the virtual environment (VE) is value-laden. But there are a variety of complex issues which arise in VE that make it necessary to think about ethics and values in VE applications. Using VE in therapy and psychotherapy research or diagnostics leads to several ethical concerns. VE can impoverish those aspects of life that are essential to social development, interpersonal relations, and emotional growth. This paper will focus on the concept called "virtual world," a question of metaphysics. After reviewing general ethical principles, ethics in applied sciences will be described. Ethical decision-making, code of conduct, and specific issues in VE will be discussed. The VE community is challenged to set guidelines around VE, and its ethics and values. A set of measurement tools around ethical codes and values in a virtual world ought to be discursively gained in this virtual world and in real society as a whole.

The emergence of serious or non-leisure uses of game technologies and virtual world applications has been swift and dramatic over the last few years. As a result, there has been little time to develop the theoretical and academic rigor in the emerging field. Another factor affecting the field has been the general aspect of convergence between new technologies, for example augmented reality, mobile technologies, GPS technologies, sensor technologies and social software. Together, this has presented potential for developing new technologies based upon immersive and interactive interfaces and has in parallel produced many unanswered questions in terms of the usage of the formats. As a result there is a clear need to consider new frameworks, theories, methods and design strategies for making serious applications of games and virtual world technologies more effective and useful. The term ‘Serious Games’ covers a broad range of applications from flash-based animations to totally immersive and code-driven 3D environments where users interface with large volumes of data through sophisticated and interactive digital interfaces. This shift towards immersive world applications marks the beginning of new challenges that offer real scope for collaborative and multi-disciplinary research solutions and real opportunities for innovative development. The 1st International Conference in Games and Virtual Worlds for Serious Applications (VS-GAMES’09) gathered individual researchers and games’ developers that focus on serious games methodologies, mobile games, virtual and augmented environments, and 3D social worlds.

This chapter explores virtual worlds, or the metaverse, as spaces of and for the artistic imagination, in which the entanglement of the physical with the virtual is being exploited for its creative and imaginative potential. In particular, there are opportunities to investigate, subvert, invert, and even question our current understanding of virtual space (and the concept of the metaverse itself) through an exploration and manipulation of time, space, and identity. Creative opportunities not ordinarily available in physical world spaces are enabling artists to create new spaces of the imagination that are almost tangible, almost physical, and most certainly a ‘real’ experience for the audience. For Patrick Lichty, it is the particular characteristics of virtual worlds that closely echo the logic of the physical world that are providing ‘an existential overlay’ to the experience of virtual world spaces, and he considers how the ‘invocation of this metaphor [raises] questions about the intent of the creation of such worlds and how (or if) artists will choose to engage with virtual environments in this manner’ (Lichty, 2009, pp. 2–3). However, it is this ‘play’ of the imagination between the virtual and physical that underpins many of the themes explored by artists in projects developed on one virtual world platform, Second Life, during its lifetime (Doyle, 2010).KeywordsVirtual WorldVirtual SpaceRemote IslandWorld SpacePossibility SpaceThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

In general, the use of VE (Virtual Environment) systems has required users to travel to specialized facilities in which an expensive infrastructure has been pre-installed. Often these facilities allow only one user at a time, and users usually move through virtual worlds by means of an artificial interface device or a movement metaphor. In short, most VR facilities are centralized, specialized, and expensive, and have thus been relatively unavailable to a great majority of the population. This paper reports on the development of an innovative immersive VE system that is completely portable. The proposed system would allow multiple users to be immersed simultaneously and could be used in any large indoor or outdoor area such as a gymnasium or parking lot. Users will move through virtual worlds naturally, and will be able to walk for miles in a virtual world without ever becoming aware of the physical limits of the tracking space or the locations of other users. The portable system will cost far less than current systems. The research hinges on three emerging techniques. The first is redirected walking (RDW). The second is self-contained inertial position tracking (SCIPT). The last is the use of ultrasonic and/or laser ranging systems to simultaneously localize and map (SLAM) within an unfamiliar tracking area. The paper describes the effort that is underway and reports preliminary results which have been obtained in each of these areas over the last few years.