A Direct Manipulation Technique for Specifying 3D Object Transformations with a 2D Input Device

In this article, we will describe our work aiming to realize direct manipulation of 3D virtual objects used as virtual props by a human playing a role of an actor for recording a live video content. For direct manipulation of the 3D virtual props, the actor wears a data glove, which is replaced by a realistic virtual hand in the same 3D shape as the actor's hand using chroma key before the video is delivered to viewers. When the virtual hand replaces the data glove on the actor's hand, its posture is adjusted so that it grasps the virtual props without misalignment. For modeling the virtual hand, the actor's hand is measured by the light stripe triangulation while moving with various postures. The 3D point set obtained as the result is segmented into the points corresponding to the parts of the actor's hand based on the difference in motion among those parts so that the segmented points are integrated into a 3D description of an articulated object. The 3D virtual objects used for virtual props are modeled by recovering the 3D shapes of the real objects by the volume intersection method. The 3D shape recovered by the method is improved using motion of the object and random pattern backgrounds

The line drawing editor:Schematic diagram editing using pattern recognition technique

Many bidirectional programming languages, which are mainly functional and relational, have been designed to support writing programs that run in both forward and backward directions. Nevertheless, there is little study on the bidirectionalization of object-oriented languages that are more popular in practice. This paper presents the first bidirectional object-oriented language that supports programmatic and direct manipulation of objects. Specifically, we carefully extend a core object-oriented language, which has a standard forward evaluation semantics, with backward updating semantics for class inheritance hierarchies and references. We formally prove that the bidirectional evaluation semantics satisfies the round-tripping properties if the output is altered consistently. To validate the utility of our approach, we have developed a tool called BiOOP for generating HTML documents through bidirectional GUI design. We evaluate the expressiveness and effectiveness of BiOOP for HTML webpage development by reproducing ten classic object-oriented applications from a Java Swing tutorial and one large project from GitHub. The experimental results show the response time of direct manipulation programming on object-oriented programs that produce HTML webpages is acceptable for developers.

Recently released, depth-sensing-capable, and moderately priced handheld devices support the implementation of augmented reality (AR) applications without the requirement of tracking visually distinct markers. This relaxed constraint allows for applications with significantly increased augmentation space dimension, virtual object size, and user movement freedom. Being relatively new, there is currently a lack of study on issues concerning direct virtual object manipulation for AR applications on these devices. This paper presents the results from a survey of the existing object manipulation methods designed for traditional handheld devices and identifies potentially viable ones for newer, depth-sensing-capable devices. The paper then describes the following: a test suite that implements the identified methods, test cases designed specifically for the characteristics offered by the new devices, the user testing process, and the corresponding results. Based on the study, this paper concludes that AR applications on newer, depth-sensing-capable handheld devices should manipulate small-scale virtual objects by mapping directly to device movements and large-scale virtual objects by supporting separate translation and rotation modes. Our work and results are the first step in better understanding the requirements to support direct virtual object manipulation for AR applications running on a new generation of depth-sensing-capable handheld devices.

As computing gets embedded into our everyday lives and moves from PCs to wide varieties of devices, we need new and flexible ways to interact with technology. Recently, the appearance of multimodal systems has been recognized to offer richer user experiences by combining different natural input modes, i.e., speech, touch, pen, hand gestures, etc.. in a coordinated manner with multimedia system outputs. As a new generation of multimodal systems begins to define itself, one dominant theme is the integration and synchronization requirements for combining different modes strategically into whole systems; where we can distinguish basically three modalities: visual, auditive, and tactile (physiology of senses). In order to provide a broader range of signal integrations, actually limited by the conventional input devices of computers, it becomes necessary to ensure that these interfaces interact in a manner akin to the way human interacts in real environments. Therefore, The design and integration of a novel multimodal device developed at PERCRO is described. This multimodal device, named Haptic Desktop, has been designed to replace the conventional input devices used for accessing the resources of computers. Furthermore, to assure the usability of the interface so that operators can extract efficiently the principal features of each perceptual channel from the device, the graphical visualization has been integrated to the haptic system on a single desktop to satisfy the coherence and co-location designing issues.

This paper presents a virtual reality (VR)-based computer-aided design (CAD) system for conceptual design by applying VR interfaces and geometric modelling techniques to improve the human computer interaction. The research has studied the human factors for designing an intuitive computer-aided design system. The results and experiences have led to a deeper understanding of the limitation of current CAD tools that are used for conceptual design and the requirements for the VR-based CAD system so the design activities that define the user interfaces of VR-based CAD system can be determined. The system uses a CyberGlove, developed by Virtual Technology Inc., as an input device so that conceptual designers are allowed full freedom to use different kinds of gestures to conduct various geometric shape operations instead of depending solely on keyboard and two-dimensional mouse. The implemented shape modelling techniques offered by the system include destruction, construction, and novel techniques for freeform feature creation and modification. The system operates on the Windows NT platform and hence inherits a number of Windows operating styles. It provides a friendly human-computer interactive mechanism, especially for those users who are familiar with the Windows operating system. The VR-based user interfaces are superior and better match the human sensory capabilities than those that are dependent on conventional keyboard and mouse input devices.

Due to the rapid growth of Internet resources and technology, Internet access has become an important method of accessing and retrieving information. However, for some physically handicapped persons whose hand coordination and dexterity are impaired by such ailments as muscle atrophy, cerebral palsy, or other severe handicaps, the use of a conventional keyboard or similar standard input devices is not an option. Therefore, we designed and implemented, in this study, an easy‐to‐operate input interface, which was specifically designed for physically disabled Mandarin users to facilitate access to the Internet for them. Morse code, with an easily operated 1/2/6‐switch input device, was selected as an adaptive communication tool. The experimental results show that three physically impaired users could easily gain access to the Internet after three weeks’ practice on this proposed system.

Simulator sickness is still an unresolved issue in the VR field. Especially in driving or flight simulators, a perceptual discrepancy often leads to nausea and dizziness. This study tested the assumption that a mild stimulus to the sense of balance prevents or reduces this discrepancy using a VRGo, which acts as a control in contrast to conventional tilt input devices. A trend towards reduction in both time to nausea and severity of symptoms of simulator sickness were found supporting this hypothesis.

Teleoperation has been widely used to perform tasks in dangerous and unreachable environments by replacing humans with controlled agents. The idea of human-robot interaction (HRI) is very important in teleoperation. Conventional HRI input devices include keyboard, mouse and joystick, etc. However, they are not suitable for handicapped users or people with disabilities. These devices also increase the mental workload of normal users due to simultaneous operation of multiple HRI input devices by hand. Hence, HRI based on gaze tracking with an eye tracker is presented in this study. The selection of objects is of great importance and occurs at a high frequency during HRI control. This paper introduces gaze gestures as an object selection strategy into HRI for drone teleoperation. In order to test and validate the performance of gaze gestures selection strategy, we evaluate objective and subjective measurements, respectively. Drone control performance, including mean task completion time and mean error rate, are the objective measurements. The subjective measurement is the analysis of participant perception. The results showed gaze gestures selection strategy has a great potential as an additional HRI for use in agent teleoperation.

Recently, many researchers have worked on tabletop systems. One issue with tabletop interfaces is how to control the table without using conventional desktop input devices such as a keyboard or mouse. A second issue is allowing multiple users to simultaneously share the tabletop system. In this paper we explore how Augmented/mixed reality (AR/MR) technology can be used to explore these issues. One advantage of AR technology is being able to bring 3D virtual objects into the real world without needing to use a desktop monitor and allows users to intuitively interact with the objects. In this paper we describe a Tabletop Mobile AR system that combines a tabletop and a mobile interface. The Tabletop system can recognize user gestures and objects and intuitively manipulate and control them. In addition, multiple users have equal access for information on the table enabling them to easily share digital content. This makes possible unique entertainment and education applications. In this paper we describe the technology, sample entertainment interfaces and initial user feedback.

The ability to determine an object's position accurately and quickly is important in many robotics tasks. Monocular scene analysis based on perspective projection can be successfully used to solve this problem if a-priori knowledge on objects is available. In this paper analytic procedures for perspective inversion of quadrics of revolution--in particular for spheres, cones and cylinders--are presented. Preliminary experimental results on real images of a test object are provided, with the main goal to test the procedures accuracy and the suitability of the available low-level processing.

In this paper, we introduce a tilting interface that controls direction based\napplications in ubiquitous environments. A tilt interface is useful for\nsituations that require remote and quick interactions or that are executed in\npublic spaces. We explored the proposed tilting interface with different\napplication types and classified the tilting interaction techniques. Augmenting\nobjects with sensors can potentially address the problem of the lack of\nintuitive and natural input devices in ubiquitous environments. We have\nconducted an experiment to test the usability of the proposed tilting interface\nto compare it with conventional input devices and hand gestures. The experiment\nresults showed greater improvement of the tilt gestures in comparison with hand\ngestures in terms of speed, accuracy, and user satisfaction.\n

Quicktime virtual reality technology in light microscopy to support medical education in pathology

Preshaping Hand Behaviour for Direct and Indirect Manipulation of 3D Objects

This chapter is focused on 3D tomographic imaging. In 3D, we will consider the parallel line-integral projections, parallel plane-integral projections, and cone-beam line-integral projections, separately. For the 3D parallel lineintegral projections and parallel plane-integral projections, there exist the central slice theorems, from which the image reconstruction algorithms can be derived. However, for the cone-beam projections the situation is different; there is no central slice theorem for cone-beam. We have to somehow establish a relationship between the cone-beam projections and the 3D image itself. Since the cone-beam image reconstruction is an active research area, this chapter spends a significant effort on discussing cone-beam reconstruction algorithms, among which the Katsevich algorithm is the latest and the best one.