Keynote speaker: Towards immersive multimodal display: Interactive auditory rendering for complex virtual environments

Current computer games are being set in increasingly more complex and dynamic virtual environments. Massively multiplayer online games, for example, are played in persistent virtual worlds, which evolve and change as players create and personalize their own virtual property. In contrast, technologies for controlling the behavior of nonplayer characters that populate virtual game worlds are frequently limited to preprogrammed rules. Characters using fixed rule-sets lack the ability to adapt in time with their environment. Motivated reinforcement learning offers an alternative to character design that can achieve nonplayer characters that both evolve and adapt in dynamic environments. This article presents and evaluates two computational models of motivation for use in nonplayer characters in persistent computer game worlds. These models represent motivation as an ongoing search for novelty, interest, and competence. Two metrics are introduced to evaluate the adaptability of characters controlled by motivated reinforcement learning agents using different models of motivation. These metrics characterize the behavior of nonplayer characters in terms of the variety and complexity of learned behaviors. An empirical evaluation of characters in simulated game scenarios shows that characters motivated by the search for competence are more adaptable in dynamic environments than those motivated by interest and novelty alone.

Virtual friend recommendations in virtual worlds

VR has the potential to provide experiences and deliver results that cannot be otherwise achieved. However, interacting with immersive applications is not always straightforward and it is not just about an interface for the user to reach their goals. It is also about users working in an intuitive manner that is a pleasurable experience and devoid of frustration. Although VR systems and applications are incredibly complex, it is up to designers to take on the challenge of having the VR application intuitively communicate to users how the virtual world and its tools work so that those users can achieve their goals in an elegant and comfortable manner.

There is widespread interest in the development and use of decentralized systems and virtual world environments as possible new places for engaging in collaborative work activities. Similarly, there is widespread interest in stimulating new technological innovations that enable people to come together through social networking, file/media sharing, and networked multi-player computer game play. A decentralized virtual activity system (DVAS) is a networked computer supported work/play system whose elements and social activities can be both virtual and decentralized (Scacchi et al. 2008b). Massively multi-player online games (MMOGs) such as World of Warcraft and online virtual worlds such as Second Life are each popular examples of a DVAS. Furthermore, these systems are beginning to be used for research, deve-lopment, and education activities in different science, technology, and engineering domains (Bainbridge 2007, Bohannon et al. 2009; Rieber 2005; Scacchi and Adams 2007; Shaffer 2006), which are also of interest here. This chapter explores two case studies of DVASs developed at the University of California at Irvine that employ game-based virtual worlds to support collaborative work/play activities in different settings. The settings include those that model and simulate practical or imaginative physical worlds in different domains of science, technology, or engineering through alternative virtual worlds where players/workers engage in different kinds of quests or quest-like workflows (Jakobsson 2006).

Virtual worlds for children are becoming increasingly popular, and yet there are few accounts of children’s use of these worlds. Young children are spending increasing amounts of time online as technology continues to create significant changes in social and cultural practices in the 21st century. Some of children's online interactions can be categorized as playful in nature; however, play and technology are frequently positioned as oppositional. In this article, I explore the tensions surrounding the relationship between play and technology and relate it to similar discourses concerning the concepts of ‘real’ and ‘virtual’. I then move on to consider the growing popularity of virtual worlds with young children and examine the way in which the worlds have been marketed to children and parents/carers on the basis of their propensity to offer online play in a safe environment. The article provides an overview of two virtual worlds currently targeted at young children and draws on a survey of primary children’s use of virtual worlds in order to identify the nature of play in these environments. One hundred and seventy-five children aged 5—11 completed an online survey and 15 took part in group interviews in which their use of virtual worlds was explored. This article focuses on the data relating to 17 children aged from five to seven years who used virtual worlds. Findings indicate that virtual worlds offered these young children a wide range of opportunities for play and that the types of play in which they engaged relate closely to ‘offline’ play. The implications for early years educators are considered.

There are benefits for social and behavioral researchers to conduct studies in online virtual worlds. However, typically learning scripting takes additional time or money to hire a consultant. We propose a prototype Virtual World Toolkit for to help researchers design, set up, and run experiments in Virtual Worlds, with little coding or scripting experience needed. We explored three types of prototype designs, focused on a traditional interface with pilot results. We also present results of initial expert user study of our toolkit to determine the learnability, usability, and feasibility of our toolkit to conduct experiments. Results suggest that our toolkit requires little training and sufficient capabilities for a basic experiment. The toolkit received a great feedback from a number of expert users who thought that it is a promising first version that lays the foundation to more future improvements. This toolkit prototype contributes to enhancing researchers’ capabilities in conducting social/behavioral studies in virtual worlds and hopefully will empower social and behavioral researchers by proving a toolkit prototype that requires less time, efforts and costs to setup stimulus responses types of human subject studies in virtual worlds.

The development of information and communication technology (ICT) has been promoting virtual consumption, such as purchasing virtual goods in online virtual worlds. One of the representative markets where virtual consumption is prevalent is the game market. In particular, despite the overall decline in the economy due to COVID-19, the number of users and revenue of online games have shown a very high increase. In addition, as the use of digital technology becomes common in everyday life, virtual consumption is very likely to expand in the future, and it is also highly likely to change the existing traditional consumption ecosystem. Therefore, this study focuses on the phenomenon of virtual consumption in games and attempts to verify the main factors affecting the promotion of virtual consumption. This study presents a research model by deriving the influencing factors that reflect the characteristic values that are considered more important in game virtual worlds, in order to analyze the consumption value that affects the behavioral intention of virtual consumption in the game virtual world from multiple aspects. In addition, this study conducts a comparative analysis between Chinese and Korean users to verify whether the influencing factors of virtual consumption differ between user groups in the countries with different social and cultural backgrounds. Based on the analysis results, this study can provide useful implications to those involved in the game and virtual world industries by identifying the main influencing factors on the promotion of virtual consumption phenomenon in the rapidly growing game virtual worlds.

Objective: Pilot test GoWoman, a small-group weight management intervention for mobility impaired women that was a disability- and gender-responsive adaptation of the Diabetes Prevention Program delivered in the online virtual world of Second Life®. Objectives were to (1) examine pre-/post-intervention differences in weight, waist circumference, diet, physical activity, self-efficacy for diet and physical activity, nutrition knowledge and social support for weight management, (2) determine intervention feasibility (fidelity, attrition, engagement, acceptability).Design: Single-group modified interrupted time series quasi-experimental design whereby participants served as their own controls.Results: Thirteen women attended ≥8 of 16 GoWoman weekly sessions and lost an average of 5.97 pounds (2.71 kg) (3.31%) body weight (Cohen’s d = 0.74) and 1.44 inches (3.66 cm) (3.58%) waist circumference (Cohen’s d = 0.83). There were significant improvements in physical activity, diet and self-efficacy for diet and physical activity. All benchmarks for feasibility were met. Ratings of intervention content, group interactions and support and virtual world experiences were highly positive.Conclusion: Findings suggest that a disability- and gender-responsive weight management intervention with peer group support delivered in an online virtual world is feasible, meaningful and may assist with weight management for mobility impaired women.Implications for RehabilitationThis study addresses a gap in the general and rehabilitation research literature by addressing the disproportionately high rates of obesity among women with mobility impairments, who are generally excluded from tests of weight management interventions if they have limited ability to engage in vigorous physical activity.The GoWoman program is an adaptation of the Diabetes Prevention Program Lifestyle Change curriculum that is tailored to meet the unique weight management needs of women with mobility impairments, and was created to become a publicly available, disability- and gender-responsive intervention that can be used in community and rehabilitation settings.More rehabilitation and health promotion program should be offered in the free, online, virtual world of Second Life® since participants in this pilot study offered many favorable comments about the new learning and social opportunities available to them there and they did not have to deal with the disability-related environmental and health challenges that often prevent them from participating in face-to-face workshops.Preliminary indications of improvements in body weight, waist circumference, diet and physical activity after attending the GoWoman weight management intervention offered in Second Life® tell us that these strategies are feasible for helping women with mobility impairments manage their weight and should undergo further testing.

This chapter describes the research being done to create a healing center for veterans of recent conflicts in the online virtual world Second Life. This work, started in 2008 and funded by the US Army’s Research Office, is building a dedicated online virtual world space to provide not only social connectivity for widely dispersed veterans, but also to offer stress-reducing health care and activities, especially in the area of Complementary and Alternative Medicine. The social space is populated by several intelligent agent-avatars that serve various functions. For example, there is a knowledgeable guide that can show people the various activities and spaces the site has to offer, find out and store a list of the visitors’ interests, and keep a record of what they have seen and done. There are also intelligent agents who can provide guidance for certain activities such as walking a labyrinth, and a story activity called “The Warriors’ Journey.” Working with experts in Mindfulness-Based Stress Reduction (MBSR), we are implementing MBSR therapy in this Second Life space for veterans. These experts will run the in-world group sessions, inhabiting an avatar for this purpose. Experiments with returning veterans are planned for 2010 that will test the efficacy of such therapies deployed in the virtual world. We expect the use of virtual worlds for both group and individual therapies to increase in the future and this project is a first step in determining their viability.

Introduction and wide use of information technologies in all spheres of human life including for the purpose of entertainment has led to the emergence and spread of various types of Internet addiction. Numerous studies have established the effect of the nature of activity in the virtual world on the state of the human mental condition. In our research was conducted analysis of the sleep disorders in technical university students, who use computer games and video films for recreation and entertainment. We have researched in the sleep disorder mechanisms in technical university students, who use computer games and video films for recreation and entertainment. The authors of the article K. N. Meziyanaya, K. D. Yashin, K. M. Karaneuski developed the questionnaire "Method of Screening Diagnostics of Computer Addiction and its Influence on Physical and Mental Health" to study of formation of sleep disorders [1]. Interviews with 142 2nd to 5th year students studying in one of the technical universities of Minsk city were conducted in order to study the formation of sleep disorders under the influence of the computer media environment: males – 119 (84.4%), females – 23 (16.2%). The average age of respondents was 19.7 (±1.5) years. The average duration of the virtual world use was 9 (±2) years. It is established that 101 students (71.1% of the number of respondents) are forced to wake up from fear and anxiety. The statistical relationship is shown between the symptom of sleep disturbance: they say in a dream "I just dream it" and the symptom: duration of stay in the virtual world 40 or more hours per week. χ² criterion = 5.63; p = 0.05. The relationship between combining computer games with watching movies and waking up from fear and anxiety is found, χ² criterion = 4.38, p = 0.05. The findings suggest that the continued use of computer games in combination with viewing video films contributes to the development of malfunction of the organization and maintenance of sleep. The results testify to the correlation between a sleep disorder and a prolonged stay in the virtual world, which can contribute to predicting the formation of anxiety disorders.

In this chapter we discuss our research on multimodal interaction in a virtual environment. The environment we have developed can be considered as a ‘laboratory’ for research on multimodal interactions and multimedia presentation, where we have multiple users and various agents that help the users to obtain and communicate information. The environment represents a theatre. The theatre has been built using VRML (Virtual Reality Modeling Language) and it can be accessed through World Wide Web (WWW). This virtual theatre allows navigation input through keyboard function keys and mouse, but there is also a navigation agent which tries to understand keyboard natural language input and spoken commands. Feedback of the system is given using speech synthesis. We also have Karen, an information agent which allows a natural language dialogue with the user. In development are several talking faces for the different agents in the virtual world. We investigate how we can increase the user’s commitment to the environment and its agents by providing context and increasing the user’s feeling of ‘presence’ in the environment.KeywordsHuman-computer interactionvirtual realityembodied agentstalking facestext-to-speech synthesisspeech recognitionpresence

Interaction techniques that require users to adopt new behaviors mean that designers must take into account social acceptability and user experience otherwise the techniques may be rejected by users as they are too embarrassing to do in public. This research uses a set of low cost prototypes to study social acceptability and user perceptions of multimodal mobile interaction techniques early on in the design process. We describe 4 prototypes that were used with 8 focus groups to evaluate user perceptions of novel multimodal interactions using gesture, speech and nonspeech sounds, and gain feedback about the usefulness of the prototypes for studying social acceptability. The results of this research describe user perceptions of social acceptability and the realities of using multimodal interaction techniques in daily life. The results also describe key differences between young users (18-29) and older users (70-95) with respect to evaluation and approach to understanding these interaction techniques.

We present an interactive wave-based sound propagation system that generates accurate, realistic sound in virtual environments for dynamic (moving) sources and listeners. We propose a novel algorithm to accurately solve the wave equation for dynamic sources and listeners using a combination of precomputation techniques and GPU-based runtime evaluation. Our system can handle large environments typically used in VR applications, compute spatial sound corresponding to listener's motion (including head tracking) and handle both omnidirectional and directional sources, all at interactive rates. As compared to prior wave-based techniques applied to large scenes with moving sources, we observe significant improvement in runtime memory. The overall sound-propagation and rendering system has been integrated with the Half-Life 2 game engine, Oculus-Rift head-mounted display, and the Xbox game controller to enable users to experience high-quality acoustic effects (e.g., amplification, diffraction low-passing, high-order scattering) and spatial audio, based on their interactions in the VR application. We provide the results of preliminary user evaluations, conducted to study the impact of wave-based acoustic effects and spatial audio on users' navigation performance in virtual environments.

Despite steady research advances in many aspects of virtual reality, building and testing virtual worlds remains to be a very difficult process. Most virtual environments are still implemented using traditional programming language tools like compilers and debuggers. What makes virtual environment construction more difficult is that, on top of having to tackle with the traditional computational and logical errors, it is also an exploration task. Developers must find the right combination of various types of constituents of the virtual environment like the objects, display and simulation details, interaction and modalities, etc. Unlike ordinary programming tasks, for VR, the execution and development environments are different, not just in the temporal sense, but also in the physical sense. We propose that an interactive kernel approach can partly solve this problem and promote higher efficiency in VR content development. Developers can try out and explore different configurations of the constituents of the virtual environment and immediately see their impact (within the virtual world in construction) and optimize the content along the dimensions of performance, interaction usability, realism, and presence. The interactive kernel is part of PiVOT-2 (POSTECH Virtual reality system development Tool 2), a collection of tools that support a VR system development methodology called the CLEVR (Concurrent and LEvel by Level Development of VR System), previously proposed by the authors [Seo and Kim 2002]. CLEVR/PiVOT-2 is based on a flexible virtual object model called (PiVOT Object Model), that clearly segregates the aspects of form, function and behavior and the interactive kernel allows the developers to dynamically configure the respective attributes within the virtual world in construction. In this paper, we describe the details of how POM was designed, and demonstrate the effectiveness of using the interactive kernel in constructing virtual worlds.

Read newspapers and professional educational journals, attend a national or state conference, or review the offerings of a university catalog, and you soon encounter statistics like 92 percent of children ages 2–17 play video and computer games (Beck & Wade, 2004); 60 percent of Americans play interactive games on a regular basis (Kirriemuir, 2002); 78 percent of American families have video game equipment in their homes (Simpson, 2005). You will also read that 78 percent of 18–29 year olds use the Internet in their daily lives (Ramaley & Zia, 2005), yet only 38 percent of college students report using the Internet in their classes (Ramaley & Zia, 2005). Outside the classroom, students are creating a new shared culture, showing us new ways to learn and communicate and make sense of physical and virtual identities and worlds. They are showing us that what happens in virtual worlds is often just as meaningful as what happens offline (Taylor, 2006). They work autonomously or with others—playing with others online who may protect and advise them or playing next to others who play the same game. They know where to obtain helpful resources. However, these mainly young gamers do not just play games and consult resources; they create game guides, answers to FAQs, maps, overviews, strategies, fanfic (fanfiction—stories about the gaming characters or settings written by game fans rather than by game creators), and character-planning guides. All of this adds up to a different kind of play and developmental environment, which influences a different developmental process and way of seeing the world and thinking than experienced by generations before.