Intelligent User Interfaces for Mobile Computing

In this chapter, the practical issue of realizing a necessary intelligence quotient for conceiving intelligent user interfaces (IUIs) on mobile devices is considered. Mobile computing scenarios differ radically from the normal fixed workstation environment that most people are familiar with. It is in this dynamicity and complexity that the key motivations for realizing IUIs on mobile devices may be found. Thus, the chapter initially motivates the need for the deployment of IUIs in mobile contexts by reflecting on the archetypical elements that comprise the average mobile user’s situation or context. A number of broad issues pertaining to the deployment of AI techniques on mobile devices are considered before a practical realisation of this objective through the intelligent agent paradigm is presented. It is the authors hope that a mature understanding of the mobile computing usage scenario, augmented with key insights into the practical deployment of AI in mobile scenarios, will aid software engineers and HCI professionals alike in the successful utilisation of intelligent techniques for a new generation of mobile services.

A critical problem with existing computer-based geometric modeling is the labor intensive task involved in specifying data input for the description of three-dimensional (3-D) objects. This paper describes a new, 3-D input system aimed at alleviating this problem. It is based on the use of a three-dimensional digitizer for the direct input of spatial coordinates, and an intelligent interactive user interface. The intent of this system is to create a high level, intelligent interface between the designer and a geometric solid modeler which would lighten the designer's burden in performing arduous 3-D geometric description tasks. The user interface is developed around the Knowledge Craft expert system building tool, using the rule-based Carnegie Representation Language OPS5 (CRL-OPS). The system uses schematic networks or frames and production rules to encode knowledge about geometric primitive digitization methods, object feature operators, solid modeler requirements, and input command functions. It also employs a forward-chaining inference strategy to direct the knowledge. This ensures that only a minimal amount of valid data entry is required by the user. However, if excessive data is entered the intelligent interface has the capability to extract the required information. As a result, the solid modeler can automatically create the appropriate object “primitive” or the specific object “feature” upon recognition by the expert system. It will be demonstrated that these capabilities can simplify the 3-D model description process.

In this article, we present an intelligent human-machine interface designed to control the medical robotic exoskeleton of the lower limbs “Remotion”. Intelligent bimodal interface combines tools of contactless voice control, as well as contact sensor-based control on mobile devices. The authors argue that the use of intelligent techniques of interaction between the user and the exoskeleton increases the level of ergonomics as well as effectiveness of its use in medical rehabilitation practice due to an intuitive and natural way of human-machine communication and control. The salient features of the proposed and developed system are automatic voice command recognition, conversion audio signal into text data, remote control and remote supervision of the exoskeleton through a PC, and active and informative feedback with the user, securing safety during rehabilitation sessions. The design of the interface is shown in the paper in details, outlines of engineering solutions applied for this exoskeleton are provided as well. Despite the existence of conceptually similar devices in Russia, the presented exoskeleton differs by the presence of an intelligent user interface, which significantly improves the ergonomics of the system. Another feature is a total modularity of the device. Such a solution greatly facilitates the maintenance of the exoskeleton and provides a range of adjustment opportunities.

The human-computer interaction (HCI) is a research field that supports the user interface development. Furthermore, HCI influences the intelligent user interface (IUI) area. IUI is a user interface that uses intelligent technology to reach the communication between humans and machines. The construction of IUI is not easy and requires the knowledge of different methods, techniques and technologies from many domains (such as, HCI, artificial intelligence, ergonomics, software engineering, etc.). In this way, the existence of a process is an important tool for IUI developers. The literature presents design processes and frameworks for user interface, but most of them do not offer any knowledge regarding the methods, techniques, and technologies that can be used to support the activities involved. Thus, this work proposes a development process for IUIs which offers recommendations about methods, techniques, and technologies. We hope that this process will help analysts and designers who work in IUI area.

Automated user modeling for intelligent interface

It is our great pleasure to welcome you to the 2012 ACM International Conference on Intelligent User Interfaces -- IUI'12 and to Lisboa, Portugal. Starting in 1993, IUI is now on its 17th edition and has established itself has the premier venue for reporting outstanding research and development on intelligent user interface. The IUI series continues to be the principal forum for the meeting of the Human-Computer Interaction (HCI) and the Artificial Intelligence (AI) communities. This gives rise to unique, creative, interdisciplinary contributions. This year's meeting showcases those topics, with works covering innovations in mobile interfaces, collaboration technology, affective interfaces, multimodal interfaces, pen interfaces, haptic and gesture interfaces, multi-touch interfaces, speech interfaces, gaze-based interfaces, tabletop interfaces, health applications, entertainment applications, adaptive narratives and theater, sports and in-vehicle applications, geographic applications, social media interfaces, educational interfaces, sketch recognition, human-robot interfaces, personalization and assistive technologies, end-user programming, ubiquitous and smart environments, locationaware interfaces, recommender interfaces, persuasive interfaces, web-based interfaces and agentbased interfaces. Building on previous years' success, the IUI 2012 call for papers attracted 134 full paper submissions and 78 short paper submissions. To ensure the highest possible quality we assembled a team of 48 senior program committee members and 367 reviewers. This guaranteed that every submission had at least three reviews plus one meta-review. This was followed by a rebuttal phase, keeping the procedure started two years ago. We believe that this process has guaranteed the highest quality possible for this year technical program and we would like to thank all the senior program committee members and reviewers for their hard work in making sure the most relevant works were selected. Through this very thorough review process we were able to accept 18 full papers, 15 short papers and 16 poster presentations, meaning an acceptance rate of 13% for full papers, 16% for oral presentations and 23% for combined oral and poster presentations. In addition to the full and short papers presentations and poster presentations, IUI will also feature a Demonstrations session for which 18 submissions were accepted from a total of 26 submissions. This year, the IUI programme includes three notable invited speakers sharing their innovative work and experiences with the conference participants: Alex 'Sandy' Pentland from the MIT Human Dynamic Lab, Christopher Bishop from Microsoft Research and Takeo Igarashi from the University of Tokyo. In order to foster the growth of the community, IUI 2012 will feature for the first time in the IUI series a Doctoral Consortium. Its success can be measured by the 12 high quality submissions received. Doctoral Consortium students will also be able to exhibit their work to the main conference audience in the poster session, thus promoting their insertion into the IUI community. We would also like to thank the conference sponsors and supporters that made it possible to provide financial support for Doctoral Consortium attendees and other students participating in the conference. Nine full-day workshops will take place at IUI 2012 covering diverse trending topics in the IUI area: Activity Context Representation: Techniques and Languages; Developing Intelligent User Interfaces for e-Accessibility and e-Inclusion; Context-awareness in Retrieval and Recommendation (CaRR 2012); 3rd Workshop on Semantic Models for Adaptive Interactive Systems (SEMAIS); Scent and Scensibility; User Modeling from Social Media; 2nd Workshop on Interacting with Smart Objects; 2nd Workshop on Location Awareness for Mixed and Dual Reality (LaMDa'12); 1st International Workshop on Ubiquitous Personalization (UP'2012). IUI 2012 continues and extends the cooperation with the ACM Transactions on Interactive Intelligent Systems (TiiS). Besides the streamlined processing of journal submissions based on IUI 2012 papers, IUI 2012 will feature a special session where authors of recently accepted articles for the ACM TiiS will spotlight key contributions of their work. It is our conviction that it will benefit greatly every member of the IUI community. We would like to thank Anthony Jameson and John Riedl, the Editors in Chief of TiiS, for their cooperation.

The image of a license plate is scanned by the camera attached to a mobile PDA device and the numbers on the plate are detected by the image processing parts in the proposed Mobile system. Then the numbers and the location of a mobile PDA device are encoded and transmitted along with the location information to a remote server through a wireless communication network. Finally, the server decodes the transmitted data as a text format and transmits it to the destination user. Consequently, this paper contributes a case study on the embedded system for designing of intelligent interface between a moving vehicle and a mobile PDA device, using a spatial relative distance scheme. The experimental results show that detection and tracking of a location of moving vehicle can be conducted efficiently with a mobile PDA device in real-time through wireless communication system and Internet.

Today's mobile devices enable the users to capture video clips with integrated digital cameras. However, capturing a video clip exactly as intended is often challenging -- in many cases, the possibility to edit the clip after capture would be useful. Earlier we have presented the Mobile Video Editor, an application that allows the users to edit video clips instantly after capture on their mobile devices. In this paper, we describe a field trial that we arranged to further evaluate the Mobile Video Editor concept and to better understand the usage patterns of video editing in the mobile context. The results indicate that especially the teenagers and younger adults found the capability to edit captured video clips a beneficial feature in mobile devices. In addition to being a practical tool to prepare the videos for sharing with others, the Mobile Video Editor was used as a form of entertainment.

Mobile Cloud computing has shown its capability to support mobile devices for provisioning computing, storage and communication resources. Many existing research has proposed to offload computation tasks from mobile devices to clouds in order to reduce energy consumption, where the offloading service model is usually one-to-one. Due to the development of mobile sensing and location-based mobile cloud services, the cloud edge has been extended to the mobile devices and sensors. As a result, the one-to-one model is not sufficient to model the dynamic changes of mobile cloud-based services. Thus, a many-to-many (or multi-site) mobile cloud service composition is highly desired. In this research, MoSeC is presented to model the many-to-many mobile cloud service composition, where there are multiple surrogates, such as cloud computing nodes, mobile devices, or sensors, and their services (i.e., computation, storage, sensing, etc.) can be composed to fulfill functions required by a mobile service requestor. MoSeC takes into considerations the surrogates' changes due to their mobility and resource constraints. Moreover, MoSeC takes into considerations several service metrics for service mapping (or allocation) through a mobile cloud service topology reconfiguration process. A set of algorithms are presented to address the Service Topology Reconfiguration Problem (STRP) in several mobile cloud representative application scenarios, i.e., they are modeled as finite horizon scenarios, infinite horizon scenarios, and large state space scenarios to represent ad hoc, long-term, and large-scale mobile cloud service composition scenarios, respectively.

Although smartphones are increasingly becoming more and more powerful, enabling pervasiveness is severely hindered by the resource limitations of mobile devices. The combination of social interactions and mobile devices in the form of ‘crowd computing’ has the potential to surpass these limitations. In this paper, we introduce Honeybee; a crowd computing framework for mobile devices. Honeybee enables mobile devices to share work, utilize local resources and human collaboration in the mobile context. It employs ‘work stealing’ to effectively load balance tasks across nodes that are a priori unknown. We describe the design of Honeybee, and report initial experimental data from applications implemented using Honeybee.

The increased popularity of feature-rich mobile devices in recent years has enabled widespread consumption and production of social media content via mobile devices. Because mobile devices and mobile applications change context within which an individual generates and consumes microblog content, we might expect microblogging behavior to differ depending on whether the user is using a mobile device. To our knowledge, little has been established about what, if any, effects such mobile interfaces have on microblogging. In this paper, we investigate this question within the context of Twitter, among the most popular microblogging platforms. This work makes three specific contributions. First, we quantify the ways in which user profiles are effected by the mobile context: (1) the extent to which users tend to be either fully non-mobile or mobile and (2) the relative activity of the mobile Twitter community. Second, we assess the differences in content between mobile and non-mobile tweets (posts to the Twitter platform). Our results show that mobile platforms produce very different patterns of Twitter usage. As part of our analysis, we propose and apply a classification system for tweets. We consider this to be the third contribution of this work. While other classification systems have been proposed, ours is the first to permit the independent encoding of a tweet’s form, content, and intended audience. In this paper we apply this system to show how tweets differ between mobile and non-mobile contexts. However, because of its flexibility and breadth, the schema may be useful to researchers studying Twitter content in other contexts as well.

Extensive decision support system composed of several modules providing different functionalities for the user can be a challenge in designing of a usable, plain and consistent user interface. A characteristic of advisory systems in the domain of metallurgy is their complex architecture designed to deal with multiple types of tasks in many areas. The objective of this paper is to present the possibility of creating an intelligent interface providing the user with complete freedom to act without precise knowledge of the multiple functionality of the system. The paper presents the architecture of developed advisory system in the domain of metallurgy enriched with the description of mechanisms, which enable obtaining an intelligent user interface. The intelligent user interface should guide the user at navigation through all system modules in order to meet all needs that are the reason for this system usage. In designing of such interface, the Case-Based Reasoning methodology is proposed to be used. Case-Based Reasoning ensures a pattern for building systems that are able to learn on the basis of human actions associated with human decisions and use such knowledge to resolve the currently appearing problems. Each user query directed to the system, and each of his decisions about the choice of functionalities of the system modules supplies the system with the crucial information on proper use of the whole decision support unit related with the needs of a user. Skillful use of this knowledge can provide convenient and efficient performance of intelligent interface.

This full-day tutorial will provide an overview on Intelligent User Interfaces in the context of the technological development associated with the Ubiquitous Computing paradigm. Based on a technological assessment of the main technological areas involved (miniaturization, communication and network structures, advanced display and speech technologies, as well as sensor integration and deployment) this tutorial will discuss the main approaches in context and user modeling, as well as present new interaction paradigms for ubiquitous computing environments. Examples of past and current research projects of the field will be used to explain and discuss the intersection of UbiComp and IUIs. Having discussed the technical foundations, the tutorial aims to explore the design and development of new pervasive applications for mobile devices and other distributed interfaces. It will explore how handhelds can interact with other surrounding devices, intelligent environments and with the social context. The tutorial will also look at how Participatory Design approaches and advanced in-situ evaluation techniques and ethno-methodologies such as living labs, on-the-field data collection, enactments, and 'Cultural Probes', can lead to the representation of significant communication systems.

Advances in wireless communication systems and mobile devices allow nomad users to participate in mobile collaborative activities. However the availability of hardware resources in the mobile devices participating in the collaboration process enhances or jeopardizes such activity. This paper studies how the network topology and the hardware resources distributed into a mobile network influence the collaboration activities among the participants. Several simulations were done to try to understand this issue. The obtained results show that in mobile collaboration scenarios involving a high number of resources-constraint mobile devices (e.g. handhelds), the maximum cooperation among node is obtained in a small world network topology. The results also show that another factor that encourages cooperation among nodes is the network size.

An increasing number of people use a large variety of Web-based services on their mobile devices. Many of these services contain dynamically changing items such as media content produced by various social networks; news, schedule and route information from service providers; and updated versions of the service functionality. As the number of the used services grows, user's effort needed for being up-to-date in terms of the changing content gets higher. Paying attention to new content may be burdensome especially in the mobile context, on mobile devices. We conducted a qualitative user study to explore how users currently find out what's new in the services via the mobile user interfaces. Ten users were interviewed and asked to show how they notice changes in the mobile services they use. We present the found change indicators and propose design implications for mobile service user interfaces.