MobiGolog: formal task modelling for testing user gestures interaction in mobile applications

ABSTRACTUser Interface (UI) and User Experience (UX) are two crucial factors in the success of information systems, especially web-based applications and mobile applications. Design elements in web or mobile applications can be made more atrractive, by applying several elements in human-computer interaction theory based on user segmentation, product knowledge, consumer behavior, latest marketing theories, both in terms of visuals, after-effects, display sets, content, multimedia, lay-out menus on the web or mobile applications. In the recent state development of tourism in Bali, tourism transactions and promotions are already based on E-Tourism using web media and mobile applications. Making it easier for tourists to access information and services related to ticketing, transportation, accommodation and even travel packages. This study uses the User Interface (UI) and User Experience (UX) methods to view usability and web-quality from a web-based E-Tourism websites that promotes tourism in Bali.Keywords: User Interface, User Experience, E-Tourism, Informastion System, Bali Tourism.ABSTRAKUser Interface (UI) dan User Experience (UX) adalah dua faktor krusial dalam keberhasilan sistem informasi, terutama aplikasi berbasis web dan aplikasi mobile. Elemen desain dalam web dapat dibuat lebih menarik, dengan menerapkan beberapa elemen dalam teori interaksi manusia-komputer berdasarkan segmentasi user, pengetahuan produk, perilaku konsumen, teori pemasaran terbaru, baik dari segi visual, after-effect, display set, konten, multimedia, lay-out menu di web atau aplikasi mobile. Dalam perkembangan pariwisata di Bali saat ini, transaksi dan promosi pariwisata sudah berbasis E-Tourism dengan menggunakan media web dan aplikasi mobile. Sehingga mempermudah wisatawan dalam mengakses informasi dan layanan terkait dengan ticketing, transportasi, akomodasi bahkan paket perjalanan wisata. Penelitian ini menggunakan metode User Interface (UI) dan User Experience (UX) untuk melihat usability dan web-quality dari website layanan E-Tourism yang berbasis web yangmempromosikan pariwisata di Bali.Kata kunci: User Interface, User Experience, E-Tourism, Sistem Informasi, Pariwisata Bali.

The article is devoted to the issue of expanding the functions of cartographic images for transmitting geospatial information using mobile devices. It discusses the functions of mobile mapping applications. Based on the classification of the main types of perception, a study was conducted on the correlation of user-accepted geospatial information with the means of its transmission. Since modern mobile technologies make it possible to obtain cartographic information with the help of visual, auditory, tactile perception, and also make it possible to sense space, movement and time, the map in combination with these types of perception becomes more intuitive. Actual in the visualization of the cartographic image on the screen of the mobile device is the question of exploring the user interface and displaying the contents of the map at different scale levels. In this regard, the requirements for interfaces are generalized, the principles necessary for the development of the interface of cartographic applications are formulated and presented, based on the condition: “Conditional sign — interface element”. The proposed principles can be divided into three groups: principles that define the general issues of building the interface of mobile cartographic applications, their structure as a whole and user interaction; principles governing the design of the interface of mobile cartographic applications; principles governing the development of mobile cartographic applications, taking into account the features of their use and new technical capabilities. Using the example of level-by-level compilation of elements of navigation charts, a conclusion is drawn on the correspondence between the display of spatial information on a mobile device and the user's perception of reality. Generalized features of cartographic image visualization in mobile devices, such as: intuitive readability, map adaptability, generalization by motion factor, multiscale and multimedia.

SummarySince multitouch interfaces are steadily moving out of the experimental niche, the demand for comprehensive programming support increases. Between the formation of de facto standards for multitouch gestures by the industry, and continued research and experimentation by developers or academia, the abstraction of gesture properties becomes essential. This paper contributes a revision of the Gesture Formalization for Multitouch (GeForMT) gesture description language. This language aims at a short and concise syntax, which is still able to capture the complexity of multitouch gestures. We report on a previously missing reference architecture and practical implementation. Based on a study comparing the revised GeForMT language with other formalization approaches for multitouch gestures, we introduce an editor for GeForMT.Copyright © 2013 John Wiley & Sons, Ltd.

The chapter presents a description of a novel modelling language called Alvis defined for the design of concurrent especially real-time systems. Alvis combines the advantages of formal methods and practical modelling languages. Based on CCS and XCCS process algebras, Alvis utilizes flexible graphical modelling of interconnections among agents and a high level programming language used for the description of agents behaviour. Each agent in a model can use rule-based systems to support its decisions. A small set of language statements and graphical concepts make Alvis easy to learn and use. The possibility of a formal model verification, makes Alvis a formal modelling language. Alvis modelling environment creates in parallel a model of the considered system and a labelled transition system (LTS graph) that is its formal representation. The LTS graph can be formally verified with the CADP toolbox. A survey of main Alvis features from practical point of view, is given in the chapter.

Natural language descriptions of user interface (UI) elements such as alternative text are crucial for accessibility and language-based interaction in general. Yet, these descriptions are constantly missing in mobile UIs. We propose widget captioning, a novel task for automatically generating language descriptions for UI elements from multimodal input including both the image and the structural representations of user interfaces. We collected a large-scale dataset for widget captioning with crowdsourcing. Our dataset contains 162,859 language phrases created by human workers for annotating 61,285 UI elements across 21,750 unique UI screens. We thoroughly analyze the dataset, and train and evaluate a set of deep model configurations to investigate how each feature modality as well as the choice of learning strategies impact the quality of predicted captions. The task formulation and the dataset as well as our benchmark models contribute a solid basis for this novel multimodal captioning task that connects language and user interfaces.

Augmented Reality (AR) is a technology that provides the seamless interaction between real and virtual environment. Since its emergence, AR has been gaining wide acceptance in various fields of application including the gaming industry. Its unique interface allows for integration of difference user interface and interaction techniques to create immersing and fun gaming experience. Motion detection which is related with visual user interface (UI) and gesture recognition is one such example. With the introduction of devices such as Microsoft Kinect and Nintendo Wii Remote, motion based technique has been fast gaining popularity as a more natural mean of user interface, which can be referred as natural user interface (NUI). This paper presents a conceptual design framework for motion detection based AR games. A physical prop is introduced as a reference point to enhance the game experience. A case scenario of a simple game will be presented to illustrate the concept. Potential design and implementation challenges will be highlighted and discussed.

There is substantial evidence that incorporating interactive environments for game-based instruction has a significant potential to support the development of computational thinking and programming skills in primary education students. However, it is not clear whether a simulation game (SG) with different user interface elements, created via three-dimensional (3D) virtual worlds and visual programming environments to project various problem-solving exercises in a simulated reality, can significantly influence students to think and practice "computationally" their solution plans into code. The current study aims to identify any possible added value of each instructional approach by measuring students’ game experience and learning performance. This quasi-experimental study involved ninety participants (n = 90) aged between 10 and 11 years that consisted of two comparison conditions. The experimental group (n = 45) received training using OpenSimulator in combination with Scratch4SL, whereas the control group (n = 45) was trained using Scratch. The findings indicate considerable differences in students’ game experience and satisfaction, but no statistically significant difference in their learning performance and knowledge gain was identified. This study provides several design implications for user interface and game elements to inform educational practitioners and instructors about the benefits that each approach can offer students for better knowledge acquisition and deeper disciplinary understanding.

The data binding pattern is an established technique to couple user interface (UI) elements and data objects. Various markup languages (e.g. Microsoft XAML, Adobe MXML) integrate advanced data binding concepts in order to ease application development. However, the HTML standard does not embrace means for data binding although being the Web markup language supported by millions of Web programmers. Therefore, we propose a standard-compliant WebSocket-based Data Binding (WebSoDa) framework. The WebSoDa framework synchronizes data objects and UI elements by orchestrating a Microdata-based data binding language as well as a client-side and a server-side messaging component. Thus, developers may speed up the tedious task of implementing binding associations in Web applications.

There are a diverse range of mobile healthcare applications available to download from various mobile market places, however ensuring that an application is personalised to the patients' requirements and remain creditable, requires the inclusion of healthcare professionals in the design and development process. Previous work [1] concluded that traditional mobile application development approaches are not feasible, since healthcare professionals typically do not possess the necessary skills or knowledge to develop mobile applications. Therefore proposed an ontology driven framework that enables healthcare professionals to develop personalised mHealth applications for their patients, without intervention from mobile application developers. However, existing ontology driven developments focus on a specific type of healthcare service and model user characteristics to achieve personalisation. Utilising this method to achieve personalisation would not be feasible in achieving the aim of this research, since the applications developed using the framework will need to be capable of supporting various types of healthcare services. This paper presents a novel ontology-based approach for developing personalised mHealth applications.

Recent empirical studies in the area of mobile application testing indicate the need for specific testing techniques and methods for mobile applications. This is due to mobile applications being significantly different than traditional web and desktop applications, particularly in terms of the physical constraints of mobile devices and the very different features of their operating systems. In this paper, we presented a multiple case-study involving four software development companies in the area of mobile and smartphones application. We aimed to identify testing techniques currently being applied by developers and challenges that they are facing. Our principle results are that many industrial teams seem to lack sufficient knowledge on how to test mobile applications, particularly in the areas of mobile application life-cycle conformance, context-awareness, and integration testing. We also found that there is no formal testing approach or methodology that can facilitate a development team to systematically test a critical mobile application.

Modeling approaches based on standards are of paramount importance in the field of front-end design for web and mobile applications. Problems are often encountered during the selection of tools for designing applications for developers and researchers, particularly applications that are related to time and cost in the market and academia. The Interaction Flow Modeling Language (IFML) is a recently standardized modeling language designed for managing the content expression, user interaction and behavior control of front-end applications. IFML brings several benefits to the development process of web and mobile front-end applications. Thus, several tools have been developed for the exploitation of technical artifacts offered by current specifications. In this study, we discuss the tools proposed for modeling IFML to provide a comparative analysis while considering various criteria. The result can be used as a basis of tool selection for specific aspects.

Mobile applications are an essential element in pervasive and ubiquitous computing, and they face many challenges during their generation process from the analysis of user needs to the design of specific mobile interfaces and their development in several technological platforms. Moreover, the rise of Ambient Intelligent and context-aware environments also introduces multiple interaction aspects to be considered when using mobile devices in this kind of scenarios. The present work seeks to examine the role of design patterns and ontology models in order to help with the generation of mobile applications, which can be adapted at runtime to the various user needs, different context scenarios, interactive design modes, or technology requirements. In this way, an ontology-based framework is introduced to represent, design, and support the adaptation of user interfaces in mobile applications by using design patterns according to these user needs or preferences and the context around them. This framework provides developers with a client-server architecture that enables the access to an expert knowledge base of user, context, and pattern information together with a set of inference rules, which allow the dynamic selection of interface design patterns and the runtime adaptation of the user interface features. These ontology models and inference rules are key components of the proposed framework, and their implementation has helped to produce an example of mobile application supporting user interface adaptation processes for disabled people, which can be required in Ambient Intelligent environments. Three examples of user scenarios have been considered to assess the framework potential, and usability dimensions have been tested by a limited set of users through the produced mobile application, making the usefulness of generated adaptive user interfaces apparent.

XML Based Graphical User Interface Editor and Runtime Parser for ISO 11783 Machine Automation Systems

An alternative solution for Indosport, considering the increasingly advanced smartphone development and also followed by the rapid growth of its users in Indonesia. The purpose of this study project is to produce a product in the form of a user interface (UI) and user experience (UX) design in helping to realize targeted mobile applications and not forgetting to apply the principles of visual communication design namely unity and variety, hierarchy, contrast, proportion and balance. The UI and UX design methods go through several stages of work in sequence: pre-production process, production process, and post-production. This design results in UI and UX designs that are equipped with user flow, interface designs, illustrations and prototypes. The main page of the Indosport mobile application is News, Activity, Calendar, and Profile which later on these pages there are several features that are different from the previous page. The making of this UI uses several visual languages such as medium close up, full shot, long shot, close up and medium long shot in the image section contained in the article as well as the depiction of field styles used to almost all visual UI elements. Using several principles in design, namely time to learn, speed of performance, rate of errors by users, retention over time, and subjective satisfaction. The resulting design forms will be distributed directly to the Research and Development division and the IT division from Indosport to be made a native.

In software development for people with Autistic Spectrum Disorder (ASD), it is necessary to consider specific design aspects that enable the use of web and mobile applications by such users. However, only a small part of current websites and applications is adequately designed considering the needs of these users. In this paper, we extracted quality user interface attributes regarding autistic users from guidelines found in human computer interaction literature. Based on these attributes, and through the analysis of real applications specifically designed for autistic users found in Brazil, we created new design patterns presenting practitioners with problems, solutions and examples in order to support the development of more appropriate user interfaces for ASD users. We evaluated these patterns from the point of view of practitioners involved in caring for autistic people and software engineers working in projects for autistic end-users. The results provide indicators of the usefulness and understandability of the proposed patterns, while allowing to identify improvement opportunities in their description. By providing a new set of user interface design patterns, we aim to contribute to the development of more accessible mobile and web applications for users with ASD.