A method for remote and semi-automatic usability evaluation of web-based applications through users behavior analysis

Introduction Usability evaluation is essential to make sure that software products newly released are easy to use, efficient, and effective to reach goals, and satisfactory to users. For example, when a software company wants to develop and sell a new product, the company needs to evaluate usability of the new product before launching it at a market to avoid the possibility that the new product may contain usability problems, which span from cosmetic problems to severe functional problems. Three widely used methods for usability evaluation are Think Aloud (TA), Heuristic Evaluation (HE) and Cognitive Walkthrough (CW). TA method is commonly employed with a lab-based user testing, while there are variants of TA methods, including thinking out aloud at user's workplace instead of at labs. What we discuss here is the TA method that is combined with a lab-based user testing, in which test users use products while simultaneously and continuously thinking out aloud, and experimenters record users' behaviors and verbal protocols in the laboratory. HE is a usability inspection method, in which a small number of evaluators find usability problems in a user interface design by examining an interface and judging its compliance with well-known usability principles, called heuristics. CW is a theory-based method, in which evaluators evaluate every step necessary to perform a scenario-based task, and look for usability problems that would interfere with learning by exploration. These three methods have their own advantages and disadvantages. For instance, TA method provides good qualitative data from a small number of test users, but laboratory environment may influence test user's behaviors. HE is a cheap, fast and easy-to-use method, while it often finds too specific and low-priority usability problems, including even not real problems. CW helps find mismatches between users' and designers' conceptualization of a task, but it needs extensive knowledge of cognitive psychology and technical details to apply. However, even though these advantages and disadvantages show overall characteristics of three major usability evaluation methods, we cannot compare them quantitatively and see their efficiency clearly. Because one of reasons why so-called discounted methods, such as HE and CW, were developed is to save costs of usability evaluation, cost-related criteria for comparing usability evaluation are meaningful to usability practitioners as well as usability researchers. One of the most disputable issues related to cost of usability evaluation is sample size. That is, how many users or evaluators are needed to achieve a targeted usability evaluation performance, for example, 80% of overall discovery rate? The sample size of usability evaluation is known to depend on an estimate of problem discovery rate across participants. The overall discovery rate is a common quantitative measure that is used to show the effectiveness of a specific usability evaluation method in most of usability evaluation studies. It is also called overall detection rate or thoroughness measure, which is the ratio of 'the sum of unique usability problems detected by all experiment participants' against 'the number of usability problems that exist in the evaluated systems', ranging between 0 and 1. The overall discovery rates were reported more than any other criterion measure in the usability evaluation experiments and also a key component for projecting required sample size for usability evaluation study. Thus, how many test users or evaluators participate in the usability evaluation is a critical issue, considering its cost-effectiveness.

The complexity of an electronic medication administration record (eMAR) has been underestimated by most designers in the past. Usability issues, such as poorly designed user application flow in eMAR, are therefore of vital importance, since they can have a negative impact on nursing activities and result in poor outcomes. The purpose of this study was to evaluate the usability of an eMAR application during its development. A usability evaluation was conducted during the development of the eMAR application. Two usability methods were used: a heuristic evaluation complemented by usability testing. Each eMAR application version provided by the vendor was evaluated by 2 hospital pharmacists, who applied the heuristic method. They reviewed the eMAR tasks, detected usability problems and their heuristic violations, and rated the severity of the usability problems. Usability testing was used to assess the final application version by observing how 3 nurses interacted with the application. Thirty-four versions were assessed before the eMAR application was considered usable. During the heuristic evaluation, the usability problems decreased from 46 unique usability problems in version 1 (V1) to 9 in version 34 (V34). In V1, usability problems were categorized into 154 heuristic violations, which decreased to 27 in V34. The average severity rating also decreased from major usability problem (2.96) to no problem (0.23). During usability testing, the 3 nurses did not encounter new usability problems. A thorough heuristic evaluation is a good method for obtaining a usable eMAR application. This evaluation points key areas for improvement and decreases usability problems and their severity.

Determination of the effectiveness of two methods for usability evaluation using a CPOE medication ordering system

With the rapid growth of e-commerce, consumer demand for personalized packaging solutions has grown significantly. To address this issue, this article constructs an intelligent recommendation model that combines user behavior data and sentiment analysis techniques. Firstly, a user behavior preference model is established by collecting potential preferences and behavioral characteristics of users, in order to explore their network behavior characteristics. Next, the Bidirectional Encoder Representations from Transformers (BERT) word vector is used to represent the comment text, and a bidirectional recurrent neural network is used to quantify the emotional information in the comment. Based on the emotional rating, the rating matrix is updated to map the shallow features of users and resources. Subsequently, by combining convolutional neural networks and self attention mechanisms, deep features of users and resources are extracted from comment texts, and shallow and deep features are fused through multi-layer neural networks to model the nonlinear interaction between users and resources and predict the rating values of recommended resources. In addition, this article proposes a hierarchical attention enhanced recommendation model incorporating User behavior and sentiment analysis (HAER-UBSA), which obtains feature information from comment texts through attention mechanisms and models comment level embedded representations of users and items. The experimental results show that compared with other baseline models, the mean absolute error (MAE) and root mean squared error (RMSE) indicators of our model have improved by 10.34% and 10.00%, respectively.

User behavior visualization as a powerful tool can help analysts better understand user behavior data. Previous research such as Information Architecture believes that a user journey map is the regular way to visualize user behavior patterns. The technique is a diagram that shows the steps, touch-points, and emotions that users go through when using their products or services. Current studies mainly focus on the procedure and framework of applying user behavior data into visualization tools and its benefit to business development. Studies also found that visualization in user behavior analysis has difficulty in dealing with complex data. However, few studies have discussed the possible solution for the current limitation of visualization in user behavior analysis. With the popularization of the Internet and mobile devices, the scale and scale of user behavior data and the complexity will continue to increase, and user behavior visualization will be an important tool for analyzing this data, helping to tap potential business opportunities and improve the efficiency of data analysis. Therefore, we would make the review on visualization in user behavior analysis. We will first provide the background of the user behavior and its visualization and then introduce the definition of user behavior. Afterward, we will discuss the current process of visualization in user behavior analysis and its current limitations. Next, we will propose the possible solution of introducing machine learning, deep learning, predicting the data with a sequence model, and using cluster analysis with usability analysis tools to contribute to the future development of visualization in user behavior analysis. We believe that with data amount and scale keep increasing, the visualization tools need to develop themselves in diversity and data processing speed.

This paper presents a continuous authentication system based on user behavior analysis that makes use of environmental context information, users' behavior analysis and Neuro-Fuzzy Logic. This system must be able to acquire information in context, making them into a computational environment. This information is the basis of user behavior. The System, based on the evidences of the behavior, establishes if it can trust the user or not. According to the user behavior, levels of trust are released, to access the application software. Weights are attributed in the fuzzyfication process, according to the rules that were previously established for the parameters which help to establish the evidences of behavioral trust, in its different degrees. The neuro-fuzzy logic allows that the user behavioral database be continuously updated, interacting with the fuzzyfication mechanism, so as to keep trust levels updated according to the user behavior, in a more accurate and faithful way.

Usability evaluation methods have gained a substantial attention in networks particularly in Intrusion Detection System (IDS) as these evaluation methods are envisioned to achieve usability and define usability defects for a large number of practical software’s. Despite a good number of available survey and methods on usability evaluation, we feel that there is a gap in existing literature in terms of usability evaluation methods, IDS interfaces and following usability guidelines in IDS development. This paper reviews the state of the art for improving usability of networks that illustrates the issues and challenges in the context of design matters. Further, we propose the taxonomy of key issues in evaluation methods and usability problems. We also define design heuristics for IDS users and interfaces that improves detection of usability defects and interface usability compared to conventional evaluation heuristics. The similarities and differences of usability evaluation methods and usability problems are summarized on the basis of usability factors, current evaluation methods and interfaces loopholes

Usability evaluation methods have become critical in the Web domain to ensure the success of Web applications. Aim: Since a large number of proposals have been presented during the last few years, a question arises: Which usability evaluation methods have proven to be the most effective in the Web domain? Method: This paper presents a systematic review that was motivated by previous results obtained from a systematic mapping study in the Web usability evaluation field. Results: A total of 18 studies were selected from an initial set of 206 in order to extract, code, and synthesize empirical data concerning the effectiveness of usability evaluation methods for the Web. Conclusions: We detected a need of more empirical studies and more standardized effectiveness measures for comparing usability evaluation methods. Our results suggest several evaluation methods which may be useful in allowing researchers and practitioners to perform effective Web usability evaluations.

Analyzing user behavior characteristics in a complex power grid environment is essential for user behavior planning and resource coordination optimization. Traditional user behavior analysis methods based on model-driven and causal analysis have the disadvantages of strong subjectivity and physical models that are difficult to deal with the randomness and uncertainty of user behavior in complex grid environments. In this paper, we use unsupervised learning methods to analyze user behavior in complex power grid environments, and propose user behavior analysis methods based on stacked autoencoder and clustering. We first reduce the complexity of user behavior data by proposing adaptive feature selection algorithm of user behavior based on stacked autoencoder and unsupervised learning (AFS-SAEUL). Finally, we build a user behavior analysis model based on adaptive feature selection and improved clustering (UBA-AFSIC). The model improved the performance of unsupervised classification of user behavior by fusing the adaptive generation strategy of the initial cluster centers. The simulation experiment results on two real electricity datasets and one public electric vehicle charging dataset show that compared with the existing feature selection algorithm and clustering algorithm, the algorithms proposed in this paper have higher feature selection rate and better clustering performance.

The goal of this workshop is to explore the implications of automated capture and analysis of user behaviors on HCI and UE research.

BackgroundUsability is a key factor affecting the acceptance of mobile health applications (mHealth apps) for elderly individuals, but traditional usability evaluation methods may not be suitable for use in this population because of aging barriers. The objectives of this study were to identify, explore, and summarize the current state of the literature on the usability evaluation of mHealth apps for older adults and to incorporate these methods into the appropriate evaluation stage.MethodsElectronic searches were conducted in 10 databases. Inclusion criteria were articles focused on the usability evaluation of mHealth apps designed for older adults. The included studies were classified according to the mHealth app usability evaluation framework, and the suitability of evaluation methods for use among the elderly was analyzed.ResultsNinety-six articles met the inclusion criteria. Research activity increased steeply after 2013 (n = 92). Satisfaction (n = 74) and learnability (n = 60) were the most frequently evaluated critical measures, while memorability (n = 13) was the least evaluated. The ratios of satisfaction, learnability, operability, and understandability measures were significantly related to the different stages of evaluation (P < 0.05). The methods used for usability evaluation were questionnaire (n = 68), interview (n = 36), concurrent thinking aloud (n = 25), performance metrics (n = 25), behavioral observation log (n = 14), screen recording (n = 3), eye tracking (n = 1), retrospective thinking aloud (n = 1), and feedback log (n = 1). Thirty-two studies developed their own evaluation tool to assess unique design features for elderly individuals.ConclusionIn the past five years, the number of studies in the field of usability evaluation of mHealth apps for the elderly has increased rapidly. The mHealth apps are often used as an auxiliary means of self-management to help the elderly manage their wellness and disease. According to the three stages of the mHealth app usability evaluation framework, the critical measures and evaluation methods are inconsistent. Future research should focus on selecting specific critical measures relevant to aging characteristics and adapting usability evaluation methods to elderly individuals by improving traditional tools, introducing automated evaluation tools and optimizing evaluation processes.

As computer and network intrusions become more and more of a concern, the need for better capabilities to assist in the detection and analysis of intrusions also increases. We propose a methodology for analyzing network and computer log information visually based on the analysis of user behavior. Each user's behavior is the key to determining their intent and overriding goals, whether they attempt to hide their actions or not. Proficient hackers will attempt to hide their ultimate goal, which hinders the reliability of log file analysis. Visually analyzing the user's behavior, however, is much more adaptable and difficult to counteract. This paper will discuss how user behavior can be exhibited within the visualization techniques, the capabilities provided by the environment, typical characteristics users should look out for (i.e., how unusual behavior exhibits itself), and exploration paradigms effective for identifying the meaning behind the user's behavior.

The analysis of user behavior in online communities is a prominent topic in social media research. As such, user behavior is often analyzed using a set of metrics that describe the user’s participation behavior and structural position in the social network. Yet, for Enterprise Social Networks (ESN), i.e. internally used social networking platforms, such research is lacking. While prior studies have found users to engage in knowledge-intensive interactions, e.g. discussing and developing new ideas, little is known about how to conceptualize and measure ESN user behavior. Being able to measure user behavior, however, is an important prerequisite for the identification of knowledge management-related roles in the context of ESN.

Most of usability evaluation methods for the Web domain have several limitations such as: the concept of usability is only partially supported; usability evaluations are mainly performed when the Web application has been completed; there is a lack of guidelines on how to properly integrate usability into Web development. This paper addresses these issues through the presentation of a Web Usability Evaluation Process (WUEP) for integrating usability evaluations at different stages of model-driven Web development processes. A case study was performed in order to analyze the feasibility of the approach by applying WUEP to evaluate the usability of a real Web application that was developed by using a specific model-driven development process in the industrial domain. The results suggest that WUEP provides good insights into the performance of usability evaluations. The usability problems are corrected at the model and model-transformation levels, thereby improving the usability of the final Web application.KeywordsWeb UsabilityEvaluation ProcessWeb MetricsModel-Driven Web Development

Increased use of Twitter by a large number of users has resulted in an unprecedented growth in the generation of user created data. This data is now being analyzed in various areas of research including that of user sentiment and behavior analysis during general elections. Rise in the use of Twitter for electoral campaigning has increased interest of researchers in this area. However while studies have utilized factors such as sentiment and frequency of tweets to evaluate candidate favorability, little research has been conducted in analyzing Twitter location data. This paper makes two major contributions. First, it studies the subjectivity and polarity of Twitter data based on the location of the tweets. Findings from this research will enable further fine grained analysis of Twitter data by using the location variable, allowing for more accurate results of user sentiment and behavior analysis. Second, we present a web-based system which can enable in the collection of Twitter data, extraction of the location, perform data analysis and then to plot them on a maps for visualization. We employ US presidential election data of 2016 as a case study to test our research questions and to demonstrate our results utilizing this web-based system.