Heuristic Evaluation and Usability Testing of a Computerized Patient-Reported Outcomes Survey for Headache Sufferers

Comparison of two heuristic evaluation methods for evaluating the usability of health information systems

A number of conventional methods are available for usability testing of smartphones, but their effectiveness is yet to be established. The objective of this study was to compare the effectiveness of the usability testing methods in identifying the problems of a smartphone design. Three usability methods, Think-Aloud (TA), Cognitive Walkthrough (CW), and Heuristic Evaluation (HE), were selected for comparison, and 15 smartphone users and 16 usability experts were chosen as participants in the comparative study. The results of this study indicate that HE method is the most effective one and has the highest severity rating. Additionally, this study finds that eight evaluators are sufficient for using HE method to find most of the usability problems. There has been substantial increase recently in smartphone sales that trigger a tough competition among the smartphone manufacturers. The usability of the smartphone has become among most smartphone users consider before buying their phones. This study proposed a guideline for usability testing of smartphones.

Recent research comparing usability assessment methods has been interpreted by some to imply that usability heuristic evaluation is no longer necessary because other techniques such as usability testing can find some usability problems more easily and cost-effective than heuristic evaluation. Usability testing is a technique used in user-centered interaction design to evaluate a product by testing it on users. Student portal service provides users with channels of information and resources needed for their studies this includes access to library service, course information, and materials. However, there are challenges that come with this development such as lack of effectiveness, efficiency, accuracy, and performance of the student portal service. This article provides information on usability assessment and testing on the student portal of University Utara Malaysia. The study was conducted using a l7 item questionnaire based on efficiency, effectiveness, performance, and accuracy of the student portal service. The result shows that more than 70% of the users are satisfied with student portal service and also agree to its efficiency and performance. This article will impact the designers and system developers by informing them about the views of a user on a performance of the portal and also give them the window for improvement.

The complexity of an electronic medication administration record (eMAR) has been underestimated by most designers in the past. Usability issues, such as poorly designed user application flow in eMAR, are therefore of vital importance, since they can have a negative impact on nursing activities and result in poor outcomes. The purpose of this study was to evaluate the usability of an eMAR application during its development. A usability evaluation was conducted during the development of the eMAR application. Two usability methods were used: a heuristic evaluation complemented by usability testing. Each eMAR application version provided by the vendor was evaluated by 2 hospital pharmacists, who applied the heuristic method. They reviewed the eMAR tasks, detected usability problems and their heuristic violations, and rated the severity of the usability problems. Usability testing was used to assess the final application version by observing how 3 nurses interacted with the application. Thirty-four versions were assessed before the eMAR application was considered usable. During the heuristic evaluation, the usability problems decreased from 46 unique usability problems in version 1 (V1) to 9 in version 34 (V34). In V1, usability problems were categorized into 154 heuristic violations, which decreased to 27 in V34. The average severity rating also decreased from major usability problem (2.96) to no problem (0.23). During usability testing, the 3 nurses did not encounter new usability problems. A thorough heuristic evaluation is a good method for obtaining a usable eMAR application. This evaluation points key areas for improvement and decreases usability problems and their severity.

Nowadays, e-commerce websites have become part of people’s daily lives; therefore, it has become necessary to seek help in assessing and improving the usability of the services of e-commerce websites. Essentially, usability studies offer significant information about users’ assessment and perceptions of satisfaction, effectiveness, and efficiency of online services. This research investigated the usability of two e-commerce web-sites in Saudi Arabia and compared the effectiveness of different behavioral measurement techniques, such as heuristic evaluation, usability testing, and eye-tracking. In particular, this research selected the Extra and Jarir e-commerce websites in Saudi Arabia based on a combined approach of criteria and ranking. This research followed an experimental approach in which both qualitative and quantitative approaches were employed to collect and analyze the data. Each of the behavioral measurement techniques identified usability issues ranging from cosmetic to catastrophic issues. It is worth mentioning that the heuristic evaluation by experts provided both the majority of the issues and identified the most severe usability issues compared to the number of issues identified by both usability testing and eye-tracking combined. Usability testing provided fewer problems, most of which had already been identified by the experts. Eye-tracking provided critical information regarding the page design and element placements and revealed certain user behavior patterns that indicated certain usability problems. Overall, the research findings appeared useful to user experience (UX) and user interface (UI) designers to consider the provided recommendations to enhance the usability of e-commerce websites.

National library websites are essential vehicles for the general public to access national information and resources. The purpose of this study is to examine the usability, user experience (UX), and information architecture (IA) of a sample of national library websites. Through a three-phased multi-methods investigation featuring (1) a content inventory of 28 websites, (2) a web IA and usability evaluation of five websites, and (3) usability tests for three websites, the study results revealed statistically significant gender differences in participants’ task performance. Significant correlations were found between participants’ satisfaction and the content, visual design, and IA, as well as between heuristic evaluation scores and participants’ ratings. Several factors affected participants’ satisfaction: quality of information, the trustworthiness of the content, the perceived credibility of the content producers/providers, the website’s overall structure, and the visual design. The findings of the research also helped to identify common content elements on website homepages and to present a practical procedure for assessing national library websites.

BackgroundClinical trials are the primary mechanism for advancing clinical care and evidenced-based practice, yet challenges with the recruitment of participants for such trials are widely recognized as a major barrier to these types of studies. Data warehouses (DW) store large amounts of heterogenous clinical data that can be used to enhance recruitment practices, but multiple challenges exist when using a data warehouse for such activities, due to the manner of collection, management, integration, analysis, and dissemination of the data. A critical step in leveraging the DW for recruitment purposes is being able to match trial eligibility criteria to discrete and semi-structured data types in the data warehouse, though trial eligibility criteria tend to be written without concern for their computability. We present the multi-modal evaluation of a web-based tool that can be used for pre-screening patients for clinical trial eligibility and assess the ability of this tool to be practically used for clinical research pre-screening and recruitment.MethodsThe study used a validation study, usability testing, and a heuristic evaluation to evaluate and characterize the operational characteristics of the software as well as human factors affecting its use.ResultsClinical trials from the Division of Cardiology and the Department of Family Medicine were used for this multi-modal evaluation, which included a validation study, usability study, and a heuristic evaluation. From the results of the validation study, the software demonstrated a positive predictive value (PPV) of 54.12% and 0.7%, respectively, and a negative predictive value (NPV) of 73.3% and 87.5%, respectively, for two types of clinical trials. Heuristic principles concerning error prevention and documentation were characterized as the major usability issues during the heuristic evaluation.ConclusionsThis software is intended to provide an initial list of eligible patients to a clinical study coordinators, which provides a starting point for further eligibility screening by the coordinator. Because this software has a high “rule in” ability, meaning that it is able to remove patients who are not eligible for the study, the use of an automated tool built to leverage an existing enterprise DW can be beneficial to determining eligibility and facilitating clinical trial recruitment through pre-screening. While the results of this study are promising, further refinement and study of this and related approaches to automated eligibility screening, including comparison to other approaches and stakeholder perceptions, are needed and future studies are planned to address these needs.

Heuristic evaluation is an informal method used to analyze the presentation of an interface design to several evaluators. Heuristic evaluation is done by conducting several interface tests and analyzing the interpretation of the interface. English Kids Talk application is an English learning application for elementary school children. Based on the test results on the initial version of the English Kids Talk Application, user interface improvements are essential. English Kids Talk (Ekids) application development of the Ekids application still does not pay attention to UX. However, User Experience improvements are also necessary, minimizing disruption to the comfort and loyalty of users English Kids Talk application. In this paper, we propose prototype recommendations with a low evaluation value that improving User Experience. A prototype recommendation will be generated that suits the user's needs. The result is usability testing and an interaction cost between the initial version of the English Kids Talk Application and the new version. The result shows higher interaction costs from the new English Kids Talk application than the old one. Usability testing of the new English Kids Talk application is better than the old one.

Introduction: A quality criterion for evaluating information systems (ISs) is their usability, which can be determined by examining the user interface. User interface issues may generate user errors, dissatisfaction, and inefficiency, making IS adoption difficult. Therefore, the International Standard Organization recommends usability testing to evaluate IS acceptability and deployment. So, this study aimed to develop a native model for the usability testing of hospital information systems (HISs) via heuristic evaluation (HE). Method: This multi -staged qualitative study was conducted in 2020; ScienceDirect, PubMed, Web of Science, Scopus, Civilica, and Magiran databases were searched. After meeting the inclusion criteria, the articles were scored. Then, the initial model was created in the following stage by extracting data from library sources and validated using the Delphi technique, and consequently, the final model was presented. Results: : The native model was developed using thirteen principles: 1) visibility of system status, 2) the match between the system and the real world, 3) users’ control and freedom, 4) consistency and standards, 5) error prevention, 6) recognition rather than recall, 7) flexibility and efficiency of use, 8) aesthetic and minimalist design, 9) helping users recognize, diagnose, and recover errors, 10) documentation and help, 11) skill , 12) pleasurable, respectful interaction with users, and 13) privacy protection. Conclusion: This study demonstrated that localizing the usability evaluation of HISs based on trustworthy sources and HE is a wise strategy, because not only it avoids repetitive processes, but also it uses the best up -to -date resources to enable the presentation of the highest evaluation quality.

Combined Expert and User-Driven Usability Assessment of Trauma Decision Support Systems Improves User-Centered Design

Recent research comparing usability assessment methods has been interpreted by some to imply that usability testing is no longer necessary, because other techniques, such as heuristic evaluation, can find some usability problems more cost-effectively. Such an interpretation grossly overstates the actual results of the studies. In this article, we, as authors of studies that compared inspection methods to usability testing, point out the rather severe limitations to using inspection methods as a substitute for usability testing and argue for a more balanced repertoire of usability assessment techniques.

Usability in a medical technology context assessment of methods for usability evaluation of medical equipment

The growth of the Internet and related technologies has enabled the development of a new breed of dynamic websites that is growing rapidly in use and that has had a huge impact on many businesses. One type of websites that have been widely spread and are being widely adopted is the educational websites. There are many forms of educational websites, such as free online websites and Web-based server software. This creates challenges regarding their continuing evaluation and monitoring in order to measure their efficiency and effectiveness, to assess user satisfaction and, ultimately, to improve their quality. The lack of an adaptive usability checklist for improvement of the usability assessment process for educational systems represents a missing piece in ‘usability testing’. This paper presents an adaptive Domain-Specific Inspection (DSI) checklist as a tool for evaluating the usability of educational systems. The results show that the adaptive educational usability checklist helped evaluators to facilitate the evaluation process. It also provides an opportunity for website owners to choose the usability area(s) that they think need to be evaluated. Moreover, this method was more efficient and effective than user testing (UT) and heuristics evaluation (HE) methods.

Assessing health-related quality of life (HRQoL) via Computerized Adaptive Tests (CAT) provides greater measurement precision coupled with a lower test burden compared to conventional tests. Currently, there are no European pediatric HRQoL CATs available. This manuscript aims at describing the development of a HRQoL CAT for children and adolescents: the Kids-CAT, which was developed based on the established KIDSCREEN-27 HRQoL domain structure. The Kids-CAT was developed combining classical test theory and item response theory methods and using large archival data of European KIDSCREEN norm studies (n = 10,577-19,580). Methods were applied in line with the US PROMIS project. Item bank development included the investigation of unidimensionality, local independence, exploration of Differential Item Functioning (DIF), evaluation of Item Response Curves (IRCs), estimation and norming of item parameters as well as first CAT simulations. The Kids-CAT was successfully built covering five item banks (with 26-46 items each) to measure physical well-being, psychological well-being, parent relations, social support and peers, and school well-being. The Kids-CAT item banks proved excellent psychometric properties: high content validity, unidimensionality, local independence, low DIF, and model conform IRCs. In CAT simulations, seven items were needed to achieve a measurement precision between .8 and .9 (reliability). It has a child-friendly design, is easy accessible online and gives immediate feedback reports of scores. The Kids-CAT has the potential to advance pediatric HRQoL measurement by making it less burdensome and enhancing the patient-doctor communication.

The importance of evaluating the usability of e-commerce websites is well recognised. User testing and heuristic evaluation methods are commonly used to evaluate the usability of such sites, but just how effective are these for identifying specific problems? This article describes an evaluation of these methods by comparing the number, severity and type of usability problems identified by each one. The cost of employing these methods is also considered. The findings highlight the number and severity level of 44 specific usability problem areas which were uniquely identified by either user testing or heuristic evaluation methods, common problems that were identified by both methods, and problems that were missed by each method. The results show that user testing uniquely identified major problems related to four specific areas and minor problems related to one area. Conversely, the heuristic evaluation uniquely identified minor problems in eight specific areas and major problems in three areas.